Partner With Us

← All lessons / AI in practice / Lesson 12 of 12

Interactive lesson

Validating AI in use: is it still working?

Testing before launch isn't enough. Data, processes and models change, and quality drifts. Learn how to validate an AI system in use with regular sample checks, honest uncertainty and clear rules for when to act.

You’ll learn to
  • Inspect monitoring samples and their uncertainty.
  • Plan a response when inputs or measured quality change.

Validation is a cycle, not a sign-off

Lesson 07 covered testing before you rely on a system. Validation in use asks a different question: is it still good enough, for this purpose, today? The data a system receives, or the patterns linking its inputs to outcomes, can change. This is called drift. It may happen gradually or suddenly, for example after a new referral form changes the inputs. Performance may change too, so inspect actual outputs. A model update is another reason to revalidate.

Set the bar

Agree acceptance criteria before launch: for example, at least 90% of summaries need no corrections.

Check samples

Every week, a person reviews a random sample of outputs against the source.

Act on evidence

Agree in advance what happens when results fall below the bar.

Interactive

Spot the drift

An AI tool summarises referrals for a fictional charity. Each week, a reviewer checks a random sample and records how many summaries were correct. In week 12 a new referral form is introduced, and the true quality drops from 92% to 78%. Change the sample size and the alert level, and see which weekly results fall below your chosen line.

Weekly sample resultBelow the alert line95% uncertainty range

An alert means the sample fell below your chosen quality line. It is a reason to investigate, not proof that quality changed. A line above 92% also flags the original baseline as below target; those alerts are not necessarily false alarms. The chart shows the full 0–100% range so uncertainty bars remain visible.

Simulated data. The sample results are generated in your browser from fixed random numbers, so they're the same every time.

Key ideas to take away

  • Define “good enough” before launch. Write acceptance criteria tied to the purpose, and who signs them off.
  • Check real outputs regularly. A small random sample, reviewed against the source, is the core of validation in use.
  • Small samples are noisy. Ten checks a week can trigger alerts even when the underlying rate has not changed, or miss a genuine drop. Size the sample to the risk and interpret the alert against your chosen target.
  • Agree the response in advance. Investigate, add extra checks, roll back or pause. Decide before you need to.
  • Revalidate after any change. A new form, data source, template or model version is a reason to check again.

Check your understanding

4 quick questions. Try as often as you like. Your progress stays on this device; answers clear when you reload.

Want to go deeper?Methods, checklists and standards. Optional.

How uncertain is a weekly sample?

If k of n sampled outputs are correct, the lab shows a 95% Wilson interval around k/n:

centre = (p̂ + z²/2n) / (1 + z²/n)
half-width = z·√( p̂(1−p̂)/n + z²/4n² ) / (1 + z²/n)        p̂ = k/n, z = 1.96

With n = 20 and 18 correct, the true rate could plausibly be anywhere from about 70% to 97%. That's why one bad week is a reason to look closer, not proof of failure.

Borrowing from model risk management

Banks have validated models for decades. Their practices translate well: keep an inventory of models (your AI register), validate independently of the people who built or chose the tool, set monitoring thresholds in advance, and record decisions. The PRA's SS1/23 principles are a clear example.

Beyond accuracy

  • Input drift: are the inputs changing (new form, new language, new topic)?
  • Fairness: is quality lower for some groups or reading levels?
  • User feedback: are staff correcting more outputs than before?

You've finished the path

Put what you've learned to work

You now know how AI works, how to use it responsibly, and how to choose, embed and validate it in real work. That's exactly what healthcare and research teams need help with.

Volunteer with us Partner with us Get updates on new lessons

Continue exploring

We've picked resources that explain these ideas from different angles. They are independent of Code With Impact.

Sources for this lesson

The referral tool, weekly results and drift scenario are simulated and original to Code With Impact. The lesson draws on the frameworks listed under Continue exploring; no text or diagrams from them are reproduced.

How we reference: we link and credit, we never copy other educators' text, diagrams or code. See our full Sources & credits register.