FDA Weighs a Competency Test for AI Medical Devices

FDA does not yet have a settled answer for how to regulate a medical device whose behavior is generated, not programmed. On 18 August 2026 the agency's Digital Health Center of Excellence published a discussion paper on considerations for regulating generative AI (GenAI)-enabled medical devices — not a rule, not draft guidance, but an open question posed to the field: how do you evaluate a device that can produce a different, plausible-sounding answer every time you ask it the same thing?

Why a discussion paper, and not a rule

FDA has cleared AI-enabled devices under its existing framework for years — mostly locked, single-function algorithms whose behavior is fixed at clearance and monitored for drift afterward. Generative AI breaks that assumption at the root: a large language model or a diagnostic assistant built on one can generate open-ended text, and the same prompt can yield different phrasing, different emphasis, or a different recommendation on separate runs. CDRH's paper says plainly that it is not proposing or implementing policy changes, and is not solidifying regulatory expectations. It is soliciting the kind of technical and clinical detail — from device makers, clinicians, researchers, and the public — that a workable framework would need before FDA commits to one.

The two-axis risk framework

The paper's starting point for sorting GenAI devices by risk is a grid, not a single tier. One axis is the type of activity the device performs — summarizing a chart note reads very differently from recommending a dosage change. The other axis is the severity of the consequence if the device's output is wrong. A device that sits high on both axes — consequential activity, severe potential harm — would be expected to face the deepest scrutiny; a low-consequence informational tool would not need the same bar. The framework is meant to replace a single up-or-down risk call with something that can flex across the wide range of things a generative model might be asked to do inside a clinical workflow.

A competency test, not just a technical audit

• Non-clinical device benchmarking: testing the model against a curated set of cases and questions before it ever sees a patient, the way a technical exam checks knowledge before licensure.
• Clinical confirmation: validating that the benchmarked performance holds up in real clinical use, not just against a static test set.
• Foundation models: general-purpose models fine-tuned or wrapped into a specific device raise separate questions about where the manufacturer's responsibility for the underlying model's behavior begins and ends.
• Agentic AI: systems that take multi-step actions on their own — not just generating text but acting on it — are flagged as needing their own risk considerations, distinct from a model that only produces output for a human to review.

What is not decided

Nothing here binds FDA or industry yet. The agency has posed targeted questions inside the paper rather than positions to ratify, and it has said the document does not solidify regulatory expectations. The comment window runs to 19 October 2026, after which CDRH is expected to use the feedback to inform whatever formal proposal — guidance, or eventually rulemaking — follows. For a device class already moving faster than FDA's traditional locked-algorithm framework can track, the paper is the agency's attempt to get ahead of the next hard question rather than answer it in public first.