
Your AI Can Write the Quality Document. It Can't Sign It.
Can you use AI in GxP documentation? Yes, to draft. The FDA's first detailed action on AI in quality systems did not fault the model for writing the SOP. It faulted the humans who never read it, under a rule written decades before AI agents existed. What that means for the documents your team is drafting now.
On April 2, 2026, the FDA sent a warning letter to a small drug manufacturer in Livonia, Michigan. Most of it reads like a routine CGMP action: no microbial testing, no identity testing on incoming components, insects in the facility. Then it reaches the question of how the company wrote its paperwork. The firm had used an AI agent to generate its specifications, its procedures, and its master production and control records, and nobody checked the output. When investigators noted that the company had never performed process validation before shipping product, the firm's reply went into the public record: "You replied that you were not aware of the legal requirement, as the AI agent you used (b)(4), never told you it was required."
So, plainly, can you use AI in GxP documentation? Yes, to draft. What you cannot do is let it author. A qualified human still has to read the output against the requirement, verify it and sign it, and the FDA holds that human, and the quality unit behind them, accountable for every word. Drafting is delegable. Accountability never was.
That (b)(4) is an FDA redaction, the trade-secret marker, sitting almost certainly where the vendor's name would be.
If your team has used an LLM to draft a validation protocol, a risk assessment, a design history file entry or a supplier questionnaire response, and if we are being honest most teams now have, then set aside the insects and read that sentence again. It is the clearest thing a regulator has written yet about what happens when your compliance evidence gets produced by something that does not know it is accountable.
FDA didn't need a new rule. That's the part to notice.
The instinct is to file this under "small firm, sloppy shop, not us." What the agency actually cited should give you pause.
It didn't reach for an AI regulation. It cited 21 CFR 211.22(c), the rule that makes the quality unit responsible for approving procedures and specifications — written decades before anyone had an AI agent to blame. The agency's language, as reported by the trade press covering the letter, is about as direct as regulators get:
"If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP."
Notice what that permits. The FDA did not say don't use AI. It didn't ask for a model card, a validation package for the LLM, or an AI governance committee. Drafting with AI is fine. The obligation that survives is the one that was always there: a human quality unit reviews the document and owns what it says.
It is worth being precise about the novelty here, because the internet is already overstating it. This was not the FDA's first warning letter to mention AI. Exer Labs got one in February 2025 over an AI algorithm marketed beyond its 510(k) exemption, and NutraIngredients counts Purolea as the fourth letter to reference the technology. What compliance consultant Asa Waldstein, quoted in that piece, calls it first for is the part that matters to you: it "is the first warning letter to go into detail about best practices and the potential flaws when using this technology in quality systems."
That puts it in a different category from every AI enforcement action before it. The earlier ones were about AI as a product — the thing you sell, the thing that gets cleared, the thing you are afraid to improve after clearance. This one is about AI as the tool you used to produce your own regulatory evidence, and almost nobody has a policy for that. An inspector is now equipped to ask about it too, since the agency's AI-informed one-day inspection pilot rewards teams whose evidence surfaces fast. That checklist is a subject of its own: FDA AI inspection readiness.
The rulebook you're waiting for wont arrive first
The reasonable objection is that guidance is coming and you will build the controls when it lands. Look at where things actually stand.
The FDA's draft guidance on using AI to support regulatory decision-making — the one with the risk-based credibility framework that explicitly reaches manufacturing — was issued in January 2025. Comments closed that April, and it is still draft. The EU's GMP Annex 22 on artificial intelligence went out for consultation in July 2025, closed in October 2025, and also remains draft; as written, it covers static, deterministic models while largely leaving LLMs out of critical GMP applications. That is a strange place for a regulator to stand while firms are drafting batch records with chatbots. The EMA and FDA published ten joint guiding principles for AI in drug development on January 14, 2026, and made a point of calling them non-binding. The MHRA's sandbox is for AI-as-device. ICH has nothing on point.
And the EU AI Act, which everyone budgeted around, just moved. The Council gave final green light to the Digital Omnibus on June 29, 2026, pushing high-risk obligations to December 2, 2027 for standalone systems and August 2, 2028 for AI embedded in regulated products like medical devices and vehicles. Read that as a reprieve and you walk into the trap. While the AI-specific rulebook slips 12 to 16 months, the general rulebook never stopped applying. FDA proved it with a decades-old citation and no new legal theory. Nor did the sector-specific rulebook pause: the EU Cyber Resilience Act's reporting obligation lands on 11 September 2026, on schedule, giving anyone who ships a connected product 24 hours to file an early warning. Your quality unit regulation, your ISO 13485 document control, your ISO 26262 work-product reviews, your model documentation obligations: none of those were waiting for an AI annex.
The vacuum isn't protection. It just means the first enforcement lands under a rule you already agreed to follow.
Your evidence has an author now
This series has circled one argument: the model is roughly 15% of the work, and the other 85% ( the integration, the monitoring, the evidence layer) is what actually ships. Every post in it asked a version of the same question: can you prove your AI is safe? The eval harness, the audit trail, the traceability matrix, the observability stack that doubles as evidence — all of it exists to produce documents a regulator will accept.
This warning letter asks the question sitting underneath that one, and we had not asked it before. Who wrote the proof?
Back in May 2025 we framed it as design discipline: you are not just building a product, you are building an evidentiary trail. It reads differently now. An evidentiary trail has an author, and the author is the thing being attested to. When an AI drafts the SOP, the validation summary, or the hazard analysis, the trail is still there; the attestation behind it may be hollow.
What makes this a systems problem rather than a matter of discipline is the asymmetry underneath it. AI drove the cost of producing regulated documentation close to zero. It did nothing to the cost of reviewing that documentation, which is what it always was: a qualified human, reading carefully, against the requirement. So the ratio inverts. A quality unit that once reviewed roughly what it could write now faces a queue it could never read, and every incentive in the system nudges the review toward becoming a signature. A record the quality unit never actually read is just a draft with a signature on it.
This is not a competence gap at small firms, either. In June 2025 FDA's own Chief AI Officer told RAPS that the agency's internal tool Elsa, pointed at a document library, "can't hallucinate." Five weeks later CNN reported FDA staff saying it confidently fabricated studies that did not exist. The regulator writing letters about AI overreliance was living the same failure mode at the same moment. That is not a gotcha, it is the point. Confident, well-formatted and wrong is the default output, and it is precisely the output that survives a tired reviewer at 6pm.
One honest caveat, because we would rather flag what isn't known than invent a number. We went looking for a rigorous survey isolating how widely regulated teams draft SOPs, validation documents, and submissions with generative AI, and we could not find one. There is plenty of adjacent research on AI adoption and on quality-system engagement, but nothing that isolates this question. The silence is its own finding: this is happening at a scale nobody has measured, in a category nobody has named, under rules that already apply.
Try this at your next quality review
None of this needs a governance program. It needs an afternoon and an uncomfortable question.
- Ask which of these documents had an AI in the loop. The likely answer is that nobody knows, because nothing captures it, and that is finding number one — you cannot review what you cannot identify. Then write down, on one page, where AI may draft and where a human must author. Drafting a protocol from a template, fine. Generating the acceptance criteria that protocol tests against, not fine. The line is yours to draw, but an undrawn line defaults to "anywhere."
- Make review produce evidence rather than a checkbox: who reviewed it, against what source, when, and what changed as a result. If your document control cannot answer that for an AI-drafted record, you have the Purolea problem with better facilities.
- Plant an error and see whether review catches it. Put a known-wrong acceptance criterion into a draft and route it normally. If it comes back approved, your review is theater, and you found out this week instead of during an inspection.
- Name the accountable human per document class. Not a committee. The quality unit's responsibility has never been delegable, and it is certainly not delegable to a vendor's model whose name your regulator redacts as someone else's trade secret.
Most teams adopted AI for documentation precisely because documentation was the bottleneck, and that bottleneck was load-bearing. It was the step where a human had to understand the thing well enough to write it down. Remove it and you don't get a faster quality system; you get the same quality system with an unreviewed layer between you and the regulator.
That layer is the work we do. A First Build runs about a month, FDA-regulated work included, spent on the machinery that makes AI-assisted evidence hold up: where the model is allowed to draft, what review has to produce, how the trail stays attributable, and what your quality unit is actually signing when the draft came from a model. If AI is already writing documents you would have to defend to an inspector, don't wait for the policy that settles it. Scope a First Build while the line is still yours to draw, rather than something an inspector defines for you in a letter that quotes your own records back at you.
Related Insights

Insight
Moving Clinical AI from a Data Export to a Live EHR
Clinical AI models that look finished on a de-identified export routinely stall against a live EHR. The 2026 FHIR mandates standardize the pipe, not the pipeline: what still breaks is terminology mapping, stale reads, and provenance, and all three are engineering problems.
Read MoreInsight
Engineering Medical Devices from Silicon to Clinical Workflow
Moving telemetry from a medical device into a clinical workflow requires more than working hardware. This guide details the 6-layer architecture, from firmware and BLE gateways to FDA 524B compliance and RPM billing, needed to ship connected medtech into live clinical environments.
Read More
Insight
24 Hours to Report an Exploit Your Model Can't Log
Learn how to meet the 24-hour reporting rules for embedded AI. Master edge provenance, model reconstruction, and CSIRT notifications.
Read MoreFacing a problem like this one?
Tell us what the business needs to win and who owns it on your side. If it is a fit, a First Build starts within weeks. If it is not, we will tell you why.