Courts Don't Ban AI — They Sanction Unverified Output

Every defender who has considered running AI on a case file has the same quiet fear: one hallucinated citation, one fabricated quote, and you're standing in front of an angry judge explaining yourself. That fear is real — but it points at the wrong target. Courts don't punish lawyers for using AI. They punish lawyers for filing AI output nobody verified.

The pattern is now well documented. In February 2026, the Fifth Circuit ordered attorney Heather Hersh to pay a $2,500 sanction after her brief contained 21 fabricated quotations and misrepresentations of law. Judge Jennifer Walker Elrod wrote that pleading ignorance of generative AI risks "is certainly no longer" an excuse, and noted Hersh "misled, evaded, and violated her duties as an officer of this court." Notably, the same court had considered — and declined to adopt — a first-of-its-kind appellate rule regulating generative AI, concluding existing conduct rules were sufficient. AI itself was never the problem.

The numbers behind this are accelerating. As of early 2026, researchers had documented 239 US cases of AI-generated hallucinations in lawyer filings, out of 712 legal decisions worldwide addressing hallucinated content — roughly 90% of them written in 2025 alone. And the penalties are climbing:

• Early sanctions typically ran $1,500–$5,000.
• A District of Oregon lawyer was fined $15,500 after citing fake cases without being "adequately forthcoming, candid, or apologetic."
• An Illinois court imposed a combined $59,500 award against a firm and a partner.
• The Southern District of Texas issued a standing order cautioning attorneys and pro se litigants against using AI without verifying its output.

Judge Marina Garcia Marmolejo of S.D. Texas captured the judicial mood: busy dockets have "scant resources to spare ferreting out erroneous AI citations."

Notice what every one of these cases has in common. The AI was used; the output was filed. The failure was never the tool — it was the missing verifiable source. That's why the ABA's Formal Opinion 512 accepted generative AI under existing ethics rules, but tied it to the competence obligation: lawyers must review AI output for accuracy before it touches a filing.

For criminal-defense work, the standard is practical: every AI-assisted statement needs a checkable source. That's the design principle behind Evidence Copilot — every AI answer carries a citation to the exact second of video or page of the PDF it came from, and every output is marked AI-generated and pending human review. When your work product can be traced to the original bodycam file or police report, verification stops being a separate chore and becomes part of the answer itself.

Courts have told us exactly what they want. Give every claim a source a human can check.

The Ethics Rules You Already Follow Are the Standard

Here's the part most defenders miss: you don't need to learn a new rulebook to use AI ethically, because the rulebook you already follow covers it. The American Bar Association's Formal Opinion 512, issued July 29, 2024, concluded that existing Model Rules — 1.1 (Competence), 1.6 (Confidentiality), 1.4 (Communications), and 1.5 (Fees) — principally apply to generative AI use. No new AI-specific rules were needed, and the ABA's first formal ethics opinion on generative AI makes clear that lawyers must review AI output for accuracy under their competence obligations.

That matters because bar approval is a real adoption barrier. In a survey of defense attorneys, 50% cited bar association approval as a top consideration before adopting AI tools, and 94% reported at least one concern. But the framework is not a moving target. The rules haven't changed — the standard is verification and human review, the same diligence you already owe under Rule 1.1.

Louisiana made that duty explicit. HB 178, effective August 1, 2025, made it the first state with an AI-evidence framework, requiring attorneys to "exercise reasonable diligence to verify the authenticity of evidence before offering it to the court," with contempt and disciplinary consequences for failure. If you practice in Louisiana, the new AI-evidence framework turns what was already good practice into a statutory requirement.

The enforcement side confirms the same principle. Courts are not banning AI — the Fifth Circuit considered a first-of-its-kind appellate rule regulating generative AI and declined to adopt it in 2024, concluding existing lawyer conduct rules were sufficient. What courts punish is unverified output: 239 US cases of AI-hallucinated filings were documented as of February 2026, and sanctions are escalating beyond the early $1,500–$5,000 range toward a $15,500 fine in Oregon and a $59,500 combined award in Illinois.

So the compliance answer is concrete, not abstract:

• Every AI-assisted claim traces to a verifiable source before it leaves your desk.
• AI output is labeled as such and reviewed by a human before filing or offering in court.
• Originals stay immutable, so you can always show the court what the actual evidence says.
• Your workflow produces an audit trail demonstrating the diligence HB 178 and Rule 1.1 demand.

This is the design logic behind Evidence Copilot: every AI answer carries a clickable citation into the exact second of video or page of the PDF it came from, every output is marked AI-generated and pending human review, and the tool never makes legal conclusions. That's not a feature bolted on for marketing — it's the structure that makes AI use defensible under the rules you already follow. If your current stack can't show where a quote came from, the gap isn't the technology. It's the verification layer.

Grounded Citations: How Citation-Compliant AI Review Works

The courts aren't asking whether AI touched your work. They're asking whether you can prove, on the record, where every assertion came from. That's the question a citation-grounded workflow answers before it's ever asked.

The stakes are real. As of February 2026, 239 US cases document AI hallucinations in lawyer filings, and the Fifth Circuit fined attorney Heather Hersh $2,500 after her brief contained 21 fabricated quotations. Judge Jennifer Walker Elrod's opinion made clear that pleading ignorance of generative AI is "certainly no longer" an excuse, and that candor about AI use affects the sanction itself. Judges have scarce docket time to verify citations for you — Judge Marina Garcia Marmolejo of the Southern District of Texas noted courts have "scant resources to spare ferreting out erroneous AI citations."

So the fix is structural, not aspirational: build the verification into the workflow itself. Here's what that looks like in practice on a discovery dump of bodycam, interrogation footage, 911 audio, jail calls, and scanned police reports:

• Timestamped, diarized transcripts with word-level timing, so any quote resolves to the exact second of video — not a paraphrase you have to re-verify by scrubbing footage.
• OCR'd document text, so scanned reports and photo exhibits become searchable, citable text tied to the specific page.
• Auto-signals at ingest — every new file is compared against earlier evidence, surfacing candidate inconsistencies like a police report's account versus the bodycam, before anyone thinks to ask.
• An AI evidence agent where every answer carries a clickable citation into the exact video second or PDF page, so checking the AI's work takes seconds instead of an afternoon.

That last point matters most. In Evidence Copilot, every AI output is marked AI-generated and pending human review, originals stay immutable, and every action is audited. The tool never makes legal conclusions — it shows you the source and leaves the judgment to you. That design maps directly onto the duties already on the books: the ABA's Formal Opinion 512 holds lawyers responsible for reviewing AI output for accuracy under Model Rule 1.1, and Louisiana's HB 178 — the first state AI-evidence framework — requires reasonable diligence to verify authenticity before offering evidence to the court.

Defense attorneys are already working this way. In one survey, 26% reported using AI to identify inconsistencies in witness statements, and 71% have used AI in their practice. The outcomes can be decisive: in one case reported by the North Carolina Advocates for Justice, a judge dismissed a case after AI-assisted review revealed detectives withholding exculpatory evidence.

Contrast that with the free DIY stack — local transcription plus a general-purpose chatbot. It has no citations, no audit trail, no immutable originals, and no review marking. When sanctions are escalating past the early $1,500–$5,000 range into five figures, that gap isn't a convenience issue. It's the difference between work product you can defend and output you have to disavow.

Grounded citations aren't a feature. They're the price of admission the courts have already set.

ctaText: See how citation-grounded review works on your own discovery — book a 20-minute walkthrough at evidencecopilot.com/book, or start on the Solo plan at evidencecopilot.com/signup.

socialProofText: Every AI answer cites the exact source moment it came from — always marked AI-generated and pending human review, with immutable originals and a full audit trail.

Your AI Verification Checklist for Court Filings

The judge reading your brief doesn't care which AI tool you used — only whether you verified what it gave you. With 239 documented US hallucination cases in lawyer filings as of February 2026, verification has become a de facto professional duty.

Start with citations. Never file an AI-suggested case or quote without opening the source yourself and confirming the language, holding, and pinpoint. The Fifth Circuit's $2,500 sanction against attorney Heather Hersh involved 21 fabricated quotations or misrepresentations in a single brief — and Judge Jennifer Walker Elrod wrote that pleading ignorance of generative AI risks "is certainly no longer" an excuse, per Reuters' report on the sanction. Hersh's penalty was likely increased because she misled the court about her AI use; candor would have cost her less.

Next, label everything. Keep AI output marked as "generated — pending human review" until you or a supervising attorney has actually reviewed it. The ABA's Formal Opinion 512 determined that existing Model Rules — including Rule 1.1 competence — principally apply to AI use, and that lawyers must review AI output for accuracy. A label is not a formality; it is the artifact that shows the review happened before filing, not after.

Third, document your diligence. If a court or the bar ever asks how you verified evidence authenticity — Louisiana's HB 178 now requires "reasonable diligence" with contempt and disciplinary consequences for attorneys offering evidence — an audit trail of who reviewed what, and when, is your answer. This is the gap that separates grounded evidence-review workspaces like Evidence Copilot, where every AI answer cites the exact video second or PDF page it came from and originals stay immutable, from ad hoc tool use.

Finally, never let a tool make a legal conclusion. AI can flag that a police report's timeline conflicts with the bodycam audio; it cannot decide whether that conflict supports a suppression argument. Suggested cross-exam questions and inconsistency signals are inputs to your judgment, not substitutes for it.

Contrast this with the free DIY stack some defenders assemble: local Whisper for transcription plus NotebookLM for Q&A. It breaks at case scale — no diarization quality control, no citations, no audit trail — and under current enforcement, unverified output is precisely what courts sanction. Penalties have escalated from the typical early range of $1,500–$5,000 to a $15,500 fine in the District of Oregon and a $59,500 combined award in Illinois, as Bloomberg Law documented. A tool that gives you answers you cannot trace back to a source moment isn't free — it's an unpriced sanction risk.

Your checklist, in short:

• Verify every AI-suggested citation against the source before it enters a brief.
• Keep all AI output labeled as pending human review until an attorney signs off.
• Maintain an audit trail documenting who reviewed what, and when.
• Never let a tool reach a legal conclusion — flag inconsistencies, decide the argument yourself.

Courts have made the rule clear: use AI if you can stand behind every word it produced.

Getting Started Without Betting Your License

The safest way to adopt AI in your practice is not a policy memo — it's a supervised pilot on one real case. Courts punish unverified AI output, so your first workflow should be built around verification from day one. That's what keeps you inside the lines the Fifth Circuit drew when it sanctioned an attorney $2,500 over 21 fabricated citations: use the tool, but check everything.

Start small and concrete. Pick an active case with a manageable discovery dump — a police report, a couple of bodycam files, maybe a 911 call. Upload everything to a grounded workspace like Evidence Copilot, which produces timestamped, diarized transcripts and OCR'd text you can actually cite. Then run the intake signals:

• Let the auto-signals compare the police report against the bodycam audio and flag candidate inconsistencies before you've asked a single question.
• Open the AI agent and ask case-specific questions — every answer arrives with a clickable citation into the exact second of video or page of the PDF.
• Draft cross-examination questions from a review signal; each suggested question is tied to a quoted source, with no credibility conclusions drawn for you.
• Export an exhibit pack — clips, still frames, quoted excerpts, and a citation manifest — so anything you hand to co-counsel or file is traceable to the original.

The human-review layer is not optional decoration. Every AI output in the workspace is marked as AI-generated and pending your review, and the originals stay immutable. That structure maps directly onto your obligations under ABA Formal Opinion 512, which holds that competence requires lawyers to review AI output for accuracy. It also answers the authenticity-diligence standard Louisiana adopted in 2025 — the first state framework requiring attorneys to verify evidence before offering it.

Compare this to the free DIY route. A local Whisper-plus-NotebookLM setup breaks at case scale: no diarization quality control, no citations, no redaction, no audit trail. Given that sanctions have escalated well past the early $1,500–$5,000 range — including a $15,500 fine in Oregon and a $59,500 combined award in Illinois — a workflow with no verification trail is a professional-responsibility risk, not a bargain.

Pricing is published, so you can budget the pilot without a sales call. The Solo plan runs $39/user/month with 15 media-processing hours included; overage is metered and hard-capped at 3x included hours, so a heavy intake month can't surprise you. Every plan includes the full workspace — there are no feature tiers holding the citation and audit features hostage.

If you want to see the workflow on your own discovery before committing, book a 20-minute walkthrough at evidencecopilot.com/book, or start directly on the Solo plan at evidencecopilot.com/signup. Either way, the rule of the pilot is the same one courts are enforcing: the AI drafts, you verify, and every claim traces back to the source.