The most dangerous AI output is the polished answer nobody checks.
In December 2025, a federal magistrate in Oregon imposed what became a combined $110,204 in sanctions, fees and costs on two attorneys, the largest reported monetary consequence in an AI-hallucination case, after the legal team filed three summary judgment briefs. The briefs reached the court without adequate verification of whether the cited authorities existed.
That pattern links many of the AI failures that reach courtrooms, client files and regulators. As AI takes over the execution layer, the decisions that direct that execution carry more weight, not less. One bad judgment call and the system acts on it at scale. The value of good judgment has not gone down. It has been multiplied by how fast and how broadly AI acts on it.
Why AI multiplies judgment
When a person does a task by hand, a bad decision affects one output. When AI executes at scale, that same bad decision propagates across hundreds of outputs before anyone notices. The person who reviews an AI output before it ships is making a high-stakes call, often without realizing it.
Far less attention goes to this than to the tool layer: which model, which platform, which workflow. Too few organizations are building for the judgment layer: who reviews what, by what standard, and with what accountability when something goes wrong.
What it looks like when judgment fails: Legal
Couvrette v. Wisnovsky, an Oregon dispute over a family-owned winery, shows how far this can go. Over three briefs filed across five months, the plaintiffs' lawyer submitted fifteen AI-generated fake case citations and eight fabricated quotations. The court sanctioned the lead attorney $15,500 directly, then added $94,704.38 in shifted fees and costs. Combined, the case cost the legal team $110,204, more than twenty times the $5,000 fine that started this pattern back in 2023 with Mata v. Avianca.
The citations looked real. The formatting was correct. The language was persuasive. The only thing missing was a human who knew enough to ask whether the cases actually existed.
This is not an isolated incident. In a separate Colorado federal case, two attorneys representing Mike Lindell were fined $3,000 each after filing a brief containing nearly 30 defective citations. In an unrelated disciplinary matter, Colorado lawyer Zachariah Crabill received a one-year-and-one-day suspension, with 90 days to be served, after filing ChatGPT-generated fictitious authorities without checking them. Charlotin's database had catalogued 719 cases by January 2026. By April, HAQQ reported 1,313 court proceedings involving AI-hallucinated content, 496 of them involving licensed attorneys. ABA Formal Opinion 512 is explicit: lawyers who use generative AI keep their duties of competence and candor, and the obligation to verify their own work stays with the lawyer. The model does not share your professional liability. The person who hits send does.
What it looks like when judgment fails: Financial services
The failure mode in wealth management is subtler, and harder to catch, which makes it more dangerous.
40% of investment adviser firms have implemented AI tools internally, but 44% of those firms have no formal testing or validation of their outputs. Of every 100 advisory firms, roughly 40 use AI internally. Approximately 18 of the original 100 both use AI and lack formal validation.
The SEC has told firms what it expects. Its fiscal year 2026 examination priorities direct examiners to assess whether firms have adequate policies to monitor and supervise their use of AI, and whether their representations about their AI capabilities are accurate.
The specific risk here is output that is plausible and wrong, which is harder to catch than a fabricated citation. Picture an AI-generated client summary that looks authoritative, uses the right terminology, and gets the broad strokes right, while treating a client's actual tax situation as the default case or missing a constraint mentioned earlier in the file. A 2024 study in the Journal of Risk and Financial Management found that AI financial recommendations tend to be generic and often overlook information specific to a client's situation. The advisor who accepts an AI-generated summary without checking it against the client's file is doing what the interface invites them to do.
The three questions that matter
Before acting on any AI output, the person responsible should ask three questions. First: what is the highest-stakes element in this output, and have I verified it independently? In legal, that is every citation. In finance, that is every number touching a client's actual position. Second: does this output reflect the specific facts of this case, or is it applying a general pattern that does not quite fit? Third: if this output is wrong, what does it cost, and who is accountable?
Most organizations skip the third question. The answer to it tells you how much scrutiny the output actually deserves.
Building judgment into the workflow
Treating judgment as a personal trait, something an employee either has or does not, misses half the problem. It is also a workflow design question.
A well-designed system defines where a human has to make a call, what information they need to make it well, and what happens when they flag something. It does not leave the review moment to chance, habit, or how tired someone is on a Thursday afternoon.
In the workflows we build, intervention rate and cost per outcome are the two metrics we track. How often does a human have to fix something before it ships? What does that intervention cost in time and money? Those numbers tell you whether your judgment layer is working. Tokens consumed tell you nothing about either.
The firms that navigate AI well over the next five to ten years will not be the ones with the most capable models. They will be the ones that worked out where human judgment belongs in the process, and built their systems around that answer.
Sources
- The sanction itself, the fifteen nonexistent case citations and eight fabricated quotations across three summary judgment briefs filed over five months, and Magistrate Judge Mark D. Clarke's December 12, 2025 order: Couvrette v. Wisnovsky, No. 1:21-cv-00157-CL (D. Or.), reported by the ABA Journal and analysed in detail by the Washington State Bar Association.
- The figure reached $110,204.38 across two orders rather than one. The December 2025 order set the monetary sanction at $500 per nonexistent case and $1,000 per fabricated quotation; the fee award followed in a separate order in March 2026.
- Running counts of AI hallucination sanctions across US courts: Damien Charlotin, AI Hallucination Cases Database, accessed August 2026, and HAQQ Legal AI, "AI Hallucinations in Law: 1,313 Court Cases and Counting," May 2026. Both are trackers rather than primary sources, and their totals move weekly.
- An earlier and much smaller Colorado sanction, for context on how fast the penalties escalated: People v. Crabill, No. 23PDJ067 (Colo. O.P.D.J., November 22, 2023).
- The professional duty to verify AI output rather than rely on it: ABA Formal Opinion 512, July 2024.
- Regulatory attention to AI use in financial services: SEC Division of Examinations, Fiscal Year 2026 Examination Priorities, November 17, 2025, and ncontracts, "AI Compliance for Firms and RIAs in 2026," March 2026.
- Research on AI use in financial decision contexts: Schlosky, Karadas and Raskie, "ChatGPT, Help! I Am in Financial Trouble," Journal of Risk and Financial Management 17(6), 241 (2024).
- Sanction totals and tracker counts reflect what was published at the time of writing. Both continue to rise.
A note on images across this site. Illustrations and workflow diagrams are made with AI, from prompts we write and refine, and we edit most of them afterwards. Screenshots taken in n8n are not, since they show workflows we built in the tool.