Legal AI Answers: Words May Be Right, But Substantive Errors Persist
Legal AI systems often deliver correct legal terms but still generate substantively wrong answers.
Why it matters: Lawyers and legal tech developers risk mistakes and liability if they rely solely on legal AI output. Understanding these limitations is critical to safe and ethical adoption of AI tools in practice.
- Michael Krallmann, TransLegal CEO, warns that correct legal terminology from AI can mask substantive errors.
- Stanford studies found hallucinations in 69%-88% of legal AI responses, and 17% in dedicated legal AI tools.
- Complex queries, like precedent analysis, increase the odds of AI-generated hallucinations.
- Two Oregon lawyers were fined $110,000 in April 2026 for filing AI-generated fictitious case law.
Legal professionals are grappling with the reliability of AI-generated legal answers. As Michael Krallmann, CEO of TransLegal, points out, "Legal AI can produce the right words and still point to the wrong answer. The closer the terminology appears to be, the easier it is to miss the gap." This issue is acute when dealing with cross-border legal matters, where subtle distinctions carry serious consequences.
- A Stanford University study found that large language models produced hallucinations—misleading or false legal information—between 69% and 88% of the time. This risk grew with query complexity, such as when analyzing relationships between precedents.
- Even more specialized, purpose-built legal AI tools were found to hallucinate 17% of responses, per a follow-up Stanford study.
- Stanford researchers warn, "Legal hallucinations are pervasive and disturbing. These models often lack self-awareness about their errors and tend to reinforce incorrect legal assumptions." As a result, lawyers may need to verify every AI-generated citation or legal assertion, substantially limiting promised efficiency gains.
- The consequences of misplaced trust can be severe. In April 2026, two Oregon lawyers were sanctioned $110,000 for submitting AI-generated filings that cited fictitious cases.
- In United States v. Heppner, using generative AI to prepare legal documents resulted in a waiver of attorney-client privilege, as the tool was not considered an attorney.
Legal teams must weigh these risks and perform rigorous review of AI-generated outputs to maintain professional and ethical standards.
By the numbers:
- 69%-88% — Hallucination rate for LLMs in legal queries (Stanford study)
- 17% — Hallucination rate in purpose-built legal AI tools (Stanford study)
- $110,000 — Fine against Oregon lawyers for submitting AI-generated fictitious cases in April 2026