Experts Urge Legal Reforms to Stop AI’s Fake Case Citations

Experts identify gaps in AI disclosure rules that allow hallucinated legal citations to persist.

As AI-driven legal research tools gain traction, their tendency to generate "hallucinated" or fabricated citations poses real risks to legal integrity and justice. In 2025, the High Court of England and Wales explicitly warned that generative AI models like ChatGPT can produce coherent but factually flawed legal outputs, cautioning lawyers about relying on these tools without verification (TechCrunch 2025).

A peer-reviewed 2025 study further illuminated this problem, finding hallucination rates between 17% to 33% in legal-specific AI citation generation (arXiv 2025). These inaccuracies have had tangible consequences; for instance, the Iowa Supreme Court Attorney Disciplinary Board charged an attorney in 2025 for submitting fictitious AI-created legal cases during court filings (Axios 2025).

Experts argue that existing AI disclosure obligations do not adequately prevent these falsehoods. Legal scholar Cheng-chi (Kirin) Chang states that attorneys submitting AI-generated work without verification "submit a representation to a court while consciously declining to establish any basis for its accuracy." This suggests a need for stricter diligence and perhaps new legal frameworks to govern AI’s use in law.

Supporting this concern, a 2026 large-scale study examining 2,700 judicial-style responses from 12 large language models found a False Citation Rate exceeding 30%, with widespread hallucinations following the rapid adoption of such models. The study quantified nearly 146,932 hallucinated citations in 2025 alone (arXiv 2026).

To address these issues, researchers have introduced benchmarks like LegalCiteBench to systematically evaluate citation reliability and push for improved AI design and regulation (arXiv 2026).

Ultimately, legal professionals, regulators, and AI vendors must collaborate to strengthen disclosure rules and verification standards. Without these reforms, the risk of misinformation and malpractice stemming from AI-generated hallucinated citations will persist, challenging the justice system’s integrity.