Agentic AI Poses Fresh Hallucination Risks for Law Firms
Emerging agentic AI systems can autonomously act on hallucinated information, exposing law firms to unique risks.
Why it matters: Law firms adopting agentic AI face novel dangers: these systems can independently execute erroneous actions—like sending misinformation or manipulating data—without human oversight. Understanding and mitigating these risks is critical to avoid costly breaches and reputational harm.
- Agentic AI refers to systems that act with autonomy, performing complex tasks without constant human prompting.
- Erroneous outputs—'hallucinations'—can now trigger real-world actions, not just generate false text.
- A TechRadar survey found 89% of security pros report unauthorized AI use at work, increasing risk exposure.
- Agentic AI has already faked the voice of a CEO for cybercrime, as documented by ACM Europe.
Agentic AI—a term for artificial intelligence systems that can operate without ongoing human instruction—marks a shift from tools like basic chatbots to programs that autonomously execute multi-step tasks.
- These systems can read emails, transfer files, schedule meetings, or make decisions with little or no human involvement, according to the American Bar Association.
- The risk: If the AI "hallucinates"—produces convincing but false information—it can now act directly on those errors, automating tasks like sending fraudulent messages or even changing client data, as detailed by Progress.
- The new risk is concrete: Agentic AI recently faked a CEO's voice to authorize a fraudulent transfer—a technique linked to emerging operational vulnerabilities in law firms, per ACM Europe.
A TechRadar survey of security professionals shows widespread use of unapproved AI tools, suggesting weak oversight in high-risk environments like law. Nearly 90% admitted to using unsanctioned AI solutions, and organizations experiencing shadow AI faced an average breach cost increase of $670,000.
With agentic AI blurring the boundaries between data generation and data action, legal departments must expedite policy development, adopt risk assessments, and require human review points before autonomous tools can execute sensitive actions. As the American Bar Association recommends, "plausibility checks" are now a necessity—not a luxury—inside major law firms.
By the numbers:
- 89% — Security professionals reporting unauthorized workplace AI use (TechRadar survey)
- $670,000 — Average increase in data breach costs for organizations using unapproved AI tools (TechRadar)
Yes, but: No major law firm data breaches have yet been publicly attributed specifically to agentic AI, but cases in finance and corporate settings illustrate the risks.
What's next: Look for updated ABA guidelines on autonomous AI in professional practice later this year.