All articles
Why a General Chatbot Can't Be Trusted With Indian Law (and Grounded AI Can)
Articles4 min read

Why a General Chatbot Can't Be Trusted With Indian Law (and Grounded AI Can)

L

LawWorld

Two AI tools can sit side by side, both fluent, both fast, both confident — and only one of them can be trusted to cite a real Supreme Court judgment. Understanding why is not a technical indulgence. It is the single most important thing an advocate can learn about AI, because it explains the entire hallucination problem and points to the fix.

How a general chatbot "knows" law

A general-purpose model like a consumer chatbot is a prediction engine. During training it absorbed enormous amounts of text and learned the statistical patterns of language. When you ask it for a case on a point of Indian law, it does not look anything up. It generates the sequence of words most likely to follow your question — a plausible case name, a plausible citation, a plausible holding.

Most of the time the general shape is right. But when you ask for something specific — an exact citation, a paragraph number, a precise holding — the model fills the gap with an invention that looks correct. This is not a bug that a future update will quietly fix. It is inherent to how the technology works. That is why courts keep catching fabricated citations, and why the Supreme Court's 2026 Draft AI Regulations had to formally define "AI hallucination" as fictitious or inaccurate legal content, including fabricated precedents and quotations.

25% read

Keep reading — it’s free.

Create a free LawWorld account to finish this piece and unlock the full library of practice guides written for advocates in India.

  • Finish every guide, free
  • Save articles to read later
  • New practice insights weekly

Free forever. No card required.

Comments

Sign in to read the discussion and share your thoughts.