Why "Chatting With Your Database" Breaks in Production
Letting an LLM query your database looks harmless in demos. In production, it quietly breaks the assumptions databases were built on.
Over the past year, "chat with your database" has become one of the most common LLM demos. Type a question in natural language, watch SQL appear, and get an answer instantly.
It feels powerful. It sells well. And it works — right up until it touches real production data.
The Assumption That Breaks Everything
Databases are designed around a core assumption: queries are written intentionally by humans.
LLMs don't behave like humans. They explore. They infer. They combine information in ways that satisfy the prompt, not the security model.
Read-only access does not mean safe access when the actor is a language model.
Failure Mode #1: Aggregates Leak Data
Aggregates are often considered safe because they don't return raw values. That assumption fails quickly.
SELECT COUNT(*) FROM users WHERE email = 'user@example.com';This query returns a number — not an email address. But it answers a sensitive question directly.
Small-group aggregates, filtered counts, and repeated queries can all be used to reconstruct restricted data.
Failure Mode #2: Joins Create Inference Paths
Joins are where things get subtle.
A model doesn't need direct access to a sensitive column if it can join against something correlated.
Over time, multiple "harmless" queries combine into a clear picture of data you never intended to expose.
Failure Mode #3: Read-Only Isn't Read-Safe
Read-only database users prevent writes. They do not prevent:
- Bulk data exfiltration
- Targeted inference queries
- Exploratory probing
- Reconstruction via repeated queries
From the database's perspective, everything is working as designed. From a security perspective, it isn't.
Failure Mode #4: Logs Without Intent Are Useless
When something goes wrong, database logs tell you what query ran. They don't tell you:
- Who initiated it
- Why it was allowed
- What policy was evaluated
- What safer alternatives were rejected
With LLMs, "what happened" is not enough. You need to know why.
The Missing Layer: Intent Enforcement
Databases enforce syntax. Applications enforce authentication.
But when an LLM is generating queries, no system enforces intent.
That gap is where things break — quietly and without obvious errors.
What We Had to Build
We didn't set out to build a product. We set out to stop an LLM from doing things it technically had permission to do, but absolutely shouldn't.
That meant introducing a layer that:
- Treats the LLM as untrusted
- Understands schema semantics
- Enforces role-aware policies
- Blocks inference paths
- Produces explainable audit logs
The Takeaway
Chatting with your database isn't inherently wrong.
Doing it without enforcement is reckless.
The problem isn't that LLMs generate SQL. It's that SQL was never designed to be generated by something that doesn't understand consequences.
If LLMs are going to touch production data, they need constraints that databases were never built to provide on their own.