How to ask your business data questions in plain English — and actually trust the answer

A beverage distributor running nine vans out of a single depot had a new ERP with an "AI assistant" chat bubble in the corner, added in the last update. The ops manager typed the question she asked herself every Monday: "which customers are more than 30 days late?" The bot answered instantly, with a clean list of eleven store names and balances. Confident, specific, formatted like a report. She forwarded it to the collections rep before double-checking it against the aging report she normally pulled by hand.

Three of the eleven had paid in full the week before. Two accounts that actually were 40+ days late weren't on the list at all. The bot hadn't lied exactly — it had generated a plausible-looking answer from a plausible-looking guess at how the ledger was structured, and the guess was wrong in ways that were invisible unless you already knew the real numbers. The collections rep called two paid-up customers asking for money they didn't owe. That's the moment the ops manager stopped using the assistant for anything she couldn't verify in thirty seconds, which was most things — which made the feature pointless.

Why "ask your data a question" so often gets the wrong answer

The appeal is obvious: instead of navigating a report builder or waiting for someone in the office to run a query, you type a plain-English question and get an answer immediately. This is what natural language query — asking business data questions in plain English — is supposed to deliver, and vendors are racing to bolt an LLM chat box onto every dashboard and ERP.

The failure mode is specific and well documented in how these systems actually work. Under the hood, most of them translate your question into a database query on the fly. The model doesn't truly know your schema — it pattern-matches against what a query like that usually looks like, then writes SQL that reads as correct. When a business has more than one table that could plausibly hold "customer balance," or a join that isn't obvious from column names, or a status field where "late" isn't a boolean but a derived calculation involving grace periods and partial payments, the model quietly picks a version that runs without erroring and returns a wrong number with total confidence. Research on these systems puts the bulk of failures right there — the model reaching for the wrong table, inventing a filter, or missing a join — not in the language understanding part, which is usually fine. The English is understood correctly. The data behind it is guessed.

For a distributor this isn't an abstract accuracy statistic. It's the exact kind of question that gets asked constantly and matters when it's wrong: which accounts are overdue, how much of a SKU sold last week net of returns, what a route's margin looked like after commission, how much stock is actually on a van right now versus what the manifest says. These aren't generic BI questions with one obvious table behind them — they involve catch-weight variance, route-level ledgers, and money math that has to net to the cent. A model guessing at that structure will get the shape of the answer right and the substance wrong, which is worse than no answer, because a wrong number that looks like a report gets acted on.

What actually makes the answer trustworthy

The fix isn't a smarter model — it's not letting the model guess at the data model in the first place. The assistant has to be built against the same fixed, known layer the rest of the software already uses to generate its real reports, not a fresh interpretation of the raw database on every question. Concretely, that means three things:

It answers from the live workspace, not a stale export. If the assistant is querying a snapshot or a warehouse that syncs nightly, "how much did we sell today" is wrong by design before the model even runs. The answer has to come from the same data the day's dashboard is showing right now.

It's grounded in defined business logic, not inferred schema. "Overdue" should mean exactly what the aging report means by overdue — the same grace period, the same partial-payment handling — because that logic already exists and is already correct. The assistant should be calling into it, not re-deriving it from column names.

It shows its work. A trustworthy answer to "which customers are 30+ days late" comes with the eleven accounts and the balances behind them, in a form you could hand straight to the aging report and cross-check in ten seconds. An answer you can't verify quickly is an answer you'll eventually stop trusting, which is what happened at the depot above.

This is the same discipline that made the ops manager's aging report trustworthy in the first place — a fixed definition of "late," applied consistently, pulled from the current ledger. An AI assistant only earns the right to skip the manual report if it inherits that same discipline instead of reinventing it per question.

What to check before you trust an "ask your data" feature

• Ask it a question you already know the answer to, and check the number against the report you currently trust — before you use it for anything that leaves the building.
• Ask if it queries live data or a periodic sync/export. A nightly snapshot makes "today" and "right now" questions wrong by default.
• Ask whether it reuses the same business logic (aging buckets, commission rules, catch-weight variance) as your existing reports, or re-derives its own interpretation each time.
• Ask for the underlying rows or accounts behind any summary answer — if it can't show its work, don't act on the summary.
• Watch for the exact failure mode above: the wrong table, a missing filter, a plausible number for the wrong period. It's rarely an obviously broken answer; it's a confident, well-formatted one that's quietly off.

iotoms' assistant is built on top of the same live workspace data and reporting logic that drives the 48 standard reports — routes, ledgers, catch-weight variance, and aging all come from one source of truth, so an answer in the chat matches the number you'd get pulling the report by hand.