In our benchmark study, WisdomAI reached 93% accuracy with automatic learning alone. This post covers the last mile: how one customer took their own question set from 94% to 100%, and what it takes to keep it there.
Same question, two right answers
A few weeks ago, a customer sent us an email I haven't been able to forget. Someone on their team asked a question in WisdomAI. A little later, someone else (or maybe the same person in a different tab) asked the same question again.
Here’s the question: What was the In-Store Sell-Through for Spain in June 2026?
They got two different numbers: 41.2% and 40.3%. They are only about a percentage point apart, a discrepancy many wouldn’t notice or question. And as we dug into it, we discovered that both numbers were technically correct based on reviewed, approved SQL queries.
When we talk about closing the accuracy gap in AI analytics accuracy, this is what that usually means: resolving ambiguities so that AI can interpret a question with precision and repeatability, delivering the right answer 100% of the time.
In our last post, we discussed how we built the benchmark that tested WisdomAI against Claude Code. Spoiler alert: WisdomAI reached 93% accuracy with automated learning alone. In this post, we’ll show you how we close that last 7%, which I’ll demonstrate by describing what happened after this email.
Editor’s note: I’ve changed the metric, the country, the figures, and the people to protect the customer identity.
Both answers are right, and that’s the problem
If your team uses AI for analytics, you’ve likely encountered a similar situation. Here’s what we found when we looked into it:
WisdomAI hadn't made up a join or invented a column. It did exactly what it had been told to do. The customer’s domain had two reviewed queries — i.e., saved examples that a person has checked and approved — for In-Store Sell-Through.
One left out pet care, because that’s how the category leads calculate it. The other included pet care, because that’s how the country managers calculate it. Each was written by someone with a good reason, and each was correct on its own merit.
These were two correct answers to two different questions with the same name. These types of conflicts have always existed, but in the past, data analysts sat between the question and the answer.
They could deploy judgment to manually solve ambiguity and lock the right definition into a dashboard or report. Today, your AI agent may receive thousands of questions every week. It needs a way to encode that knowledge as enterprise context so it can apply the right judgment at scale.
The Context Development Lifecycle that closes the accuracy gap
You may have heard of the software development lifecycle (SDLC), where code gets written, reviewed, versioned, tested, promoted, monitored, and rolled back when it breaks. Context rarely gets the same treatment, even though it shapes production answers just as directly.
To build a context layer that delivers accurate answers 100% of the time, you need a context development lifecycle (CDLC) — a process to help you centralize context, identify wrong answers, fix the definitions behind them, prove the fixes work, and publish them to everyone at once. Here’s what that process looks like in WisdomAI:
Start with a verifiable evaluation set
To test AI accuracy, you can’t skip this step. Every AI analytics answer must be verifiable, meaning you must be able to check it against the answer your business agrees is correct.
Start with one domain area. Then, write down the questions your business asks the most. WisdomAI will also suggest an evaluation set based on the context that’s been added. Next to each question, document which answer your business agrees is correct — not just the answer itself, but also the logic AI needs to use to get there.
This is your evaluation set. With it, you can run every question, score every answer, and run them all again in a fresh run or on a separate surface. This is the same approach we took in the benchmark study.
The customer in this situation had a 300 question evaluation set. The first run scored 94%, close to the 93% WisdomAI reached in our benchmark with zero human intervention. That meant 18 questions failed, including the Spain sell-through question.
Trace each failure
With this short list of failures, you can track down the cause of each failure and encode the solution. In WisdomAI’s Enterprise Context Layer, you can see all of a domain’s context in one consolidated view, including all of your reviewed queries, definitions, and connected sources.
When we analyzed the sell-through failure, the Context Builder compared the SQL in the two reviewed queries line by line and found the one clause that differed. One query had a filter excluding pet care, and the other didn’t. WisdomAI automatically linked that difference to the failed question. The domain owner never set which definition was the default, so the question itself was ambiguous.
The SQL diff, the one clause that differs, and the question it explains.
Draft the missing context
Next, the context builder proposed a fix in two parts. The first was a knowledge entry stating that sell-through excludes pet care unless the question says otherwise. The second was a rule naming the exception: when a question asks to include pet care, use the country-manager definition.
The Context Builder saved both of these context entries to a private draft, and then reran the failed question five times. It got the same correct answer in every run.
The Knowledge Entry and the rule, on a draft before publication.
Encode the context
This is where the human enters the loop. The domain administrator on the customer’s team read the Knowledge Entry and the rule, checked them against how her company calculates sell-through, and published them.
She made one decision that took a minute, closing the context gap without having to write any SQL. This was the moment the company decided its default definition. From then on, everyone who asks about Spain’s sell-through gets the same number: the one the company agreed to report on.
Repeat this step with every failed query
The other 17 failures were fixed the same way. The Context Builder investigated, drafted a fix, and tested it, then the domain admin reviewed and published the new context in a versioned push. By the end of the same day, the evaluation set scored 100% on all 300 questions.
Without WisdomAI, that same process would have taken weeks of manual work, reading chat logs, and comparing SQL. And you’d still have no easy way to guarantee the deployed changes wouldn’t break the context another question relies on.
Reaching 100% accuracy: Why analytics needs a harness
Both of that customer’s answers were technically correct. Now with the CDLC, everyone gets the answer the company agreed is right. But context alone is not enough. You still need a specialized analytics harness to apply the definitions the company has agreed on, on every surface, and to keep them current as the business changes. That’s the full analytics architecture that WisdomAI provides.
The Enterprise Context Layer and Analytics Harness work in tandem with the model and the human in the loop to deliver the agreed answer on every run and every surface. That’s how WisdomAI delivered 100% AI analytics accuracy for this customer and many others. And it’s the same path you can take, too.
Want to see what 100% accuracy looks like on your enterprise data? Schedule an intro call with our team today.
