AI context fundamentals
What you will learn in this module
The definition of AI context and the context gap
The six types of enterprise context
Why more context is not always better
The catalyst for context
AI is changing how teams get answers from data. Instead of writing SQL or waiting on an analyst, a business user now can ask any data question in plain language and get a result back in seconds.
That’s called conversational analytics, and it’s completely transforming how people consume data and what your role on the data team looks like.
Here’s what’s making this possible: text-to-SQL. It’s the technology that lets an AI model take a natural-language question and turn it into a structured database query.
The AI model handles the translation; that part’s automatic. Whether that translation and the resulting answer are accurate or not depends on something else entirely: the AI context the model is given.
What is AI context in data analytics?
Context is the business-specific knowledge an AI system needs to correctly interpret a question and return an answer that’s actually tailored to a specific user in your organization.
Without context, the resulting answer is wrong in over 70% of interactions. That’s not a guess; it’s based on our internal Text-to-SQL benchmarks.
The breakdown occurs when the AI translates your question into SQL or Python correctly, resulting in a well-formed query that runs without errors — but answers the wrong question.
Why? Because the model doesn’t know things like how your company defines revenue, whether a region was redrawn last quarter, or who’s allowed to see what data.
Why the model alone is not enough
So why isn’t the model alone enough? Every AI model draws on two kinds of knowledge. Let’s explore the differences.
Parametric
knowledge
This is what the model learned during training: general patterns like SQL syntax, common query structures, and how natural language maps to them.
This is what lets a model attempt text-to-SQL in the first place. But it’s generic — and has nothing to do with your business specifically.
Non-parametric knowledge
This is information supplied at the moment you ask your question. It’s the enterprise-specific details a model could never have picked up from public training data: your table structures, your metric definitions, your data quirks, and your access rules.
This information is external to the AI; it doesn’t live inside the model. It has to be supplied.
Example: Exploring the context gap
The following example shows why parametric knowledge alone is not enough. Below, we’ve asked an AI model “how is revenue trending by region?”
As you can see, the model has used its parametric knowledge and provides a general answer based on publicly available links. But what if you want to ask this same question about your own company data? To do this, the model needs to know your data, business, and governance context: which regions your business actually reports on, how revenue metrics are defined, where revenue data is stored, how it is structured, and whether the person requesting the data is permitted to see it.
None of that lives inside the parametric knowledge. The gap between what the model knows and what it needs to know is the context gap.
Go Deeper
Read the article Analyze CSV Data with ChatGPT: Tutorial, Challenges, and Limitations to see the benefits and challenges of using AI to analyze numerical data.
The six types of enterprise context
To close the context gap, you first need to know what kind of knowledge the model is missing. There are six types of enterprise context that fall into three main categories: data context, business context, and governance context.
1. Data context
Data context tells the model what your data is and what it means.
Models and descriptions explain how your data is structured and what’s inside it: which tables exist, how they join, what each column actually contains, and which sources can be trusted. Asked for last quarter’s revenue, a model might reach for the rev_ table in the CRM. Data context tells it that Finance doesn’t trust that table, and that revenue should come from rev_rec in the accounting system instead.
Semantics define your metrics and KPIs: how your company calculates revenue, churn, or active users. If a “strategic account” means a customer with a contract of three years or more, the AI can answer “how many strategic accounts do we have?” without guessing or asking what you mean.
2. Business context
Business context tells the model how your organization works and what the numbers mean in practice.
Ontology maps the entities your business cares about and how they relate to each other: customers and the accounts under them, product lines and their individual product SKUs, regions and the territories inside them. When a sales leader asks about pipeline in NA-West, ontology tells the model which territories make up NA-West and which deals roll up to it.
Institutional knowledge is the history and judgment that usually lives only in people’s heads. For example, it’s knowing that last quarter’s revenue spike came from a one-time data migration, not real growth. So, the AI should leave it out of next year’s forecast. This type of context builds over time as people ask questions, correct answers, and show which results they actually rely on.
3. Governance context
Governance context tells the model who can see what and what it’s allowed to do.
RBAC and guardrails set the rules for access and behavior. Role-based access control (RBAC) determines which data and domains each role can reach. Guardrails set boundaries on the AI’s behavior, such as flagging when an answer relies on preliminary, unaudited figures or declining questions outside its approved scope.
Row-level and column-level security (RLS and CLS) apply access rules to the data itself. Row-level security limits which records a user can see, so a rep in EMEA sees only EMEA deals. Column-level security hides sensitive fields, like salary or customer PII, from anyone who isn’t authorized to see them.
Miss one of these, and the model will still answer your question, but the answer is more likely to be wrong for the business: technically valid SQL, but a wrong conclusion. Below are three examples showing real-life consequences of missing context in AI.
The context window
In order for an LLM to correctly interpret and answer a natural language data question, the AI Context described above must be included in the AI’s context window. You can think of this as the AI’s working memory.
An AI’s context window is its short-term working memory: the information it can hold in view during a single interaction.
For analytics, that window has to carry enough of the six context types for the LLM to interpret the request and generate a query you can trust. But more is not always better. AI exhibits an uneven distribution of attention across the context window, a phenomenon often termed the “lost in the middle” problem.
Lost in the Middle limitation of AI (Source)
Studies have repeatedly demonstrated that tokens situated at the beginning and end of the input receive the most attention, leading to a notable decrease in focus on the middle section. Consequently, placing critical information in the middle of the input can reduce accuracy by over 30% compared to placing it at the start or end.
Key takeaway
Context stuffing actually limits reasoning, so curating and optimizing what enters the context window is critical.
Why context matters
When implemented correctly, context changes the AI response in three ways:
Fewer hallucinations and consistency: Anchored to your real definitions, the LLM invents fewer plausible but wrong queries. Context acts as a semantic referee, mapping terms to the specific user’s company, department, and individual needs. When applied correctly, context ensures answers aren’t guesses from a fundamentally probabilistic system (the LLM).
Governance by default: Context holds your security, access, and compliance rules. By making this information available to an LLM, that helps ensure these controls are applied at query time. So, users only see what they are allowed to see.
Better accuracy and auditability: The model reasons from your organization’s actual structure and language rather than a generic guess. It also gives the LLM the opportunity to “show its work.” Instead of returning an orthogonal stat, it can reference the specific semantic definitions, data sources, and calculations used, and creates a pathway for users to audit, trace, and validate answers.
The payoff shows up in production, where properly applied context lets business users self-serve analytics safely, with confidence in the reliability of the answers.
How much context the model needs is not fixed; it scales with the complexity of the question. A common, textbook-style lookup may need almost none, because the model has seen thousands like it. A question rooted in your business needs far more context because the model was not trained on your data and how the business interprets it. Working from general knowledge alone, it can handle the common cases, but it will return plausible-looking guesses that break on your organization’s real tables and terminology.
Real-world example
Take a support lead who asks, “which accounts are at risk this month?”
Without context: the model grabs whatever churn-like field it finds first, ignores who is allowed to see revenue numbers, and returns a confident-but-wrong answer that nobody catches until an account, that wasn’t identified but should have been, churns.
With context: the model applies your churn definition, respects your access control rules, and returns an accurate list the team can act on.
Key takeaways
Text-to-SQL is only as trustworthy as the AI context around it, not the AI model’s SQL skill alone.
There are six types of context, split into three categories: data context, business context, and governance context.
More context is not automatically better: what enters the context window must be curated.
With the right context, AI analytics returns fewer hallucinations, respects governance, and produces answers teams can trust.