AI context methodologies
What you will learn in this module
How context is provided to AI systems
Prompt engineering vs context engineering
How to manage context as a lifecycle
How do you build AI context?
An AI system never answers a question alone. It draws from a bundle of inputs assembled at the moment of the request, and five core components carry that context:
System instructions: The standing rules a model was trained on, as well as the configurations set up by an admin on your team.
User input: The question as the person typed it, otherwise known as a prompt, and what they actually meant by it.
Conversation history: The filters, prior queries, and intermediate results carried over from earlier turns in the session.
Retrieved knowledge: The table definitions, metric definitions, and documentation pulled in for this specific question.
Connections: The systems the model is permitted to call, and the instructions for calling them correctly.
Here’s how those components are used to answer one question:
Imagine you ask AI: “What was churn last quarter?”
System instructions supply the model with a generalized understanding of what churn is and that quarters run in three-month instances.
The question is the user input.
Conversation history may tell the LLM that you’ve asked about churn before.
Retrieved knowledge brings in the customer tables and the key they join on.
A connection routes the query to an authorized warehouse connection.
The model writes the SQL, but almost every judgment and decision that influences that SQL is determined by a separate input. Each AI context methodology is defined by the way it satisfies those five input categories. Early text-to-SQL work manually filled the inputs, a practice known as prompt engineering.
As databases, question complexity, and AI adoption grew, teams realized they needed a way to build systems that assemble context per request. This is what the industry now calls context engineering. The sections below walk through the progression from static prompts to dynamic context and, finally, a managed context layer, describing how context governance interacts with all three methodologies.
1. Prompt engineering
Prompt engineering is the craft of writing context by hand and shipping it with the request. System prompt design sets the model’s role (“you are a SQL analyst for a retail company”), its rules (revenue excludes canceled orders), and its guardrails (never answer with a column that is not in the supplied schema). Few-shot prompting adds two or three examples, pairing a question with the SQL you want back, which makes the model more likely to follow that pattern next time.
Picture a PostgreSQL sales database. Asked to “show revenue by region” with nothing else supplied, the model reaches for whichever table looks plausible and sums whatever column is named something like “revenue.” The resulting number may quietly include canceled orders.
Now give it a system prompt that includes the three relevant table definitions, the rule that revenue means completed-non-returned orders only, and two example question-and-SQL pairs. The same model is far more likely to return the correct answer.
Advantages
Fast to prototype: Test ideas without backend builds.
Flexible: Ideal for single-turn work without retrieval layers.
Transparent: All instructions live in one readable file.
Disadvantages
Sensitive to phrasing: Variable wording leads to inconsistent answers.
Bounded window: Large schemas can crowd out key instructions.
Fragile: Complexity makes prompts difficult to maintain safely.
Prompt engineering is quick to iterate, but ultimately unscalable for complex databases and enterprise user bases.
2. Grounding with dynamic context injection
If the prompt cannot hold everything, the only answer is to stop trying to load everything. This methodology moves context out of the hardcoded template: the system fetches the relational context this question needs and injects it at query time.
Two techniques help the scaling problem:
Dynamic schema retrieval: Vector search across a data catalog fetches the table definitions, foreign keys, and column descriptions most relevant to the question at hand. A 150-table schema that would swamp a prompt becomes a payload of three. This is much more manageable for the AI and cost-efficient for the business.
Semantic injection: Vague and ambiguous terms like “active monthly users” or “ARR” resolve through a semantic layer, such as a dbt semantic model or a LookML model. When in use, the model receives the definition that the business agreed on rather than one it infers from a column name.
Sounds better, right? Agreed. However, this method still contains limitations. Namely, retrieval delivers a static snapshot of a data source that is constantly in motion.
When schema migrations rename columns or planning cycles update how active users are calculated, retrieval will keep serving the same outdated artifacts unless freshness is explicitly verified. This is known as context drift, which is the primary hurdle in this form of contextual grounding. This failure mode is particularly dangerous in an enterprise setting where the resulting answer sounds superficially plausible, even as the underlying logic grows obsolete.
A related failure manifests as conflicting context. For example, when a data catalog and a semantic layer disagree on a metric like revenue, the model may ingest both definitions, select one arbitrarily, and completely fail to report that a contradiction existed.
3. Connecting to a managed context layer
Look at where context sits now. Rules live in prompt templates, definitions live in a semantic layer, structure lives in a catalog, and corrections live in a Slack thread AI cannot access. Fragmented context produces fragmented answers: the same question asked in two applications returns two different numbers, each traceable to a defensible source.
A context layer centralizes those fragmented systems, giving data teams one place to manage context. With the right supporting architecture in place, the LLM will access the context layer at query time. As a result, relevant context is applied and answers become more accurate and efficient. We’ll cover more on this architecture in module three.
There are three main benefits to this context management approach:
Centralization: the context layer provides one interface where data teams can define business rules, map terminology, and manage sources. This hub connects to version-controlled tools such as dbt, LookML, and data catalogs, so an upstream schema or metric change can propagate across the platform. So a revised churn rule is edited once instead of chased through fragmented systems.
AI-augmented refinement: best-of-breed context layers deploy AI to automatically suggest revisions to resolve known logic clashes. Of course, there may still be some cases when internal conflicts exist. For instance, finance and product disagree about what counts as an active user. In both of these cases, a dedicated domain owner reviews these suggestions to codify them into the context layer.
Feedback loops: advanced context layers can also loop in chat logs and user-provided feedback, using AI to curate instances where there is missing or conflicting context. With the model aggregating those revisions for human review, domain owners can maintain and improve context in a continuous lifecycle. When a context layer has named owners, versioned definitions, and an audit trail behind managed changes, it works to maintain up-to-date context that can be applied at query time.
In this section, we’ve referenced a domain owner that manages the context layer, connecting context, resolving missing or conflicting context, and ensuring context is correctly applied at query time. That responsibility is becoming a role of its own — AI Context Engineer.
Go deeper
Read Anthropic’s guide to effective context engineering for AI agents for the agent-side mechanics of curating a context window, including compaction and structured note-taking.
Key takeaways
Moving beyond prompt engineering is the prerequisite for scaling LLMs across enterprise databases.
A context layer centralizes domain knowledge. With the right supporting architecture in place, the LLM will access the context layer at query time, delivering more efficient execution and more accurate answers.
Managing the context available to the AI system is a new responsibility for the data analyst.