Do you know the best way to feed your data to AI?

Your AI solution needs your support docs and business information. Feed it too little and it misses the answer. Feed it everything and you can pay more and wait longer.

Choose how the model accesses that data: long context, retrieval or agentic search. Reuse preparation for repeated tasks, and add graphs or fine-tuning when they solve a measured problem.

Option A: Long context

If all the material needed to answer a question fits inside the context window, put it straight into the prompt. This requires the least infrastructure and helps when the answer depends on comparing whole documents, such as "Which security requirements never made it into the release?"

Figure: Long context supplies all three documents for comparison

Long inputs can increase cost and latency. Models can also miss information in the middle or struggle with questions joining several pieces of evidence. Fitting the context window does not guarantee an accurate answer.

Prompt caching - making long context economical

Put stable instructions and documents first, then the changing question. Requests with an identical prefix may reuse cached processing, reducing input cost and latency. Check your provider's minimum size, cache boundaries, lifetime and pricing, and measure actual cache hits. See prompt caching.

CAG (cache-augmented generation) preloads a bounded knowledge base and reuses its attention cache to answer without query-time retrieval. The knowledge base must fit; caching does not fix missing evidence or reasoning errors.

Option B: Retrieval-Augmented Generation (RAG)

RAG retrieves relevant material and passes it to the model. Retrieval can use keywords, structured queries, graphs, vector similarity or a mix. It often suits large collections of support docs, PDFs, policies and transcripts.

Figure: Vector RAG supplies retrieved chunks rather than whole documents

SSW's RulesGPT uses RAG. Cursor chunks code and embeds it for semantic search.

Is RAG dead? A fixed chunk-and-retrieve pipeline is no longer the automatic starting point, but retrieval remains useful. Claude Projects enables RAG as project knowledge approaches the context limit, then can return to context-based processing when it shrinks.

Keep optional query enrichment bounded

Query expansion can improve retrieval, but an optional helper must not become a single point of failure. Preserve the user's original query, set a helper timeout and attempt budget, and validate the expanded queries before using them. On a helper timeout, connection error, empty result or malformed output, search with the original query and record the fallback reason. Reserve enough of the request deadline for that search.

Keep authentication, authorisation, tenant filters and policy checks mandatory on every path. Caller cancellation or a policy denial must stop the request; neither triggers fallback. If retrieval itself fails, expose that failure. If optional re-ranking fails, retain the baseline candidates rather than returning nothing.

Before release, inject each helper failure and confirm one bounded fallback with the same access filters. Test cancellation, denial and backend failure separately. Compare enriched retrieval with the original-query baseline on exact identifiers and ambiguous questions using evals.

Option C: Agentic search

Give an agent tools such as grep, a search API or read-only SQL access. It searches, inspects the results and decides what to search next, refining its query when the first results reveal a gap. Unlike a fixed RAG pipeline, the agent manages its own context and can use several retrieval methods.

Querying the live source avoids a separate retrieval index; APIs may still return cached or indexed results. For structured records, a correct SQL query computes joins and totals directly. See database access for AI agents.

Shell tools can run in a sandbox. Vercel's just-bash provides bash-like commands over a configurable filesystem, with networking off by default. Enforce file access, user permissions and execution limits outside the model; a simulated shell does not replace operating-system isolation.

Option D: Precompute once, reuse every run

For repeated tasks, do the shared reading and reasoning before requests arrive. Save a summary, brief, checklist or worked examples, then supply it with each run's new input. Sleep-time compute explores this preparation before queries arrive.

Figure: Reuse the shared research while supplying separate customer evidence

For example, a team repeatedly assesses customers against the same regulations. Prepare a reviewed brief explaining each question, acceptable evidence and common mistakes. Keep source links and effective dates, refresh it when regulations change, and test that the summary preserves the evidence needed for answers.

Precomputation saves repeated research; prompt caching saves repeated processing of an eligible prefix. Combine them by placing the shared brief before each customer's authorised documents. Keep customer evidence out of the shared brief. To improve it over time, see how prompts can learn from feedback.

Option E: Knowledge graphs

A knowledge graph stores entities and their relationships. Nodes can hold entity summaries and links to source documents. Graph retrieval is a form of RAG: traverse the relationships, optionally combine them with vector search, and pass the connected facts and sources to the model.

Use a graph when a question follows several relationships across documents. In this fictional Northwind example, ask: "Which customers bought discontinued products from German suppliers?"

Figure: The supplier's country matters, not the customer's address

❌ Figure: Bad example - Separate matching words do not establish a purchase relationship

✅ Figure: Good example - A complete purchase path supports the answer and excludes the unrelated customer

Microsoft GraphRAG also extracts entities and relationships, then builds community summaries for broader questions about a corpus. KAG combines graphs, vector retrieval and logical-form-guided reasoning.

Budget for extraction, entity resolution (merging duplicate names), schema design and updates when sources change. A visible path still needs accurate sources. If the relationships already live in a SQL database, query it directly (Option C) rather than building a second store. See knowledge graphs as agent memory.

When should you fine-tune?

Fine-tuning adapts a pretrained model using task-specific examples. Consider it for stable, high-volume classification, extraction, routing or response formatting. A smaller specialised model may reduce inference cost and latency; include training, hosting and maintenance in the comparison.

Keep changing facts in a current source. For example, a model trained on the old price of Northwind's Chai tea can still quote it after the catalogue changes. Retrieve the price; consider training the stable task of routing the question to that lookup.

Before training:

• Build held-out evals covering normal and edge cases; keep them separate from training data.
• Measure a prompted baseline with few-shot examples. If it meets your targets, stop there.
• Prepare representative demonstrations with correct outputs and permission to train on them; remove sensitive data.
• Compare quality, latency and total cost, switch only when the targets are met, and re-evaluate when the task or base model changes.

See running evals on your AI features. Fine-tuning can work alongside retrieval; it does not supply a source-update mechanism.

Which should I use?

These building blocks can be combined. For exact joins, totals or current structured records, start with a read-only source query (Option C). For document questions:

1. Does the required corpus fit with room for instructions and output? If not, retrieve relevant evidence with Option B or C.
2. Does the answer need the whole corpus at once? If so, try long context.
3. Do the same documents recur on every request? If so, try long context with caching; otherwise, try RAG.

Then add a precomputed brief for repeated research, a graph for relationship questions, a feedback playbook for recurring corrections, or fine-tuning for a stable task that justifies training.

Figure: Choose a starting candidate, then add what the task needs

For example, YakShaver turns a screen recording into a product backlog item. To choose its destination, a small authorised project list fits in context (Q1) and comparing all projects helps select the best match (Q2). That makes long context a starting candidate, with caching if the list repeats and qualifies.

Test candidates on the same held-out questions. Measure answer correctness, source coverage, cost and latency, including missing-evidence and changed-source cases. Keep the simplest approach that meets your targets.