Do you know the best way to feed your data to AI?

Your AI solution needs data: your support docs, your business information.

How the model reaches it decides your cost per query, latency, and the quality of your response. Back when LLMs had tiny context windows, RAG was the clear choice. Nowadays there's some nuance to the decision.

Option A: Long context

The newest option. Context windows reaching up to and over a million tokens turned "just put it all in the prompt" into an intuitive, simple option.

That matters most where the answer isn't in any single document. A question like "which security requirements never made it into the release?" may need two or more documents in full to answer, because the answer is the gap between them.

It has caveats worth knowing. Positioning matters, and context in certain areas is predisposed to being missed – such as in the middle and in multi-hop conversations.

Making long context economical

Long context becomes even better when accounting for caching. Caching keys on how a prompt starts, not the whole thing. Put the documents at the front and the user's question at the end, and every request opening with those same documents reuses the cached copy at a fraction of the price, independent of the following question. Check the numbers for your own provider and model first - the minimum cacheable size, how long the cache lives and the size of the discount all differ. See prompt caching.

That discount is what makes putting a whole corpus into every request affordable, so the ordering is an important design decision here.

Option B: Retrieval-Augmented Generation (RAG)

This is the OG. RAG means retrieving the relevant material first and passing only that to the model. The retrieval itself can be keyword, structured, graph-based, vector similarity, or a mix; vector search over embeddings is the most common.

Best for large unstructured prose: support docs, PDFs, policies, transcripts. SSW's RulesGPT works this way, and so does Cursor - it chunks your repo and keeps the embeddings in a vector DB.

On the "RAG is dead" comment: RAG as the default pipeline (i.e. chunk, embed, top-k, then stuff the prompt) is no longer an obvious choice, yet as a capability, it is still an important element of AI solution architecture. Claude Projects switches RAG on when a project outgrows the context window, then backs off when it shrinks.

Option C: Agentic search

Hand the model glob, grep or a query API and let it look. There's no index to keep in sync, so nothing goes stale - as long as the tools read the live source and not a snapshot or a cache. Permissions can come from the access path too, but only if that path authenticates the caller and filters the results before the model ever sees them. Claude Code dropped its vector store for this.

That does not have to mean handing an agent a real shell. Vercel's just-bash is a simulated bash over a virtual filesystem - grep, find, glob and friends, no child processes, and network access off by default - so the model gets the search primitives without terminal access to the host.

The agent chooses its own search at query time, and can reach for any of the same tools RAG uses. The key difference is the AI manages its own context rather than the system.

So which should I use?

These are composable building blocks, not mutually exclusive architectures.

That said, 3 questions settle most cases:

• whether the corpus fits with room to spare
• whether the answer needs all of it at once
• whether the same documents come back on every request

Figure: Decision AI Tree – which retrieval strategy do I want to use?

Using the Decision AI Tree

✅ Figure: Good example - The decision tree recommended using long context for YakShaver

The decision tree is a good starting point. Remember hybrid solutions and more complex context engineering are also options if you need to further optimise the solution.