RAG Cost per Answer Calculator

A RAG cost calculator prices a retrieval-augmented generation pipeline across both of its cost centres — building the index and serving queries — and then divides by the answers that are actually good enough to use. That last step is what separates a real unit-economics figure from a token estimate, because an answer the system produces but a human rejects still consumed full cost.

Ingestion cost comes first and is the one teams instinctively budget for. Embedding a corpus means processing every document once: documents multiplied by tokens per document gives the corpus size, priced at the embedding rate per million tokens. Crucially this is not a one-off if your content changes, so the calculator multiplies by how many full re-index passes you run per month. Query cost is the other half and is usually far larger at any meaningful volume. Each query sends retrieved context as input — typically several thousand tokens of chunks pulled from the vector store — and generates an answer as output, priced at the separate input and output rates. Multiplied by monthly query volume, this is where the money goes. Dividing the combined monthly total by the number of answers that clear your quality bar produces cost per good answer.

The structural insight is that retrieved context is the dominant lever, and it is charged on every single query. Retrieving ten chunks instead of five doubles the input tokens on every request forever, while typically improving answer quality only marginally — and sometimes making it worse, since irrelevant context distracts the model. This is why better retrieval beats more retrieval: a well-tuned reranker that surfaces three excellent chunks outperforms naive top-k retrieval of twenty mediocre ones on both quality and cost simultaneously. Embedding cost, by contrast, is usually modest unless you re-index constantly, so aggressive re-embedding schedules deserve scrutiny; incremental indexing of changed documents only is almost always the right pattern. The answer-quality input matters because it converts spend into value: a pipeline at 50% acceptable answers costs twice as much per usable result as one at 100%, even with identical token consumption. The source research recommends including parsing, OCR, chunking, embeddings, storage, reranking, generation, refresh, and answer-quality yield — this models the dominant terms; add vector-database hosting, reranker calls, and document parsing separately if they are material to you.