What building your own RAG stack actually costs

The Curiosity Team

• 2026-07-02

A RAG prototype is cheap to build: a weekend, a vector database, an API key. A version that runs in production for a whole company costs a lot more, and most of that cost is not visible from the prototype. To see where it goes, start with what a production system is made of.

What a production RAG system is made of

A prototype is one or two of these components. A production system needs all of them, integrated and kept running:

• Connectors and ingestion: pull content from each source, parse it, chunk it, and keep it in sync as it changes.
• Storage and indexing: a vector database for embeddings, usually alongside a keyword index.
• Embedding models: turn content into vectors, and re-embed when documents or models change.
• Retrieval: hybrid keyword and semantic search, often with a reranking step.
• Knowledge graph: the relationships between records, because company knowledge is in the connections as well as the documents (why that matters).
• Permissions: resolve each user's access at retrieval time, so answers only use what that user is allowed to see (why this belongs at retrieval).
• LLM and orchestration: the model itself, plus query processing, prompt construction, caching, and fallbacks.
• Monitoring and evaluation: relevance tracking, quality checks, and audit logs.
• API and front end: how people and applications reach it.

None of these is exotic on its own. The cost is in building, integrating, and running all of them together.

Where the cost goes

Building it. A realistic first build is a team of about four engineers for a year. At €150k per engineer, that's roughly €600k before anything reaches users, and it assumes you can hire people who have built this before.

The opportunity cost of the delay. If the finished system would save the business, say, €2M a year, then every year spent building instead of running is €2M of value not yet realized. A one-year build carries about €2M in opportunity cost on top of the build itself. It is often the largest figure, and it depends entirely on the value the system provides once it runs.

Running it. Once live, the system needs maintenance: connectors break as source APIs change, retrieval needs tuning, models get upgraded. Budget at least one full-time engineer ongoing (about €150k a year), plus the security and compliance work that any system touching company data requires.

Component licenses. A build stitches together separate licensed products, each with its own recurring fee: a graph database, a search engine, a vector database, and others. A bought platform replaces those with a single license. Exact figures depend on vendors and scale, so there are no numbers here.

Costs that are the same either way. In this example, LLM inference runs around €50k a year and infrastructure for a large instance (compute and storage) around €100k a year, whether you build or buy. They are the cost of running the workload at all, and sit outside the trade-off.

A few costs are easy to forget: hiring and ramp-up time for scarce specialists, the key-person risk when the people who built the system move on, security audits and certifications, and the migration work each time an underlying component or model changes.

Summary

The figures are illustrative; put your own in. Inference and infrastructure cost the same whichever way you go. A bought platform has its own license, and customization and upkeep are not free either, but it comes without the build, the delay and the separate component licenses, which is where most of the difference sits.

The takeaway

Building can be the right choice. The full figure is the build, the opportunity cost of the delay, the ongoing maintenance and the component licenses, on top of the inference and infrastructure you pay for either way. For how to weigh building against buying, see make or buy: the RAG stack decision.

To see the layer running on your own sources and access model, talk to an engineer.