Production-Ready RAG in 2026: Vector Databases, Embedding Models, and Enterprise Knowledge Search Architecture

In short. Production RAG is not vector search bolted onto a language model. It is a pipeline in which each layer compensates for the weaknesses of the next. A naive top-k pipeline fails on a real corpus for four predictable reasons, and the fix is engineering at every layer rather than a stronger model. For companies operating in Russia, one more dimension cannot be retrofitted afterwards: chunk-level access control, a closed perimeter, and 152-FZ compliance. On a corpus of roughly 1 million chunks at about 5,000 queries a day, a self-hosted setup works out at around ₽7.8m over three years against ₽15.5m for cloud APIs, with the break-even near 2,000–2,500 queries a day.

Costs below are given in rubles, since they reflect the Russian market this article describes. Dollar equivalents are approximate, at roughly ₽77 to $1.

Any developer can wire vector search to a language model in an evening. Getting that prototype to a state where lawyers, support agents and the finance team will actually rely on it is a different job. RAG — retrieval-augmented generation, where the model answers from fragments retrieved out of your own documents rather than from its training data — has to be accurate, has to refuse to invent, and has to never surface a document the person asking is not cleared to see.

This article treats enterprise knowledge search as an engineering system: what it is assembled from, what to pick at each layer, and how to model the cost. Data sovereignty, chunk-level access control and running models inside a closed perimeter under 152-FZ get their own section, because for anyone operating in the Russian market those constraints shape the architecture rather than decorate it.

Naive RAG and why it never reaches production

The naive pipeline is simple. Documents are cut into pieces, each piece becomes a vector, the vectors go into a database. On a query, the system pulls the top-k nearest fragments by cosine similarity and hands them to the model along with the question. Across a demo folder of ten PDFs this works. Across a corpus of hundreds of thousands of documents, accuracy falls to levels nobody will sign off on.

The gap has a specific cause. Vector search matches on meaning, while corporate queries frequently hinge on exact strings: a contract number, a part code, a surname, the abbreviation of an internal policy. An embedding — a numerical representation of text in which passages with similar meaning sit close together in vector space — handles paraphrase well and gets lost where the literal characters matter. Add chunks sliced through the middle of a table, superseded document versions still sitting in the index, and no quality measurement at all, and the result is a system that answers wrongly with complete confidence.

Four failure modes of a naive pipeline: precision, recall, context, hallucination

Naive RAG fails in four distinct ways, and each is addressed by a different layer.

• Precision. Fragments come back that are topically similar but do not answer the question. Fixed by reranking and hybrid search.
• Recall. The right fragment exists in the corpus but never reaches the top-k, crowded out by chunks that scored closer on vector distance while carrying nothing useful. Fixed by widening the candidate window and by better chunking.
• Context. The fragment has been torn out of its structure — a contract clause with no section heading, a table row with no column names. Formally relevant, practically unanswerable. Fixed by hierarchical chunking and metadata.
• Hallucination. When the retrieved material is thin, the model fills the gap from its own parameters. Contained by strict prompting, grounding against sources, and inline citation.

No single layer solves this on its own. That is the point of the architecture: production RAG is a conveyor in which each stage covers for the stage beside it.

RAG vs fine-tuning vs long context: which approach fits

Before building a pipeline, it is worth asking whether RAG is the right tool at all. Three approaches compete for the same job.

Fine-tuning changes a model's style and behaviour, and suits facts that change weekly badly. Long context is seductive — put everything in the prompt and skip the retrieval layer — but runs into both cost and the lost in the middle effect, where models retain information from the centre of a long input less reliably than from its edges. RAG wins where the corpus is large, changes often, and has to be traceable to a source. In practice the approaches combine: a model fine-tuned on your domain vocabulary, sitting on top of RAG, outperforms either on its own.

Preparing the knowledge base: chunking, embeddings, and Russian-language models

The quality of an answer is set well before generation. No language model can rescue weak retrieval, because a fact it never received is a fact it cannot use. Corpus preparation and the choice of embedding model therefore decide more than the choice of LLM sitting on top.

Chunking and ingestion of difficult documents: PDFs, scans, tables, slide decks

Chunking is the process of cutting documents into fragments for indexing. Splitting naively every N characters breaks sentences and severs tables, so production systems use more deliberate strategies.

• Semantic chunking cuts on meaning boundaries — paragraphs, sections, numbered clauses. A useful target is around 512 tokens with a 10–15% overlap, so that an idea spanning a boundary is not lost.
• Hierarchical (parent/child) chunking indexes small, precise fragments but passes the whole parent block to the model. Retrieval stays sharp and the context stays complete.
• Metadata enrichment attaches source, section, date, version and access rights to every chunk. Those fields drive filtering later, and for security they are not optional.

Ingesting difficult formats is a problem in its own right. Scanned PDFs need OCR, slide decks lose their logic under linear extraction, and tables are the worst offender: split row by row, a table becomes a list of numbers with no column headings. In practice, tables are handled as a separate chunk type with the header repeated in every fragment, or converted into a structured form — Markdown, or key-value pairs — during parsing. Use a purpose-built layout model to recognise document structure, not regular expressions.

Embedding models for Russian: dimensions, context length, running inside a closed perimeter

If your corpus is in Russian, model choice matters more than it does in English, because quality varies noticeably between models — and the shortlist narrows again if the data cannot leave your perimeter.

Three parameters drive the decision. Dimensionality determines index size and memory: 1024 against 3072 is a threefold difference in how much database you are paying to keep in RAM. Context length caps the largest chunk the embedder can take whole — BGE-m3, with an 8192-token window, indexes long fragments without splitting them. Offline operation is mandatory for a great many Russian customers: e5 and BGE-m3 both deploy on your own GPU with no outbound connection. GigaChat Embeddings, from Sber, is the notable domestic option for teams that want a Russian-tuned model without self-hosting it.

In our experience, a sensible default for a Russian-language corpus under perimeter restrictions is self-hosted BGE-m3 or e5-large, with foreign APIs reserved for data that is demonstrably non-sensitive. There is no universal winner, so measure on your own corpus before committing.

Vector databases, hybrid search, and reranking

The storage and retrieval layer usually gets reduced to a list of names — Qdrant, Pinecone, Weaviate, Chroma, pgvector. What matters is where each one runs out of road, and why vector search alone is never enough.

Choosing a vector database for real load: ANN, HNSW, IVF, metadata filtering

Every vector database is built on ANN (approximate nearest neighbour) search. Exhaustive comparison across millions of vectors is too expensive, so the database builds an index instead. Two types dominate. HNSW is a layered graph: fast, highly accurate, and hungry for RAM. IVF clusters vectors: lighter on memory, slightly less accurate. Where accuracy is the priority, HNSW is the de facto standard.

• pgvector. A strong starting point if the data already lives in PostgreSQL. It runs out of road somewhere around a few million vectors, and sooner if you filter heavily on metadata: the HNSW index is awkward to update in place, and post-filtering by permissions damages recall.
• Qdrant. A dependable self-hosted choice. Native metadata filtering applied alongside vector search, a built-in hybrid mode, and predictable behaviour under load. For a closed perimeter it is one of the first candidates to look at.
• Weaviate, Chroma, Pinecone. Weaviate is feature-rich and modular; Chroma is well suited to prototypes; Pinecone is convenient as a managed service, but it is an external cloud, which usually rules it out for sensitive data.

Plan memory in advance. An HNSW index over 1 million vectors at 1024 dimensions in float32 needs roughly 4 GB for the vectors alone, plus the graph edges on top — budget 6–8 GB of RAM per million. The graph also degrades as the corpus changes: sustained bulk inserts and deletions eventually force an index rebuild.

Hybrid search and reranking: dense + sparse, RRF, BGE and Cohere rerank

Pure semantics fails on exact matches. The industry's answer is hybrid search, which combines dense (vector) retrieval with sparse (lexical) retrieval. The lexical half is handled by BM25, the classic ranking algorithm that scores documents on term overlap and reliably catches codes, names and rare terminology.

The two result sets are merged with RRF (reciprocal rank fusion), a method that combines ranked lists using the reciprocal of each item's rank. Because it uses positions rather than scores, it is immune to the fact that dense and sparse retrieval produce values on entirely different scales. The output is a single candidate list that accounts for both meaning and literal wording.

The final layer is reranking. After hybrid search, an expanded candidate set — say the top 50 — is passed through a cross-encoder, a model that scores the query and the fragment together as a pair rather than comparing two vectors computed separately in advance. It is more accurate and more expensive, which is why it only ever sees a short list. For Russian, BGE-reranker is the strongest open option to run locally; Cohere Rerank and Voyage rerank are the cloud equivalents. Expect 50–150 ms of added latency and a substantial accuracy gain. Hybrid search followed by reranking closes most of the distance between a demo and a production system.

Security, access control, and data sovereignty

Here enterprise RAG acquires a dimension that public chatbots do not have. The corpus almost always contains material that not everyone is allowed to see: contracts, personal data, financials, HR records.

Chunk-level access control and row-level security at retrieval time

The classic mistake is filtering access after the fact, once the answer has been generated. If the model has read a confidential fragment and built its answer on it, a filter that fires at the end has already lost: suppressing the source link does nothing about the content that has leaked into the wording.

The correct approach is filtering at retrieval. Access rights are written into each chunk's metadata at indexing time, and the search query is constrained from the outset to what the user is permitted to see. This is the same principle as row-level security in a relational database, applied to knowledge fragments instead of rows: an employee cannot physically receive something they have no rights to, because it never enters the candidate set. Technically it requires a vector database that filters efficiently on metadata during search — otherwise, at scale, the filter either slows retrieval down or destroys recall. This is where engines with native filtering, such as Qdrant, are preferable to pgvector with post-filtering.

On-premise and 152-FZ: closed perimeter, self-hosted LLMs and embedders

For data covered by 152-FZ, Russia's federal personal data law, the position is unambiguous. Personal data belonging to Russian citizens must be processed and stored on servers physically located in Russia, and sending it to external services is a risk both legally and operationally. The obligation attaches to whose data you process, not to where your company is incorporated — which is why it applies to foreign businesses operating in the Russian market as much as to domestic ones. In practice it rules out foreign APIs for embedding and generation whenever personal data or commercial secrets pass through them.

Hence the move to a closed perimeter: the vector database, the embedding model and the LLM itself all run inside your own infrastructure with no outbound path. Hosting in an attested Tier III data centre adds to that — attestation here means certified by FSTEC, Russia's technical and export control regulator, against the protection requirements for the relevant class of system. Tier III brings redundant power and cooling with availability above 99.98%, alongside the physical security controls the personal data regime expects.

The bottleneck in this scenario is GPU capacity. Running open models and embedders inside your own perimeter requires accelerators, and buying a fleet for a pilot is hard to justify: the hardware is expensive, in short supply, and ageing quickly. For teams that need a closed perimeter without the capital outlay, the Cloud4Y LLM platform covers exactly this gap — a cloud service for deploying and fine-tuning an open model on the provider's GPU infrastructure in a Russian data centre. You can stand up an open LLM and a Russian-language embedder, adapt them to your own data, and keep the corpus inside the perimeter without buying a single accelerator — a sovereign RAG environment assembled on rented capacity.

Quality evaluation, operations, and economics

A system nobody measures degrades quietly. Swap the embedding model, retune the chunking, refresh the corpus — and accuracy drops ten points without anyone noticing, because there was no instrument to notice with. Evaluation and operations belong in the architecture, not in a final round of polish.

Retrieval metrics and drift monitoring: golden set, Recall@k, MRR, NDCG

Evaluation starts with a golden set: a collection of representative questions paired with pre-labelled correct answers. Building one for a Russian corpus is manual work — 100 to 300 genuine questions from the people who will use the system, labelled by someone who knows the domain. It is dull work, and without it every optimisation afterwards is guesswork.

Against that set you compute the retrieval metrics:

• Recall@k — how often the required fragment appears in the first k results. Aim for 0.9 or better in production.
• MRR (mean reciprocal rank) — how high up the list the first relevant result sits.
• NDCG (normalised discounted cumulative gain) — accounts for both relevance and ordering, penalising a system that buries the important result near the bottom.

Generation quality is assessed separately using LLM-as-judge, where a stronger model scores answers for groundedness, completeness and absence of invention. The approach scales, but the judge is biased toward verbose answers and toward its own style, so its scores need periodic calibration against human labelling. Tools such as Ragas automate part of the metric pipeline without replacing the golden set. The operating rule is simple: run the evaluation on every pipeline change, and keep the metrics on a dashboard so regressions surface before users find them.

Corpus lifecycle and TCO: updates, freshness, and cost modelling

A corpus is alive. Documents are added, edited and superseded. Freshness comes from incremental updates rather than a full nightly re-index: when a document changes, only its chunks are recomputed. Near-duplicate versions are a separate problem — with three revisions of a policy sitting in the index, the model may well quote the obsolete one. Version fields in the metadata and a rule that only the current revision is eligible for retrieval will handle it.

Now the economics. Compare two scenarios over three years for a corpus of roughly 1 million chunks (on the order of 300,000–500,000 documents) at a load of about 5,000 queries a day: a private environment on rented GPU infrastructure, against paying cloud API fees per query.

The conclusion is not that cloud is always dearer. It is that the two cost structures behave differently. Self-hosted costs are almost entirely fixed; cloud costs are variable, at roughly ₽2.5 (about $0.03) per query. Break-even falls at approximately 2,000–2,500 queries a day: below that, cloud APIs are cheaper, and above it the private environment repays itself. There is a second factor that has nothing to do with price. Where the data is sensitive, foreign APIs are often excluded by 152-FZ before cost enters the discussion, and the real comparison becomes self-hosted against a Russian cloud offering GPU inside a closed perimeter. Treat the figures as indicative — the totals move by ±25% depending on the generation model, the query load and local electricity tariffs.

Key takeaways

Production RAG is not vector search layered over a language model. It is a multi-stage pipeline in which answer quality is determined long before generation. A naive top-k-plus-LLM setup breaks on a real corpus for four predictable reasons, and the cure is not a stronger model but engineering at every layer: deliberate chunking and ingestion of difficult documents, an embedding model chosen for your language and your perimeter constraints, a vector database whose trade-offs you understand, hybrid search with reranking, and — without exception — continuous evaluation against a golden set.

For enterprises in the Russian market, one further dimension cannot be bolted on later: sovereignty and security. Filtering chunks by permissions at retrieval time, keeping the corpus and the models inside a closed perimeter, meeting 152-FZ, and hosting in an attested data centre are architectural decisions, not configuration options. Model the economics over several years, separating fixed from variable costs honestly, and it becomes clear where cloud APIs beat a pilot and where a private GPU environment pays for itself while removing the question of data leaving the perimeter entirely. Start simple — a basic pipeline and a golden set of a couple of hundred questions — then add layers, checking each against the metrics rather than against intuition.