RAG development services, measured on retrieval quality
My RAG development services are for teams building AI features that answer from their own content – help centers, contracts, product data, documentation – and finding that the answers are vague, stale or confidently wrong. Retrieval-augmented generation usually fails at the retrieval step, not the model. I fix that, and I measure it.
Sometimes the honest answer is that RAG is the wrong tool. I’ll tell you that too.
Why most RAG implementations disappoint
The tutorial pipeline – split documents every 500 tokens, embed them, take the top five by cosine similarity, stuff them in a prompt – gets a convincing demo and a disappointing product. The failures are predictable:
- Chunks that cut the answer in half, or separate a table from its heading, so the retrieved text is technically relevant and practically useless.
- Pure vector search missing exact terms. Embeddings are poor at product codes, error messages, names and acronyms. Users search for exactly those.
- No metadata, so the system can’t filter by product, version, region, date or customer – and happily cites last year’s pricing policy.
- No permissions, so a user can retrieve a document they’re not allowed to open.
- No measurement. Nobody can say whether the last change made retrieval better or worse.
How I do a RAG implementation
- Start with the questions. I collect real queries from logs, support tickets and users, and label which documents should answer them. That labeled set drives every later decision.
- Structure-aware chunking. Split on headings, sections and list boundaries; keep tables whole; attach the document title and section path to each chunk so it makes sense on its own.
- Hybrid retrieval. Keyword search (BM25) and vector search combined, typically with reciprocal rank fusion, then a reranker over the top candidates. On most corpora this beats either method alone.
- Metadata filters and permissions applied at query time, inside the retrieval layer, not after generation.
- Grounded generation with citations. The model answers from the retrieved passages, cites them, and says when it doesn’t know rather than filling the gap.
- Freshness. Incremental re-indexing when source documents change, with deletes propagated – stale chunks are a common source of wrong answers.
Evals: measuring retrieval and answers separately
I evaluate the two halves independently. Retrieval gets recall@k and ranking metrics against the labeled questions: did the right passage appear, and how high? Generation gets checks for faithfulness to the retrieved text, citation accuracy and refusal when the answer isn’t there. When an answer is wrong, you can then tell whether retrieval missed it or the model ignored it, which tells you what to fix.
The suite runs in CI, so chunking, embedding-model or prompt changes are decided by numbers. See the LLM evaluation guide for the method and RAG architecture for the design choices.
When RAG is the wrong tool
- Questions about structured data. “What was revenue by region last quarter?” wants a SQL query, not passages of text. Text-to-SQL or a tool call against your API is more accurate.
- Teaching a style or format, not facts. That’s prompting or, occasionally, fine-tuning. See RAG vs fine-tuning.
- Small, stable corpora. If everything fits comfortably in the context window and rarely changes, put it in the prompt with caching and skip the retrieval stack.
- Tasks that need every document, like summarizing a whole contract set. Top-k retrieval will silently drop the part that mattered; use map-reduce over the full set.
Stack choices
If you already run Postgres, pgvector plus Postgres full-text search is often enough, and keeps your data, permissions and backups in one place. Dedicated vector databases earn their keep at larger scale or with heavy filtering needs. I choose embedding and reranking models by running your labeled questions through the candidates, not from benchmark leaderboards, and I keep the provider behind an interface so you can re-embed later.
Who this is for
Product teams shipping customer-facing question answering, search or assistants over their own content. If what you want is an internal assistant over company docs, tickets and code, that’s closely related but has its own problems – see AI knowledge base. If you’re still choosing which AI feature to build at all, start with AI consulting.
I’m not an LLM development company with a bench of juniors. I’m one senior engineer who embeds with your team, ships the pipeline in your codebase, and leaves your engineers able to run it. The work starts with a two-week sprint; by the end you have a labeled eval set and a measured baseline, which is usually where the real conversation begins.
Questions
What does a RAG development service include?
Building the whole retrieval-augmented generation pipeline: ingestion and chunking, indexing, hybrid search and reranking, permission filters, grounded generation with citations, freshness, and an evaluation suite that measures retrieval and answer quality separately so changes are decided by data.
Why is my RAG chatbot giving wrong answers?
Usually retrieval: the right passage wasn’t retrieved, was split badly, or was outranked by something similar but stale. Check retrieval first by logging which chunks were used for bad answers. If the right text was there and the model still erred, then look at the prompt and model.
Do I need a vector database for RAG?
Not necessarily. If you run Postgres, pgvector with full-text search covers many production workloads and keeps everything in one database. A dedicated vector store makes sense at large scale or with complex filtering requirements.
Is RAG better than fine-tuning?
For giving a model access to facts that change, yes – RAG is cheaper to update and can cite sources. Fine-tuning suits teaching behavior, format or style. Many production systems use RAG with a well-written prompt and never need fine-tuning.
AI answers that are almost right?
Send me a handful of the bad ones. In a two-week sprint we build a labeled eval set, find where retrieval is failing and ship the first fixes.
GMT+2 · Full EU overlap · US East mornings