RAG in Production
RAG in Production is an advanced-level course of 5 lessons (about 1 h 40 min), with 15 exercises and 15 quiz questions. Prerequisites: Integrating an LLM into an Application, SQL Fundamentals (SELECT, Filters, Aggregates). Exercises run in the browser, with nothing to install.
- Generative AI
- Advanced
- 5 lessons
- 1 h 40 min
- 15 exercises
- Exercises run in the browser
The course in brief
This course covers what makes a RAG usable in production. You split documents according to their structure with metadata, store and search vectors with pgvector, combine lexical and vector search, measure quality with an evaluation set, then keep the index up to date while respecting each user's rights. The exercises run in Python and SQL, with provided vectors and a simulated model.
What you will be able to do
- Build a reproducible indexing pipeline with structural chunking, overlap, and metadata
- Store embeddings in PostgreSQL with pgvector and write a filtered search consistent with its index
- Implement a hybrid search (BM25 and vectors) and merge the rankings
- Evaluate a RAG with recall@k, precision@k, and checks on citations and abstention
- Synchronize the index incrementally and atomically
- Apply isolation between organizations and access rights in the search query
Syllabus: 5 lessons
- Build a reproducible indexing pipeline: extraction, chunking, metadata, storage
- Split a document according to its structure (headings, paragraphs) with a maximum size and an overlap
- Attach to each chunk the metadata needed for citations, access rights, and updates
- 02
Embeddings and pgvector: Storing and Searching Vectors
- Explain what an embedding is and the constraints it imposes (same model, same dimension)
- Compute cosine similarity, dot product, and Euclidean distance and link each to a pgvector operator
- Write a search filtered by metadata and choose between exact search and approximate index
EmbeddingsSimilarity and distancespgvectorApproximate index (HNSW, IVFFlat)
- 03
Hybrid Search: Keywords, Vectors, and Fusion
- Explain the strengths and weaknesses of lexical search (BM25) and vector search
- Compute a BM25 score and merge two rankings (RRF or normalized weighted combination)
- Place reranking and the number of candidates in a search chain
BM25 lexical searchHybrid searchReciprocal rank fusion (RRF)Reranking
- 04
Evaluating a RAG: Search, Citations, and Answers
- Build an annotated evaluation set (questions, relevant documents, expected answers)
- Measure search with recall@k and precision@k and compare configurations
- Automatically check citations and abstention, then complete with an evaluation of the answers
Evaluation setRecall@k and precision@kCitation checkingAnswer faithfulness
- 05
Updates and Access Rights
- Synchronize the index with the source documents (additions, modifications, deletions) incrementally
- Filter the search according to the organization and the user's rights, in the query
- Handle deletion, versions, and personal data in a RAG index
Incremental indexingAccess rightsIsolation between organizationsDeletion and retention
Final project
Multi-Team Internal Document Assistant
For a fictional company of 200 people, build the core of a RAG document assistant over about twenty Markdown documents (HR, IT, management) with rights per group. Implement structural chunking, a hybrid search filtered by organization and by groups, cited answers with a simulated model, incremental synchronization, and an evaluation set of at least 20 questions (including questions with no answer and questions that a group is not allowed to ask). Compare at least three configurations and justify your choice.
Deliverables
- The indexing pipeline (chunking, metadata, fingerprints, atomic replacement)
- The SQL schema with pgvector (tables, index, filtered query) and the rights query
- The hybrid search service (lexical, vector, fusion) and the citation check
- The versioned evaluation set and the configuration comparison table (recall@k, precision@k, abstention)
- A one-page note on the choices, limits, and risks (rights, indirect injection, personal data)
Assessment criteria
- No chunk from another organization or an unauthorized group can be retrieved, backed by tests
- The measurements are reproducible and the chosen configuration is justified by the numbers
- Citations always point to provided chunks and the system abstains without context
- Synchronization reindexes only the modified documents and deletes the chunks of removed documents
- The code is readable, tested, and separates indexing from querying
Frequently asked questions
How does this differ from the RAG lesson in the NLP and LLM course?
Course DS8 presents the principle of RAG and a prototype. This course covers production: structural chunking and metadata, pgvector and indexes, hybrid search, tooled evaluation, incremental synchronization, and access rights.
Do you need a dedicated vector database?
Not necessarily. If your application already uses PostgreSQL, the pgvector extension lets you store vectors next to the data and the access rights, and filter in the same query. A dedicated database can be justified at very large scale.
Do the exercises call a real embedding model?
No. The vectors are provided or computed locally and the model is simulated, so that the exercises are fast, deterministic, and executable in the browser. The methods apply as is with a real model.
What prerequisites do you need?
The course Integrating an LLM into an Application (AI2) and SQL basics (S1): queries, joins, and subqueries.
This course, told for your job
Say what you want to be able to do: “RAG in Production” takes its place in your path, with examples from your field. The concepts stay those of the course.