Natural Language Processing and LLMs: From TF-IDF to RAG
Natural Language Processing (NLP) and LLMs is an advanced-level course of 6 lessons (about 4 h 21 min), with 19 exercises and 18 quiz questions. Prerequisites: Deep Learning with PyTorch. The course needs a full project environment, planned for a later phase of the platform.
- Data science
- Advanced
- 6 lessons
- 4 h 21 min
- 19 exercises
- Full project environment (phase 2)
The course in brief
This course covers natural language processing, from classic representations to large language models. You will build text classifiers, a semantic search and a sourced RAG system, understand attention and generation parameters, then adapt a model with LoRA while objectively measuring the gain.
What you will be able to do
- Preprocess and tokenize text, and explain subword tokenization (BPE)
- Build an interpretable TF-IDF text classifier with scikit-learn
- Set up a semantic search with embeddings and cosine similarity
- Explain attention and use an LLM with messages, temperature and structured outputs
- Design, implement and evaluate a RAG system with citations
- Choose between prompting, RAG and fine-tuning, and fine-tune a model with LoRA
Syllabus: 6 lessons
- Normalize a text (case, accents, whitespace) while knowing the trade-offs
- Split a text into tokens by word and then by subword
- Understand the principle of the BPE algorithm used by LLMs
- 02
Bag of words, TF-IDF and text classification
- Represent documents with a bag of words and with TF-IDF
- Build a text classification pipeline with scikit-learn
- Interpret the most discriminating words of a linear model
- 03
Embeddings and semantic search
- Explain what an embedding is and why it captures meaning
- Compute a cosine similarity and search for the closest documents
- Use a sentence embedding model and a vector database
- 04
Transformers, attention and large language models
- Explain the attention mechanism and the causal mask
- Distinguish encoders (BERT) from decoders (GPT) and their uses
- Use an LLM: messages, generation parameters, structured outputs
AttentionTransformer architectureLarge language modelDecoding (temperature, top-p)
- 05
RAG: retrieval-augmented generation
- Describe the architecture of an end-to-end RAG system
- Split documents into chunks and build a search index
- Assemble a sourced prompt and evaluate retrieval and answers
- 06
Adapting a model: lightweight fine-tuning (LoRA) and evaluation
- Choose between prompting, RAG and fine-tuning depending on the need
- Understand LoRA and QLoRA and estimate the number of trained parameters
- Evaluate a generative model with suitable metrics and protocol
Final project
RAG document assistant evaluated on a knowledge base
Build an assistant that answers questions about a set of documents (FAQ, terms and conditions, product documentation of about twenty pages, or public documents of your choice). It must cite its sources, say "I don't know" when the information is missing, and be evaluated quantitatively.
Deliverables
- An indexing pipeline: extraction, chunking with overlap, embeddings and vector index
- An answer function: retrieval (dense or hybrid), prompt construction and generation with an open LLM or an API
- An evaluation set of at least 30 questions with relevant passages and reference answers
- A report comparing at least two configurations (chunk size, TF-IDF vs embeddings, with or without a reranker) in recall@k, MRR and F1
- A TF-IDF classification baseline or an analysis of unanswerable questions, and a section on risks (hallucination, prompt injection, personal data)
Assessment criteria
- Retrieval quality is measured (recall@k, MRR) and justified
- Answers are faithful to the sources, with citations and handling of "I don't know"
- Evaluation protocol that separates retrieval from generation
- Relevance of technical choices (chunking, embedding model, generation parameters)
- Consideration of security and privacy
Frequently asked questions
Do I need to know deep learning to take this NLP and LLM course?
Yes, the basics of PyTorch (tensors, networks, training loop) are recommended: the DS7 course is the prerequisite. The first lessons remain accessible with scikit-learn.
What is RAG and why use it?
RAG (retrieval-augmented generation) gives the LLM passages from your documents before it answers. It enables up-to-date, sourced and verifiable answers without retraining the model.
Do I need a GPU for LoRA fine-tuning?
A GPU is strongly recommended. LoRA and QLoRA greatly reduce the requirements, and a small model of 0.5 billion parameters can be fine-tuned at small scale on a CPU to learn the method.
Does the course use paid LLM APIs?
No, the exercises use open-source libraries and small open models. The principles (messages, temperature, structured outputs) apply in the same way to commercial APIs.
This course, told for your job
Say what you want to be able to do: “Natural Language Processing (NLP) and LLMs” takes its place in your path, with examples from your field. The concepts stay those of the course.