LLM Zoomcamp: Building LLM Applications with RAG, Agents, and Vector Search

Topics covered

• •Understand why LLMs need external context and how RAG architecture works
• •Build a working RAG pipeline from scratch based on keyword search (minsearch)
• •Master assembling a prompt from search results and calling an LLM through an API

Topics covered

• •Understand the difference between semantic (vector) search and keyword search
• •Learn to turn text into embeddings using sentence-transformers
• •Implement vector search "by hand" with numpy and via minsearch

Topics covered

• •Understand the context problem: why general-purpose AI assistants are unreliable at specialized tasks
• •Master the principles of context engineering — managing what the model sees
• •Deploy Kestra (an open-source orchestration platform) and connect API keys

Topics covered

• •Understand why systematic evaluation of LLM systems is needed and how offline evaluation differs from online
• •Generate a ground truth dataset using an LLM and structured output
• •Evaluate search quality with the Hit Rate and MRR metrics

Topics covered

• •Understand the difference between offline evaluation and online monitoring on real traffic
• •Build a chat application on Streamlit on top of the RAG assistant
• •Capture metrics for every LLM call: tokens, cost, response time

Topics covered

• •Learn the main techniques for improving retrieval quality in RAG pipelines
• •Understand and implement hybrid search: combining keyword and vector search in Elasticsearch
• •Master reranking results, including Reciprocal Rank Fusion (RRF)

Topics covered

• •See how every component of the course comes together in a finished project — a RAG fitness assistant
• •Walk the full cycle: data → RAG → retrieval evaluation → answer evaluation → API → monitoring → Docker
• •Master a project structure: an ingestion pipeline, a Flask API, logging to PostgreSQL, Grafana