This is a collection of notebooks that I used to learn about RAG with LangChain.
There are separate notebooks for each technique.
-
Chunking Chunking is the process of splitting the document into smaller chunks so that it can be processed by the LLM. There are different techniques to chunk the document like length-based, text-based, semantic-based, and document-based.
-
Query Rewriting Query rewriting is the process of rewriting the query to make it more specific and relevant to the document. There are different techniques to rewrite the query like multi-query, decomposition and step-back
-
Embedding Embedding is the process of converting text into a vector of numbers. This is useful for similarity search. There are different techniques to embed the text like HyDE and HyPE.
-
Indexing Indexing is the process of loading, chunking, embedding and indexing of documents into a vector database. There are different techniques to index the documents like Indexing API.
-
Retriever Retriever is a component that retrieves relevant documents from a vector store. There are different types of retrievers like Vector Store Retriever, BM25 Retriever, Fusion Retriever or Hybrid Retriever and Reranker Retriever.
I have created the ingestion pipeline in ingestion.py that loads and embeds the Vercel AI SDK's docs into a vector store.
You can run these RAGs with langgraph CLI. Install the CLI with
pip install -U "langgraph-cli[inmem]"and runlanggraph devto start the RAG.
-
Naive RAG Naive RAG is a simple RAG pipeline which has just 2 components:
- Retriever Tool: A retriever that uses a vector store to retrieve relevant documents from a vector store.
- Agent Node: A chat model that can answer user's question using the retriever tool.
-
Agentic RAG is an advanced RAG pipeline that implements intelligent document grading and question rewriting for improved retrieval accuracy. Unlike naive RAG, it uses a graph-based approach with conditional routing to handle irrelevant documents and optimize queries.