Constructing Retrieval-Augmented Generation (RAG) with LangChain

Introduction to LangChain for RAG

LangChain is an open-source framework designed to simplify the retrieval pattern across different data sources and vector databases. While it has expanded beyond its original scope, its core functionality remains instrumental in implementing retrieval mechanisms for RAG (Retrieval-Augmented Generation).

One of the key steps in RAG is storing documents for later retrieval. LangChain provides tools to load, transform, embed, and store documents efficiently, as illustrated in the following process:

Splitting and Loading Documents with LangChain

To enhance the retrieval mechanism, we need to break down documents into meaningful chunks. Instead of providing an entire document in a query (which can be costly and inefficient due to token limitations), we split it into relevant sections.

For example, when querying a local document, we don’t pass the whole file into the prompt. Instead, we extract contextually relevant portions to optimize the retrieval process.

Code for Splitting and Loading Documents

from langchain_community.document_loaders import UnstructuredPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

# Load the document
loader = UnstructuredPDFLoader("sample_documents/sample_report.pdf")
data = loader.load()

# Split the document into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=150,
    chunk_overlap=30,
    length_function=len,
    add_start_index=True,
)
documents = text_splitter.split_documents(data)

documents = [doc.page_content for doc in documents][50:300]

# Get embeddings for each document chunk
embeddings = [get_embedding(doc) for doc in documents]
ids = [f"doc{i}" for i in range(len(documents))]

Output:

Enter a search query (or 'exit' to stop): What is the conclusion of the report?
How many top matches do you want to see? 3
Top Matched Documents:
ID:doc112 TEXT: The report concludes that SCORE: 0.6

The PDF document is split into 150-character chunks with a 30-character overlap. This overlap ensures that meaningful information isn’t lost between segments.

Splitting Documents Using Tokens in LangChain

Tokenization is a method of breaking down text into word tokens, which helps improve semantic similarity matching. Unlike fixed-length chunks, token-based splitting adjusts dynamically based on text content.

Code for Token-Based Splitting

from langchain.text_splitter import SentenceTransformersTextSplitter
from langchain.vectorstores import FAISS
from langchain.embeddings.openai import OpenAIEmbeddings

# Load the document
loader = UnstructuredPDFLoader("sample_documents/sample_report.pdf")
data = loader.load()

# Token-based splitting
text_splitter = SentenceTransformersTextSplitter(
    model_name="all-MiniLM-L6-v2",
    chunk_size=50,
    chunk_overlap=10
)

documents = text_splitter.split_documents(data)
documents = [doc for doc in documents][10:100]

# Store the documents in a vector database
db = FAISS.from_documents(documents, OpenAIEmbeddings())

def query_documents(query, top_n=2):
    docs = db.similarity_search(query, top_n)
    return docs

Output:

Observations

• This approach improves document retrieval by considering token-based similarity.
• Tokenization eliminates unnecessary whitespace and enhances semantic understanding.
• The results improve when searching with semantically similar phrases.

For example, entering:

Would still return relevant results related to the “Key Findings” section.