RAG Background.
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating retrieval mechanisms with generative capabilities. It improves response accuracy by accessing external databases for relevant information, overcoming LLM limitations in knowledge cut-off and hallucinations. RAG combines the strengths of retrieval (precise, up-to-date data) and generation (contextual, fluent language), making it essential for complex queries, factual correctness, and dynamic knowledge. It optimizes performance, especially in specialized or rapidly evolving fields, ensuring comprehensive, accurate, and contextually relevant outputs, thus significantly enhancing the utility and reliability of LLMs in practical applications.
In this tutorial, I delve into the key topics and advancements related to Retrieval-Augmented Generation (RAG) using executable codes. Each topic is meticulously explained through video tutorials, offering detailed yet accessible discussions and working demonstrations, accompanied by the corresponding code. Some code segments are sourced from relevant libraries for demonstration purposes. This comprehensive tutorial covers the following topics, providing an in-depth understanding of RAG and its practical applications.
- Basics of RAG (Retrieval Augmented Generation) with LLM
- How to use LLM + RAG to Construct Knowledge Graph.
- How to construct Flow-Diagram by Using LLM + RAG.
- Graph Based RAG (Retrieval Augmented Generation) Techniques.
Note: In addition to this, it also provides a linked tutorial on a pressing topic: "Use of Long Text Sequences with LLMs Trained on Shorter Text Sequences.". In the future, I will introduce new research advancements in the field of Large Language Models.
1. Basics of RAG (Retrieval Augmented Generation) with LLM.
Video Tutorial.
Basic RAG Code.
import ollama
import chromadb
documents = [
"Quantum mechanics is a fundamental theory in physics that describes the behavior of nature at and below the scale of atoms.",
"It is the foundation of all quantum physics, which includes quantum chemistry, quantum field theory, quantum technology, and quantum information science.",
"Quantum mechanics can describe many systems that classical physics cannot.",
"Classical physics can describe many aspects of nature at an ordinary (macroscopic and (optical) microscopic) scale, but is not sufficient for describing them at very small submicroscopic (atomic and subatomic) scales.",
"Most theories in classical physics can be derived from quantum mechanics as an approximation valid at large (macroscopic/microscopic) scale.",
"Quantum systems have bound states that are quantized to discrete values of energy, momentum, angular momentum, and other quantities, in contrast to classical systems where these quantities can be measured continuously.",
"Measurements of quantum systems show characteristics of both particles and waves (wave–particle duality), and there are limits to how accurately the value of a physical quantity can be predicted prior to its measurement, given a complete set of initial conditions (the uncertainty principle)."
]
# Create database
client = chromadb.Client()
collection = client.create_collection(name="docs")
# store each document in a vector embedding database
for i, d in enumerate(documents):
response = ollama.embeddings(model="mxbai-embed-large", prompt=d)
embedding = response["embedding"]
collection.add(
ids=[str(i)],
embeddings=[embedding],
documents=[d]
)
# an example prompt
prompt = "What are the key benefits of using quantum mechanics over classical physics?"
# generate an embedding for the prompt and retrieve the most relevant doc
response = ollama.embeddings(
prompt=prompt,
model="mxbai-embed-large"
)
results = collection.query(
query_embeddings=[response["embedding"]],
n_results=1
)
data = results['documents'][0][0]
# generate a response combining the prompt and data we retrieved in step 2
output = ollama.generate(
model="llama3",
prompt=f"Using this data: {data}. Respond to this prompt: {prompt}"
)
print(output['response'])
2. How to use LLM + RAG to Construct Knowledge Graph..
Video Tutorial.
Code to Generate Knowledge Graph Triplets.
import ollama
import chromadb
documents = [
"Quantum mechanics is a fundamental theory in physics that describes the behavior of nature at and below the scale of atoms.",
"It is the foundation of all quantum physics, which includes quantum chemistry, quantum field theory, quantum technology, and quantum information science.",
"Quantum mechanics can describe many systems that classical physics cannot.",
"Classical physics can describe many aspects of nature at an ordinary (macroscopic and (optical) microscopic) scale, but is not sufficient for describing them at very small submicroscopic (atomic and subatomic) scales.",
"Most theories in classical physics can be derived from quantum mechanics as an approximation valid at large (macroscopic/microscopic) scale.",
"Quantum systems have bound states that are quantized to discrete values of energy, momentum, angular momentum, and other quantities, in contrast to classical systems where these quantities can be measured continuously.",
"Measurements of quantum systems show characteristics of both particles and waves (wave–particle duality), and there are limits to how accurately the value of a physical quantity can be predicted prior to its measurement, given a complete set of initial conditions (the uncertainty principle)."
]
# Create database
client = chromadb.Client()
collection = client.create_collection(name="docs")
# store each document in a vector embedding database
for i, d in enumerate(documents):
response = ollama.embeddings(model="mxbai-embed-large", prompt=d)
embedding = response["embedding"]
collection.add(ids=[str(i)], embeddings=[embedding], documents=[d] )
# an example prompt
prompt1 = "What are the key benefits of using quantum mechanics over classical physics?"
prompt2 = "List all entities, and generate the knowledge graph triplets by using all entities."
# Generate Answers - for Prompt-1:
# generate an embedding for the prompt and retrieve the most relevant doc
response1 = ollama.embeddings(
prompt=prompt1,
model="mxbai-embed-large"
)
results1 = collection.query(
query_embeddings=[response1["embedding"]],
n_results=1
)
data1 = results1['documents'][0][0]
# generate a response combining the prompt and data we retrieved in step 2
output1 = ollama.generate(
model="llama3",
prompt=f"Using this data: {data1}. Respond to this prompt: {prompt1}"
)
print("Response for the question -1",output1['response'])
# Generate Answers - for Prompt-2:
# generate an embedding for the prompt and retrieve the most relevant doc
response2 = ollama.embeddings(
prompt=prompt2,
model="mxbai-embed-large"
)
results2 = collection.query(
query_embeddings=[response2["embedding"]],
n_results=1
)
data2 = results2['documents'][0][0]
# generate a response combining the prompt and data we retrieved in step 2
output2 = ollama.generate(
model="llama3",
prompt=f"Using this data: {data2}. Respond to this prompt: {prompt2}"
)
print("Response for the question -2",output2['response'])
Code to Visualize the Knowledge Graph (by using above triplets).
import networkx as nx
import matplotlib.pyplot as plt
# Step 1: Define your triplets
triplets = [
("Quantum mechanics", "a fundamental theory", "Theory"),
("Theory", "in physics", "Physics"),
("Physics", "describes the behavior of", "Nature"),
("Nature", "at and below the scale of", "Atoms"),
("Atoms", "is related to the scale of", "Scale"),
]
# Step 2: Create a directed graph
G = nx.DiGraph()
# Step 3: Add edges from triplets
for subject, predicate, obj in triplets:
G.add_edge(subject, obj, label=predicate)
# Step 4: Draw the graph
pos = nx.spring_layout(G, seed=42) # Position nodes using Fruchterman-Reingold force-directed algorithm
# Draw nodes and edges
nx.draw(G, pos, with_labels=True, node_size=3000, node_color="lightblue", font_size=10, font_weight="bold", arrowsize=20)
# Draw edge labels
edge_labels = nx.get_edge_attributes(G, 'label')
nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, font_color='red')
# Display the graph
plt.title("Knowledge Graph Visualization")
plt.show()
3. How to construct Flow-Diagram by Using LLM + RAG.
Video Tutorial.
Code.
import ollama
import chromadb
documents = [
"Quantum mechanics is a fundamental theory in physics that describes the behavior of nature at and below the scale of atoms.",
"It is the foundation of all quantum physics, which includes quantum chemistry, quantum field theory, quantum technology, and quantum information science.",
"Quantum mechanics can describe many systems that classical physics cannot.",
"Classical physics can describe many aspects of nature at an ordinary (macroscopic and (optical) microscopic) scale, but is not sufficient for describing them at very small submicroscopic (atomic and subatomic) scales.",
"Most theories in classical physics can be derived from quantum mechanics as an approximation valid at large (macroscopic/microscopic) scale.",
"Quantum systems have bound states that are quantized to discrete values of energy, momentum, angular momentum, and other quantities, in contrast to classical systems where these quantities can be measured continuously.",
"Measurements of quantum systems show characteristics of both particles and waves (wave–particle duality), and there are limits to how accurately the value of a physical quantity can be predicted prior to its measurement, given a complete set of initial conditions (the uncertainty principle)."
]
# Create database
client = chromadb.Client()
collection = client.create_collection(name="docs")
# store each document in a vector embedding database
for i, d in enumerate(documents):
response = ollama.embeddings(model="mxbai-embed-large", prompt=d)
embedding = response["embedding"]
collection.add(ids=[str(i)], embeddings=[embedding], documents=[d] )
# an example prompt
prompt1 = "Generate a Mermaid diagram."
# Generate Answers - for Prompt-1:
# generate an embedding for the prompt and retrieve the most relevant doc
response1 = ollama.embeddings(
prompt=prompt1,
model="mxbai-embed-large"
)
results1 = collection.query(
query_embeddings=[response1["embedding"]],
n_results=1
)
data1 = results1['documents'][0][0]
# generate a response combining the prompt and data we retrieved in step 2
output1 = ollama.generate(
model="llama3",
prompt=f"Using this data: {data1}. Respond to this prompt: {prompt1}"
)
print("Response for the question -1",output1['response'])
4. Graph Based RAG (Retrieval Augmented Generation) Techniques.
Video Tutorial.
Code.
import ollama
import chromadb
documents = [
"The use of retrieval-augmented generation (RAG) to retrieve relevant information from an external knowledge source enables large language models (LLMs) to answer questions over private and/or previously unseen document collections.",
"However, RAG fails on global questions directed at an entire text corpus, such as “What are the main themes in the dataset?”, since this is inherently a queryfocused summarization (QFS) task, rather than an explicit retrieval task.",
"Prior QFS methods, meanwhile, fail to scale to the quantities of text indexed by typical RAG systems.",
"To combine the strengths of these contrasting methods, we propose a Graph RAG approach to question answering over private text corpora that scales with both the generality of user questions and the quantity of source text to be indexed.",
"Our approach uses an LLM to build a graph-based text index in two stages: first to derive an entity knowledge graph from the source documents, then to pregenerate community summaries for all groups of closely-related entities.",
"Given a question, each community summary is used to generate a partial response, before all partial responses are again summarized in a final response to the user.",
"For a class of global sensemaking questions over datasets in the 1 million token range, we show that Graph RAG leads to substantial improvements over a na¨ıve RAG baseline for both the comprehensiveness and diversity of generated answers."
]
# Create database
client = chromadb.Client()
collection = client.create_collection(name="docs")
# store each document in a vector embedding database
for i, d in enumerate(documents):
response = ollama.embeddings(model="mxbai-embed-large", prompt=d)
embedding = response["embedding"]
collection.add(
ids=[str(i)],
embeddings=[embedding],
documents=[d]
)
# an example prompt
prompt = "How Graph RAG (Retrieval Augmented Generation, used with Large Language Model) generates a Global Summarization for the given context?"
Entity_1 = "Graph RAG"
Entity_2 = "Global Summarization"
Community_Triplets = [("Graph RAG", "Uses", "Leiden Community Detection Algorithm"),
("LLM", "Extracts", "Entity Knowledge Graph"),
("Graph Index", "Partitioned By", "Community Detection Algorithms"),
("Community Summaries", "Used For", "Global Summarization"),]
summary_text = "Graph RAG - Uses - Leiden Community Detection Algorithm; LLM - Extracts - Entity Knowledge Graph; Graph Index - Partitioned By - Community Detection Algorithms; Community Summaries - Used For - Global Summarization"
# generate an embedding for the prompt and retrieve the most relevant doc
response = ollama.embeddings(
prompt=prompt,
model="mxbai-embed-large"
)
summary_text_embedding = ollama.embeddings(
prompt=summary_text,
model="mxbai-embed-large"
)
results = collection.query(
query_embeddings=[summary_text_embedding["embedding"]],
n_results=1
)
data = results['documents'][0][0]
# generate a response combining the prompt and data we retrieved in step 2
output = ollama.generate(
model="llama3",
prompt=f"Using this data: {data}. Respond to this prompt: {prompt}"
)
print(output['response'])
Additional Reference for the topic.
".Use of Long Text Sequences with LLMs Trained on Shorter Text Sequences." (Or use the direct link: https://www.nirajai.com/home/llm )
Reference.
- Ding, Yujuan, Wenqi Fan, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. "A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models." arXiv preprint arXiv:2405.06211 (2024).
- Wu, Kevin, Eric Wu, and James Zou. "How faithful are RAG models? Quantifying the tug-of-war between RAG and LLMs' internal prior." arXiv preprint arXiv:2404.10198 (2024).
- Li, Jiarui, Ye Yuan, and Zehua Zhang. "Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases." arXiv preprint arXiv:2403.10446 (2024).
- Edge, Darren, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. "From Local to Global: A Graph RAG Approach to Query-Focused Summarization." arXiv preprint arXiv:2404.16130 (2024).
