GraphRAG Architecture: Combining Knowledge Graphs and Vector Databases for Complex Reasoning
Vector search fails at multi-hop reasoning. Learn how Microsoft GraphRAG combines Neo4j knowledge graphs with vector embeddings to answer complex, interconnected enterprise queries.
Why Traditional Vector RAG Fails at Broad, Associative Queries
Standard Retrieval-Augmented Generation (RAG) relies on vector similarity search: chunk documents into 500-token blocks, compute embeddings, and retrieve the top-k chunks nearest to the user query in vector space.
While vector search excels at pinpoint, factual questions ('What is the return policy for Item X?'), it fails catastrophically when asked global or multi-hop synthesis questions ('What are the common failure modes across all our European logistics partners over the past three years?').
Because the answer is not contained in any single 500-token chunk, vector search retrieves random fragments, leading to incomplete or hallucinated responses. GraphRAG (Knowledge Graph + RAG) solves this limitation by modeling documents as an interconnected network of entities, relationships, and semantic communities.
In this guide, we break down the architecture of GraphRAG and how to deploy it in enterprise production.
The GraphRAG Ingestion Pipeline: Entities, Relationships, and Claims
Building a GraphRAG system requires a two-stage extraction process:
1. Entity & Relationship Extraction: An LLM analyzes source documents and extracts named entities (People, Organizations, Technologies, Locations) and directed edges between them (e.g., 'Company A -> acquired -> Company B').
2. Community Detection: Graph clustering algorithms (such as the Leiden or Louvain algorithm) group closely connected entities into thematic communities at multiple hierarchical levels.
3. Community Summaries: A summarization model generates comprehensive summaries for each community cluster, synthesizing broad architectural themes before queries ever occur.
Querying Knowledge Graphs with Cypher and Vector Blending
When a user query arrives, GraphRAG executes hybrid traversal: it performs vector search to identify candidate anchor nodes in the graph, and then executes Cypher graph queries to traverse 2-hop or 3-hop relationships.
from neo4j import GraphDatabase
driver = GraphDatabase.driver("neo4j://localhost:7687", auth=("neo4j", "password"))
def query_entity_subgraph(entity_name: str, max_depth = 2):
query = """
MATCH path = (e:Entity {name: $entity_name})-[r:RELATION*1..2]-(target:Entity)
RETURN e.name AS source,
[rel in relationships(path) | type(rel)] AS relations,
target.name AS connected_entity,
target.description AS description
LIMIT 25
"""
with driver.session() as session:
result = session.run(query, entity_name=entity_name)
return [record.data() for record in result]Local vs Global Search in GraphRAG
Microsoft's GraphRAG framework defines two complementary query modes for enterprise workloads:
Local Search: Explores entity neighborhoods for specific queries (e.g., 'Who manages the API gateway security policy?'). It combines direct node attributes with 1-hop relationship context and text chunk provenance.
Global Search: Bypasses raw document chunks entirely and searches pre-computed community summaries to answer broad organizational questions ('What are the main systemic bottlenecks discussed across all Q3 post-mortems?'). Global search utilizes a map-reduce summarization pipeline across graph clusters.
- Stage 1 (Map): Each community summary evaluates relevance to the query and produces intermediate insights with confidence scores.
- Stage 2 (Filter): Intermediate insights scoring below threshold are pruned to protect LLM context windows.
- Stage 3 (Reduce): An executive synthesis prompt unifies the surviving community insights into a coherent, cited executive report.
Evaluating Knowledge Graph Quality and Entity Resolution
A major failure mode in knowledge graphs is duplicate entity proliferation (e.g., having separate nodes for 'Apple', 'Apple Inc', and 'Apple Computer').
Implement entity resolution pipelines using embedding distance and fuzzy token matching to merge synonymous nodes into authoritative canonical entities before running community detection.
import numpy as np
def cosine_similarity(v1: np.ndarray, v2: np.ndarray) -> float:
return float(np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2)))
class EntityResolver:
def __init__(self, similarity_threshold = 0.88):
self.entities = {} # name -> {"vector": np.ndarray, "aliases": set()}
self.threshold = similarity_threshold
def resolve_or_insert(self, candidate_name: str, candidate_vec: np.ndarray) -> str:
canonical_name = candidate_name.strip().title()
# Check semantic similarity against registered canonical nodes
for existing_name, meta in self.entities.items():
sim = cosine_similarity(candidate_vec, meta["vector"])
if sim >= self.threshold:
meta["aliases"].add(candidate_name)
return existing_name # Merge into existing entity node
# Register new canonical entity
self.entities[canonical_name] = {
"vector": candidate_vec,
"aliases": {candidate_name},
}
return canonical_nameHybrid Query Routing and Reciprocal Rank Fusion (RRF)
Production systems should not execute expensive GraphRAG traversals on simple keyword lookups. Deploy an intent router at the API gateway to classify incoming requests into distinct computational tiers.
Pinpoint lookups route directly to low-cost vector search (pgvector). Associative, multi-hop queries route to GraphRAG. For ambiguous queries, execute both in parallel and merge results using Reciprocal Rank Fusion (RRF):
RRF_Score(d) = SUM(1 / (k + rank_i(d)))
This guarantees that documents confirmed by both semantic vector proximity and graph topological relevance surface at the absolute top of the agent context.
- Query Complexity Classifier: Routes requests with 0-hop lookups to pgvector and multi-hop queries to Neo4j.
- Parallel Retrieval Execution: Simultaneously gathers vector nearest neighbors and 2-hop graph paths.
- Score Harmonization: Combines disconnected rank orders without requiring raw distance normalization.
How much more does GraphRAG cost to index than traditional vector RAG?
Indexing with GraphRAG typically requires 5x to 10x more LLM tokens upfront because the model must extract entities and generate community summaries. However, inference costs are often lower and answer accuracy is significantly higher.
What database engine should be used for GraphRAG?
Neo4j is the industry standard with native vector indexing and Cypher queries. Lightweight alternatives include Memgraph and Amazon Neptune.
Can GraphRAG handle frequently updated real-time documents?
Updating graphs incrementally is challenging because re-clustering community summaries requires recalculations. Use traditional RAG for real-time streams and GraphRAG for stable, authoritative knowledge repositories.
What is the Leiden algorithm in GraphRAG?
The Leiden algorithm is a hierarchical community detection algorithm that partitions large networks into densely connected clusters without creating disconnected sub-communities.
Does GraphRAG eliminate LLM hallucinations completely?
While it does not eliminate hallucinations 100%, grounding responses in explicit relational graph edges reduces hallucination rates by over 70% compared to vanilla vector RAG.