BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering
cmavro
Abstract
BYOKG-RAG combines LLMs with specialized graph retrieval tools to enhance KGQA, improving generalization and performance over custom knowledge graphs.
Knowledge graph question answering (KGQA) presents significant challenges due to the structural and semantic variations across input graphs. Existing works rely on Large Language Model (LLM) agents for graph traversal and retrieval; an approach that is sensitive to traversal initialization, as it is prone to entity linking errors and may not generalize well to custom ("bring-your-own") KGs. We introduce BYOKG-RAG, a framework that enhances KGQA by synergistically combining LLMs with specialized graph retrieval tools. In BYOKG-RAG, LLMs generate critical graph artifacts (question entities, candidate answers, reasoning paths, and OpenCypher queries), and graph tools link these artifacts to the KG and retrieve relevant graph context. The retrieved context enables the LLM to iteratively refine its graph linking and retrieval, before final answer generation. By retrieving context from different graph tools, BYOKG-RAG offers a more general and robust solution for QA over custom KGs. Through experiments on five benchmarks spanning diverse KG types, we demonstrate that BYOKG-RAG outperforms the second-best graph retrieval method by 4.5% points while showing better generalization to custom KGs. BYOKG-RAG framework is open-sourced at https://github.com/awslabs/graphrag-toolkit.
Community
BYOKG-RAG is a novel approach to Knowledge Graph Question Answering (KGQA) that combines the power of Large Language Models (LLMs) with structured knowledge graphs. The system allows users to bring their own knowledge graph and perform complex question answering over it.
Key Features 🔑
- Multi-strategy Retrieval: Combines multiple retrieval strategies:
- Agentic retrieval for dynamic graph exploration
- Scoring-based retrieval for relevance ranking
- Path-based retrieval for multi-hop reasoning
- Query-based retrieval for direct graph queries
- LLM-powered Reasoning: Leverages state-of-the-art LLMs for question understanding and answer generation
System Components ⚙️
Graph Store (src/graphrag_toolkit/byokg_rag/graphstore)
- Manages the knowledge graph data structure
- Provides interfaces for graph traversal and querying
KG Linker (src/graphrag_toolkit/byokg_rag/graph_connectors)
- Links natural language queries to graph entities and paths
- Uses LLMs to understand question intent
- Extracts relevant entities and relationship patterns
Entity Linker (src/graphrag_toolkit/byokg_rag/graph_retrievers)
- Matches entities from text to graph nodes
- Handles variations in entity names
- Uses fuzzy string matching for robust entity resolution
Triplet Retriever (src/graphrag_toolkit/byokg_rag/graph_retrievers)
- Retrieves relevant triplets from the graph
- Navigates the graph starting from linked entities
- Verbalizes triplets in natural language format
Path Retriever (src/graphrag_toolkit/byokg_rag/graph_retrievers)
- Finds paths between entities in the graph
- Follows metapath patterns for structured traversal
- Connects entities through intermediate nodes
Query Engine (src/graphrag_toolkit/byokg_rag/byokg_query_engine.py)
- Orchestrates all components
- Processes natural language questions
- Generates answers based on retrieved information
Performance 📈
Our results show that BYOKG-RAG outperforms existing approaches across multiple knowledge graph benchmarks:
| KGQA Hit (%) | Wiki-KG | Temp-KG | Med-KG |
|---|---|---|---|
| Agent | 77.8 | 57.3 | 59.2 |
| BYOKG-RAG | 80.1 | 65.5 | 65.0 |
Note: Full paper with detailed methodology and results coming soon! 📄
Getting Started 🚀
You can install the byokg-rag using pip:
pip install https://github.com/awslabs/graphrag-toolkit/archive/refs/tags/v3.10.1.zip#subdirectory=byokg-rag
Paper: https://arxiv.org/abs/2507.04127
Code: https://github.com/awslabs/graphrag-toolkit
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper