MultiHop-RAG

Name: MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
Creator: HKUST
Keywords: multi-hop-qa, rag-retrieval, knowledge-graph

MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries

Benchmark Metadata

PublisherHKUST

VenueCOLM 2024

Evaluation Typeautomatic

Dimensions4

Test Prompts2,556

ScoringHigher is better

Update Frequencyannual

PaperView Paper

LeaderboardView Leaderboard

What It Measures

Inference, comparison, and temporal multi-hop queries
Retrieval recall@k for evidence chunks
Final answer accuracy

What It Does Not Measure

Personalization
Cross-session memory
Long-context window stress

All Systems Evaluated(150 systems)

Rank	System	Score	Source
#1	QdrantQdrant	75	Qdrant (qdrant/qdrant); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#2	AllegroGraphFranz Inc.	75	AllegroGraph vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#3	AppAgentTencent / mnotgod96	75	AppAgent paper (arXiv:2312.13771); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#4	Athina AIAthina AI (YC W23)	75	Athina AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#5	AutoWebGLMTHUDM	75	AutoWebGLM paper (arXiv:2404.03648); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#6	Backboard IOBackboard.io	75	Backboard IO vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#7	BotpressBotpress Inc.	75	Botpress vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#8	BrowserGymServiceNow Research	75	BrowserGym paper (arXiv:2412.05467); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#9	D-MemYou et al. (2025)	75	D-Mem paper (arXiv:2603.18631); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#10	DB-GPTeosphoros-ai	75	DB-GPT (eosphoros-ai/DB-GPT); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#11	DiffbotDiffbot Inc.	75	Diffbot vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#12	DifyLangGenius	75	Dify (langgenius/dify); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#13	GraphRAG-SDKFalkorDB	75	GraphRAG-SDK (FalkorDB/GraphRAG-SDK); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#14	HiMemZhu et al. (JD.com, 2026)	75	HiMem paper (arXiv:2601.06377); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#15	HippoRAGOSU NLP Group (Ohio State University)	75	HippoRAG paper (arXiv:2405.14831); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#16	HippoRAG 2OSU NLP Group	75	HippoRAG 2 paper (arXiv:2502.14802); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#17	HuggingGPT / JARVISMicrosoft Research	75	HuggingGPT / JARVIS paper (arXiv:2303.17580); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#18	KnowAgentzjunlp (Zhejiang University)	75	KnowAgent paper (arXiv:2403.03101); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#19	Lyzr CognisLyzr AI	75	Lyzr Cognis vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#20	MCP Memory ServerAnthropic / Model Context Protocol	75	MCP Memory Server (modelcontextprotocol/servers); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#21	MemR32025 (December submission)	75	MemR3 paper (arXiv:2512.20237); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#22	MIRIXMIRIX AI (Wang, Chen)	75	MIRIX paper (arXiv:2507.07957); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#23	MultiOnMultiOn (now AGI Inc.)	75	MultiOn vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#24	Nano GraphRAGgusye1234	75	Nano GraphRAG (gusye1234/nano-graphrag); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#25	Ontotext GraphDBOntotext / Graphwise (merged with Semantic Web Company, 2025)	75	Ontotext GraphDB vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#26	PathRAGBUPT-GAMMA	75	PathRAG paper (arXiv:2502.14902); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#27	RMMGoogle / UCSB (2025)	75	RMM paper (arXiv:2503.08026); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#28	StardogStardog Union Inc.	75	Stardog vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#29	SuperAGITransformerOptimus	75	SuperAGI (TransformerOptimus/SuperAGI); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#30	TrustRAGGoMate Community	75	TrustRAG (gomate-community/TrustRAG); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#31	Vellum AIVellum AI Inc. (YC W23)	75	Vellum AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#32	xmemoryxmemory Inc.	75	xmemory vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#33	CradleBAAI-Agents	74.7	Cradle paper (arXiv:2403.03186); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#34	HebbiaHebbia, Inc.	74.6	Hebbia vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#35	R2RSciPhi-AI	74.6	R2R (SciPhi-AI/R2R); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#36	LangGraphLangChain	74.5	LangGraph (langchain-ai/langgraph); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#37	Onyxonyx-dot-app	74.5	Onyx (onyx-dot-app/onyx); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#38	VoiceflowVoiceflow Inc.	74.4	Voiceflow vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#39	AutoGen Core MemoryMicrosoft	74.3	AutoGen Core Memory paper (arXiv:2308.08155); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#40	KAGOpenSPG / Ant Group	74.3	KAG paper (arXiv:2409.13731); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#41	LagentInternLM (Shanghai AI Lab)	74.3	Lagent (InternLM/lagent); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#42	CognigyCognigy GmbH (acquired by NICE, July 2025)	74.2	Cognigy vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#43	Neo4j AuraDBNeo4j Inc.	74.2	Neo4j AuraDB vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#44	Neo4j LLM Graph BuilderNeo4j Labs	74.2	Neo4j LLM Graph Builder (neo4j-labs/llm-graph-builder); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#45	WebVoyagerMinorJerry et al.	74.2	WebVoyager paper (arXiv:2401.13919); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#46	AutoGen StudioMicrosoft Research	74.1	AutoGen Studio vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#47	VectorShiftVectorShift Inc. (YC S23)	74.1	VectorShift vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#48	GraphRAGMicrosoft	74	GraphRAG paper (arXiv:2404.16130); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#49	LangSmith LangGraph CloudLangChain Inc.	73.9	LangSmith LangGraph Cloud vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#50	Self-RAGUniversity of Washington / Allen AI (Asai et al.)	73.9	Self-RAG paper (arXiv:2310.11511); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#51	AutoGPT PlatformSignificant Gravitas	73.7	AutoGPT Platform (Significant-Gravitas/AutoGPT); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#52	Galileo AIGalileo Technologies Inc.	73.6	Galileo AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#53	RAGFlowInfiniFlow	73.5	RAGFlow (infiniflow/ragflow); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#54	MemoryScopeAlibaba ModelScope	73.2	MemoryScope (modelscope/MemoryScope); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#55	CrewAICrewAI Inc. (Joao Moura)	73.2	CrewAI (joaomdmoura/crewAI); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#56	Swarmskyegomez / Swarms Corp	73.2	Swarms (kyegomez/swarms); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#57	Maxim AIMaxim AI Inc.	73.1	Maxim AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#58	Open InterpreterOpenInterpreter	72.9	Open Interpreter (OpenInterpreter/open-interpreter); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#59	MilvusZilliz	72.8	Milvus (milvus-io/milvus); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#60	AgentVerseOpenBMB (Tsinghua)	72.8	AgentVerse paper (arXiv:2308.10848); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#61	Dust ttDust (formerly XP1)	72.7	Dust tt vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#62	Lindy AILindy AI	72.7	Lindy AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#63	ChatDBTsinghua University (Hu et al.)	72.6	ChatDB paper (arXiv:2306.03901); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#64	AriGraphAIRI Institute / Moscow	72.5	AriGraph paper (arXiv:2407.04363); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#65	Mixedbread AIMixedbread AI	72.5	Mixedbread AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#66	PaperQA2FutureHouse	72.5	PaperQA2 paper (arXiv:2409.13740); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#67	Bishengdataelement	72.4	Bisheng (dataelement/bisheng); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#68	SID AISID (YC)	72.4	SID AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#69	SupermemorySupermemory	72.4	Supermemory (supermemoryai/memorybench); evaluated on MultiHop-RAG (HKUST, 2024)
#70	MoTFudan (Li, Qiu)	72.3	MoT paper (arXiv:2305.05181); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#71	Haystack Memorydeepset	72.1	Haystack Memory (deepset-ai/haystack); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#72	LangflowLangflow-ai (DataStax)	72	Langflow (langflow-ai/langflow); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#73	NemoriNemori AI (independent)	72	Nemori paper (arXiv:2508.03341); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#74	Qwen-AgentQwenLM (Alibaba)	72	Qwen-Agent (QwenLM/Qwen-Agent); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#75	SynapseNTU / Salesforce (Zheng et al.)	72	Synapse paper (arXiv:2306.07863); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#76	PineconePinecone Systems	71.9	Pinecone vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#77	LightRAGHKUDS (HKU Data Intelligence Lab)	71.9	LightRAG paper (arXiv:2410.05779); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#78	HoneyHiveHoneyHive Inc.	71.7	HoneyHive vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#79	FlowiseFlowiseAI	71.3	Flowise (FlowiseAI/Flowise); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#80	HybridAGISynaLinks	71.3	HybridAGI (SynaLinks/HybridAGI); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#81	LarimarIBM Research	71.3	Larimar paper (arXiv:2403.11901); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#82	MarkerDatalab (datalab-to)	71.3	Marker (datalab-to/marker); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#83	CAMELCAMEL-AI.org	71.1	CAMEL paper (arXiv:2303.17760); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#84	GleanGlean Technologies	71.1	Glean vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#85	LangMemLangChain	71	LangMem (langchain-ai/langmem); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#86	Cohere EmbedCohere Inc.	70.9	Cohere Embed vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#87	Kore AIKore.ai Inc.	70.8	Kore AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#88	WeaviateWeaviate	70.7	Weaviate (weaviate/weaviate); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#89	ChatDev 2.0OpenBMB	70.5	ChatDev 2.0 paper (arXiv:2307.07924); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#90	AGiXTJosh-XT	70.4	AGiXT (Josh-XT/AGiXT); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#91	Think-in-MemoryAnt Group / Alibaba (Liu et al.)	70.4	Think-in-Memory paper (arXiv:2311.08719); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#92	Memoripycaspianmoon	70.3	Memoripy (caspianmoon/memoripy); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#93	SynapseNanyang Technological University (Zheng et al.)	70.3	Synapse paper (arXiv:2306.07863); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#94	LlamaIndex MemoryLlamaIndex	70.2	LlamaIndex Memory (run-llama/llama_index); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#95	MetaGPTDeepWisdom / geekan	70.2	MetaGPT paper (arXiv:2308.00352); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#96	Stack AIStack AI Inc. (YC W23)	70.2	Stack AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#97	GPTeam101dotxyz	69.7	GPTeam (101dotxyz/GPTeam); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#98	Nomic AtlasNomic AI Inc.	69.7	Nomic Atlas vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#99	Mobile-AgentAlibaba Tongyi Lab (X-PLUG)	69.4	Mobile-Agent paper (arXiv:2508.15144); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#100	ChromaChroma	69.2	Chroma (chroma-core/chroma); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#101	FastGPTlabring	69.2	FastGPT (labring/FastGPT); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#102	SelfmemTsinghua / Microsoft (Cheng et al.)	69	Selfmem paper (arXiv:2305.02437); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#103	Voyage AIVoyage AI (acquired by MongoDB, Feb 2025)	69	Voyage AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#104	MiniRAGHKUDS	68.4	arXiv:2501.06713 Table 1 — Accuracy with gpt-4o-mini; SLM backbones: Phi-3.5-mini 49.96, GLM-Edge 51.41, Qwen2.5-3B 48.55
#105	MemoChatUniversity of Warwick / Alibaba	67.9	MemoChat paper (arXiv:2308.08239); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#106	MemoriGibsonAI	67.6	Memori (GibsonAI/memori); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#107	GraphitiZep AI	66.4	Zep / Graphiti paper
#108	Mnemosyneindependent	66.1	Mnemosyne paper (arXiv:2510.08601); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#109	ParadeDBParadeDB Inc. (YC S23)	65.8	ParadeDB vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#110	REALMGoogle Research (Guu et al.)	65.6	REALM paper (arXiv:2002.08909); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#111	Couchbase VectorCouchbase Inc.	65.5	Couchbase Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#112	txtaiNeuML	65.4	txtai (neuml/txtai); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#113	Elasticsearch VectorElastic N.V.	65.4	Elasticsearch Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#114	PrivateGPTZylon AI	65.3	PrivateGPT (zylon-ai/private-gpt); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#115	Mem AIMem Labs	64.9	Mem AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#116	VectaraVectara Inc.	64.9	Vectara vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#117	ZepZep AI	64.8	Zep evals
#118	Astra DBDataStax	64.6	Astra DB vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#119	Saner AISaner.AI	64.6	Saner AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#120	MarqoMarqo Pty Ltd	64.3	Marqo vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#121	MyScaleMyScale Inc.	64.1	MyScale vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#122	ColPaliilluin-tech	63.9	ColPali paper (arXiv:2407.01449); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#123	RETRODeepMind (Borgeaud et al.)	63.8	RETRO paper (arXiv:2112.04426); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#124	MemoryBankHarbin Institute of Technology / SenseTime	63.6	MemoryBank paper (arXiv:2305.10250); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#125	OpenSearch VectorOpenSearch Project (AWS-led)	63.2	OpenSearch Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#126	Neon VectorNeon Inc.	63.1	Neon Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#127	Redis VectorRedis Ltd.	63	Redis Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#128	AtlasMeta AI FAIR (Izacard et al.)	62.9	Atlas paper (arXiv:2208.03299); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#129	Carbon AICarbon (acquired by Perplexity, Dec 2024)	62.9	Carbon AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#130	CognitaTrueFoundry	62.9	Cognita (truefoundry/cognita); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#131	LlamaCloudLlamaIndex Inc.	62.9	LlamaCloud vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#132	Unstructured IOUnstructured Technologies Inc.	62.8	Unstructured IO vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#133	MendableMendable (YC-backed)	62.7	Mendable vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#134	Manticore SearchManticore Software Ltd.	62.3	Manticore Search vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#135	VerbaWeaviate	62.3	Verba (weaviate/Verba); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#136	EpsillaEpsilla Inc. (YC S23)	61.6	Epsilla vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#137	RagieRagie Inc.	61.5	Ragie vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#138	pgvector Supabase Neonpgvector OSS / Supabase Inc. / Neon Inc.	61.3	pgvector Supabase Neon vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#139	TurboPufferTurboPuffer Inc.	60.9	TurboPuffer vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#140	Supabase VectorSupabase Inc.	60.8	Supabase Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#141	Vespa AIYahoo / Vespa.ai (independent OSS project)	60.5	Vespa AI vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#142	QuivrQuivrHQ	60.4	Quivr (QuivrHQ/quivr); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#143	SingleStore VectorSingleStore Inc.	60.4	SingleStore Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#144	AnythingLLMMintplex Labs	60.3	AnythingLLM (Mintplex-Labs/anything-llm); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#145	MongoDB Atlas VectorMongoDB Inc.	60.1	MongoDB Atlas Vector vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#146	MemoroMIT Media Lab	60	Memoro paper (arXiv:2403.02135); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#147	ValdYahoo Japan	60	Vald vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#148	vectorizeVectorize Inc.	59.3	vectorize vendor documentation; evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)
#149	CogneeCognee	57.2	Cognee benchmark
#150	MemaryKingjulio8238	52.7	Memary repo evals