RAGAS

Name: RAGAS: Automated Evaluation of Retrieval-Augmented Generation
Creator: Exploding Gradients
Keywords: factuality-grounding, rag-retrieval

RAGAS: Automated Evaluation of Retrieval-Augmented Generation

Benchmark Metadata

PublisherExploding Gradients

VenueEACL 2024

Evaluation Typeautomatic

Dimensions4

Test Prompts0

ScoringHigher is better

Update Frequencycontinuous

PaperView Paper

LeaderboardView Leaderboard

What It Measures

Faithfulness of generated answers to retrieved context
Answer relevance to the question
Context precision and recall
Hallucination rate

What It Does Not Measure

Cross-session consistency
Personalization
Latency

All Systems Evaluated(67 systems)

Rank	System	Score	Source
#1	Claude ProjectsAnthropic	76.5	Third-party reproduction
#2	HippoRAG 2OSU NLP Group	76.5	HippoRAG 2 paper (arXiv:2502.14802); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#3	GleanGlean Technologies	75.4	Glean vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#4	AppAgentTencent / mnotgod96	75	AppAgent paper (arXiv:2312.13771); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#5	KAGOpenSPG / Ant Group	74.3	KAG paper (arXiv:2409.13731); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#6	PineconePinecone Systems	74.2	Pinecone benchmarks
#7	HybridAGISynaLinks	74.2	HybridAGI (SynaLinks/HybridAGI); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#8	MilvusZilliz	73.4	Milvus benchmark
#9	ChatDBTsinghua University (Hu et al.)	73.4	ChatDB paper (arXiv:2306.03901); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#10	Neo4j LLM Graph BuilderNeo4j Labs	73.3	Neo4j LLM Graph Builder (neo4j-labs/llm-graph-builder); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#11	LightRAGHKUDS (HKU Data Intelligence Lab)	73.2	LightRAG paper (arXiv:2410.05779); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#12	MCP Memory ServerAnthropic / Model Context Protocol	73.2	MCP Memory Server (modelcontextprotocol/servers); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#13	GraphRAGMicrosoft	73.1	GraphRAG paper (arXiv:2404.16130); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#14	WeaviateWeaviate	73	Weaviate benchmarks
#15	StardogStardog Union Inc.	72.8	Stardog vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#16	QdrantQdrant	72.6	Qdrant benchmarks
#17	HuggingGPT / JARVISMicrosoft Research	72.3	HuggingGPT / JARVIS paper (arXiv:2303.17580); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#18	LlamaIndex MemoryLlamaIndex	72.1	LlamaIndex evals
#19	DiffbotDiffbot Inc.	72	Diffbot vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#20	MarkerDatalab (datalab-to)	71.8	Marker (datalab-to/marker); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#21	R2RSciPhi-AI	71.7	R2R (SciPhi-AI/R2R); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#22	PathRAGBUPT-GAMMA	71.6	PathRAG paper (arXiv:2502.14902); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#23	TrustRAGGoMate Community	71.5	TrustRAG (gomate-community/TrustRAG); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#24	CogneeCognee	71.4	Cognee evals
#25	AllegroGraphFranz Inc.	71.2	AllegroGraph vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#26	MiniRAGHKUDS	70.9	MiniRAG paper (arXiv:2501.06713); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#27	HippoRAGOSU NLP Group (Ohio State University)	70.8	HippoRAG paper (arXiv:2405.14831); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#28	Nano GraphRAGgusye1234	70.8	Nano GraphRAG (gusye1234/nano-graphrag); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#29	Neo4j AuraDBNeo4j Inc.	70.7	Neo4j AuraDB vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#30	DB-GPTeosphoros-ai	70.6	DB-GPT (eosphoros-ai/DB-GPT); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#31	ChromaChroma	70.5	Chroma benchmark
#32	Haystack Memorydeepset	70.4	Haystack evals
#33	GraphRAG-SDKFalkorDB	70.3	GraphRAG-SDK (FalkorDB/GraphRAG-SDK); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#34	Ontotext GraphDBOntotext / Graphwise (merged with Semantic Web Company, 2025)	69.8	Ontotext GraphDB vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#35	PaperQA2FutureHouse	69.1	PaperQA2 paper (arXiv:2409.13740); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#36	txtaiNeuML	68.9	txtai docs
#37	MarqoMarqo Pty Ltd	68	Marqo vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#38	OpenSearch VectorOpenSearch Project (AWS-led)	68	OpenSearch Vector vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#39	ParadeDBParadeDB Inc. (YC S23)	67.8	ParadeDB vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#40	Vespa AIYahoo / Vespa.ai (independent OSS project)	67.7	Vespa AI vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#41	Mixedbread AIMixedbread AI	67.5	Mixedbread AI vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#42	ValdYahoo Japan	67.5	Vald vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#43	SelfmemTsinghua / Microsoft (Cheng et al.)	67.1	Selfmem paper (arXiv:2305.02437); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#44	Cohere EmbedCohere Inc.	67	Cohere Embed vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#45	vectorizeVectorize Inc.	66.6	vectorize vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#46	AtlasMeta AI FAIR (Izacard et al.)	66.3	Atlas paper (arXiv:2208.03299); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#47	ColPaliilluin-tech	66	ColPali paper (arXiv:2407.01449); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#48	RETRODeepMind (Borgeaud et al.)	65.7	RETRO paper (arXiv:2112.04426); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#49	PrivateGPTZylon AI	65.6	PrivateGPT (zylon-ai/private-gpt); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#50	VectaraVectara Inc.	65.2	Vectara vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#51	VerbaWeaviate	65	Verba (weaviate/Verba); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#52	Manticore SearchManticore Software Ltd.	64.9	Manticore Search vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#53	MyScaleMyScale Inc.	64.7	MyScale vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#54	Supabase VectorSupabase Inc.	64.5	Supabase Vector vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#55	TurboPufferTurboPuffer Inc.	64.5	TurboPuffer vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#56	Unstructured IOUnstructured Technologies Inc.	64.5	Unstructured IO vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#57	SingleStore VectorSingleStore Inc.	64.3	SingleStore Vector vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#58	Elasticsearch VectorElastic N.V.	64.2	Elasticsearch Vector vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#59	Carbon AICarbon (acquired by Perplexity, Dec 2024)	64.1	Carbon AI vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#60	Neon VectorNeon Inc.	64	Neon Vector vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#61	LlamaCloudLlamaIndex Inc.	63.7	LlamaCloud vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#62	MendableMendable (YC-backed)	63.7	Mendable vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#63	Astra DBDataStax	63.6	Astra DB vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#64	CognitaTrueFoundry	63.6	Cognita (truefoundry/cognita); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#65	Voyage AIVoyage AI (acquired by MongoDB, Feb 2025)	63.4	Voyage AI vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#66	pgvector Supabase Neonpgvector OSS / Supabase Inc. / Neon Inc.	63.2	pgvector Supabase Neon vendor documentation; evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)
#67	REALMGoogle Research (Guu et al.)	63.1	REALM paper (arXiv:2002.08909); evaluated on RAGAS: Automated Evaluation of Retrieval-Augmented Generation (Exploding Gradients, 2309)