Back to Arena

Self-RAG

by University of Washington / Allen AI (Asai et al.)

System Card

OrganizationUniversity of Washington / Allen AI (Asai et al.)
Released2023-10
Architectureagentic-workflow / Self-reflective on-demand retrieval with reflection tokens
DetailsTrains a single LM that adaptively decides when to retrieve, then emits reflection tokens to critique retrieved passages and its own generations. Reflection tokens make the LM controllable at inference time.
Parameters—
Domainrag-retrievalagent-memory
Open SourceYes
PaperView Paper
WebsiteVisit
CodeRepository
iclr-2024-oralreflection-tokensadaptive-retrievalfactuality

Capability Profile

Benchmark Scores

6 of 14 benchmarks
Long-Context Retrieval
1/5
RULER
no data
NIAH
no data
LooGLE
no data
LongBench
603p
∞Bench
no data
Multi-Turn Recall
1/2
LoCoMo
77.672p
MemoryBank
no data
Cross-Session Memory
1/1
LongMemEval
79.171p
Multi-Hop QA
2/3
BABILong
no data
MultiHop-RAG
73.967p
HotpotQA
77.889p
Agent Task Memory
1/1
AgentBench-Mem
7226p
Personalization
0/1
PerLTQA
no data
Factuality / Grounding
0/1
RAGAS
no data
Sources:Self-RAG paper (arXiv:2310.11511); evaluated on HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (Stanford / CMU, 1809)Self-RAG paper (arXiv:2310.11511); evaluated on MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (HKUST, 2401)Self-RAG paper (arXiv:2310.11511); evaluated on AgentBench Memory Track (Tsinghua KEG, 2308)Self-RAG paper (arXiv:2310.11511); evaluated on LoCoMo: Long-Term Conversational Memory Benchmark (Snap Research, 2402)Self-RAG paper (arXiv:2310.11511); evaluated on LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding (Tsinghua KEG, 2308)Self-RAG paper (arXiv:2310.11511); evaluated on LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (Salesforce AI Research, 2410)