Back to Arena

REALM

by Google Research (Guu et al.)

System Card

OrganizationGoogle Research (Guu et al.)
Released2020-02
Architecturevector-rag / Latent retriever pretrained with MLM backprop
DetailsAugments LM pretraining with a latent knowledge retriever that attends over millions of documents. Retriever is trained unsupervised by masked-LM loss backpropagating through the retrieval step.
Parameters—
Domainrag-retrievalknowledge-graph
Open SourceYes
PaperView Paper
CodeRepository
icml-2020pretraininglatent-retrieverwikipedia

Capability Profile

Benchmark Scores

6 of 14 benchmarks
Data Transparency:6 estimated
Long-Context Retrieval
2/5
RULER
68.625pEstimated
NIAH
no data
LooGLE
no data
LongBench
603pEstimated
∞Bench
no data
Multi-Turn Recall
0/2
LoCoMo
no data
MemoryBank
no data
Cross-Session Memory
1/1
LongMemEval
59.19pEstimated
Multi-Hop QA
2/3
BABILong
no data
MultiHop-RAG
65.627pEstimated
HotpotQA
64.327pEstimated
Agent Task Memory
0/1
AgentBench-Mem
no data
Personalization
0/1
PerLTQA
no data
Factuality / Grounding
1/1
RAGAS
63.10pEstimated
Sources:Arena estimate — derived from capability profile, not independently verified