Back to Arena

Activation Beacon

by BAAI / Renmin University (Zhang et al.)

System Card

OrganizationBAAI / Renmin University (Zhang et al.)
Released2024-01
Architecturekv-cache-extension / Beacon tokens that compress activations at intervals
DetailsInserts special "beacon" tokens at intervals that compress keys and values of preceding segments at every layer. Plug-and-play module trained in ~10k steps on short sequences.
Parameters—
Domainlong-context
Open SourceYes
PaperView Paper
CodeRepository
iclr-2025beacon100x-extensionplug-and-play

Capability Profile

Benchmark Scores

6 of 14 benchmarks
Data Transparency:1 self-reported5 estimated
Long-Context Retrieval
5/5
RULER
66.715pEstimated
NIAH
63.18pEstimated
LooGLE
68.614pEstimated
LongBench
39.81pSelf-Reported
∞Bench
77.538pEstimated
Multi-Turn Recall
0/2
LoCoMo
no data
MemoryBank
no data
Cross-Session Memory
0/1
LongMemEval
no data
Multi-Hop QA
1/3
BABILong
61.34pEstimated
MultiHop-RAG
no data
HotpotQA
no data
Agent Task Memory
0/1
AgentBench-Mem
no data
Personalization
0/1
PerLTQA
no data
Factuality / Grounding
0/1
RAGAS
no data
Sources:arXiv:2401.03462 Table 3 — On Llama-2-7B-chat; avg of SQA 27.14, MQA 28.28, Sum 25.15, FSL 60.72, Code 57.83Arena estimate — derived from capability profile, not independently verified