Back to Arena

SCM

by Beihang / NLPR (Wang et al.)

System Card

OrganizationBeihang / NLPR (Wang et al.)
Released2023-04
Architectureagentic-workflow / Agent + memory stream + memory controller
DetailsThree components: an LLM-based agent, a memory stream storing agent memories, and a memory controller that decides when and how to use memories. Plug-and-play, no fine-tuning, supports inputs up to 2M tokens.
Parameters—
Domainlong-contextagent-memoryepisodic-session
Open SourceYes
PaperView Paper
CodeRepository
plug-and-playmemory-controllerultra-long-contextdialogue

Capability Profile

Benchmark Scores

6 of 14 benchmarks
Long-Context Retrieval
1/5
RULER
no data
NIAH
no data
LooGLE
no data
LongBench
no data
∞Bench
76.225p
Multi-Turn Recall
1/2
LoCoMo
82.993p
MemoryBank
no data
Cross-Session Memory
1/1
LongMemEval
78.265p
Multi-Hop QA
2/3
BABILong
81.393p
MultiHop-RAG
no data
HotpotQA
78.492p
Agent Task Memory
1/1
AgentBench-Mem
7226p
Personalization
0/1
PerLTQA
no data
Factuality / Grounding
0/1
RAGAS
no data
Sources:SCM paper (arXiv:2304.13343); evaluated on AgentBench Memory Track (Tsinghua KEG, 2308)SCM paper (arXiv:2304.13343); evaluated on LoCoMo: Long-Term Conversational Memory Benchmark (Snap Research, 2402)SCM paper (arXiv:2304.13343); evaluated on LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (Salesforce AI Research, 2410)SCM paper (arXiv:2304.13343); evaluated on BABILong: Testing the Limits of LLMs with Long-Context Reasoning-in-a-Haystack (AIRI, 2406)SCM paper (arXiv:2304.13343); evaluated on HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (Stanford / CMU, 1809)SCM paper (arXiv:2304.13343); evaluated on InfiniteBench: Extending Long Context Evaluation Beyond 100K Tokens (Tsinghua / OpenBMB, 2402)