Back to Arena

Reflexion

by Northeastern / MIT / Princeton (Shinn et al.)

System Card

OrganizationNortheastern / MIT / Princeton (Shinn et al.)
Released2023-03
Architectureagentic-workflow / Verbal reinforcement via episodic reflection buffer
DetailsAgents verbally reflect on task feedback signals, maintaining their own reflective text in an episodic memory buffer to induce better decisions in subsequent trials. Avoids weight updates by using language as a policy encoding.
Parameters—
Domainagent-memoryepisodic-sessionlifelong-learning
Open SourceYes
PaperView Paper
CodeRepository
verbal-rlself-reflectionepisodic-bufferneurips-2023

Capability Profile

Benchmark Scores

6 of 14 benchmarks
Long-Context Retrieval
0/5
RULER
no data
NIAH
no data
LooGLE
no data
LongBench
no data
∞Bench
no data
Multi-Turn Recall
2/2
LoCoMo
81.291p
MemoryBank
78.470p
Cross-Session Memory
1/1
LongMemEval
79.272p
Multi-Hop QA
2/3
BABILong
72.725p
MultiHop-RAG
no data
HotpotQA
8096p
Agent Task Memory
1/1
AgentBench-Mem
7226p
Personalization
0/1
PerLTQA
no data
Factuality / Grounding
0/1
RAGAS
no data
Sources:arXiv:2303.11366 Figure 4c / Table 5 — CoT+Reflexion with GPT-4 and GOLD context (not retrieval). Reading comprehension setting.Reflexion paper (arXiv:2303.11366); evaluated on LoCoMo: Long-Term Conversational Memory Benchmark (Snap Research, 2402)Reflexion paper (arXiv:2303.11366); evaluated on LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (Salesforce AI Research, 2410)Reflexion paper (arXiv:2303.11366); evaluated on AgentBench Memory Track (Tsinghua KEG, 2308)Reflexion paper (arXiv:2303.11366); evaluated on MemoryBank: Enhancing LLMs with Long-Term Memory (Sun Yat-sen University, 2305)Reflexion paper (arXiv:2303.11366); evaluated on BABILong: Testing the Limits of LLMs with Long-Context Reasoning-in-a-Haystack (AIRI, 2406)