Back to Benchmarks
LoCoMo
LoCoMo: Long-Term Conversational Memory Benchmark
Benchmark Metadata
PublisherSnap Research
VenueACL 2024
Evaluation Typehybrid
Dimensions4
Test Prompts300
ScoringHigher is better
Update Frequencyannual
PaperView Paper
LeaderboardView Leaderboard
What It Measures
- Single-hop conversational QA
- Multi-hop conversational QA
- Temporal reasoning over dialogue
- Open-domain knowledge updates
What It Does Not Measure
- Document QA
- Code understanding
- Numerical reasoning beyond dialogue
All Systems Evaluated(161 systems)
| Rank | System | Score |
|---|---|---|
| #1 | memUNevaMind-AI | 92.1 |
| #2 | Backboard IOBackboard.io | 90 |
| #3 | MemPalaceBen Sigman / Milla Jovovich (independent open-source) | 88.9 |
| #4 | MemR32025 (December submission) | 86.8 |
| #5 | A-MEMAGI Research / Rutgers | 86.4 |
| #6 | MIRIXMIRIX AI (Wang, Chen) | 85.4 |
| #7 | xmemoryxmemory Inc. | 85.1 |
| #8 | VoyagerNVIDIA / Caltech / UT Austin / Stanford / ASU / UW (Wang et al.) | 83.6 |
| #9 | SupermemorySupermemory | 83.5 |
| #10 | ExpeLTsinghua University (Zhao et al.) | 83.1 |
| #11 | SCMBeihang / NLPR (Wang et al.) | 82.9 |
| #12 | CrewAI EnterpriseCrewAI Inc. | 81.8 |
| #13 | Talkie AIMiniMax | 81.8 |
| #14 | ReflexionNortheastern / MIT / Princeton (Shinn et al.) | 81.2 |
| #15 | Agent Workflow MemoryCMU (Wang, Mao, Fried, Neubig) | 81 |
| #16 | CognigyCognigy GmbH (acquired by NICE, July 2025) | 80.9 |
| #17 | HiMemZhu et al. (JD.com, 2026) | 80.7 |
| #18 | Nabla CopilotNabla | 80.7 |
| #19 | DB-GPTeosphoros-ai | 80.6 |
| #20 | BrowserGymServiceNow Research | 80.3 |
| #21 | FastGPTlabring | 80.1 |
| #22 | OS-Copilot / FRIDAYShanghai AI Lab / MMLab (Wu et al.) | 80 |
| #23 | Nomi AIGlimpse AI, Inc. | 79.9 |
| #24 | AutoGPT PlatformSignificant Gravitas | 79.7 |
| #25 | AutoGen StudioMicrosoft Research | 79.6 |
| #26 | Qwen-AgentQwenLM (Alibaba) | 79.5 |
| #27 | AbridgeAbridge | 79.4 |
| #28 | NemoriNemori AI (independent) | 79.4 |
| #29 | Swarmskyegomez / Swarms Corp | 79.4 |
| #30 | CrewAICrewAI Inc. (Joao Moura) | 79.3 |
| #31 | Bishengdataelement | 78.9 |
| #32 | DifyLangGenius | 78.9 |
| #33 | Adept AIAdept AI Labs (acquired by Amazon 2024) | 78.6 |
| #34 | SID AISID (YC) | 78.5 |
| #35 | D-MemYou et al. (2025) | 78.4 |
| #36 | Plaud NotePLAUD | 78.4 |
| #37 | Lyzr CognisLyzr AI | 78.3 |
| #38 | RecallMCisco Research / independent (Kynoch & Latapie) | 78.2 |
| #39 | VoiceflowVoiceflow Inc. | 78.2 |
| #40 | BotpressBotpress Inc. | 78.1 |
| #41 | HoneyHiveHoneyHive Inc. | 77.9 |
| #42 | MetaGPTDeepWisdom / geekan | 77.9 |
| #43 | GAMVectorSpaceLab (BAAI-related) | 77.8 |
| #44 | KindroidKindroid | 77.8 |
| #45 | Self-RAGUniversity of Washington / Allen AI (Asai et al.) | 77.6 |
| #46 | Dust ttDust (formerly XP1) | 77.5 |
| #47 | Generative AgentsStanford University / Google Research | 77.5 |
| #48 | SynapseNanyang Technological University (Zheng et al.) | 77.5 |
| #49 | AutoGen Core MemoryMicrosoft | 77.4 |
| #50 | MultiOnMultiOn (now AGI Inc.) | 77.4 |
| #51 | Athina AIAthina AI (YC W23) | 77.2 |
| #52 | CAMELCAMEL-AI.org | 77.2 |
| #53 | AGiXTJosh-XT | 76.9 |
| #54 | Galileo AIGalileo Technologies Inc. | 76.8 |
| #55 | Suki AISuki (formerly Robin AI) | 76.8 |
| #56 | HEMAindependent (Ahn et al.) | 76.7 |
| #57 | MempZhejiang University (Fang et al.) | 76.6 |
| #58 | WebVoyagerMinorJerry et al. | 76.6 |
| #59 | LangflowLangflow-ai (DataStax) | 76.4 |
| #60 | ParadotWithFeeling.AI | 76.4 |
| #61 | MemoryLLMUCSD / Apple (Wang et al.) | 76.3 |
| #62 | Nuance DAXNuance Communications (Microsoft) | 76.2 |
| #63 | SuperAGITransformerOptimus | 76.2 |
| #64 | ChatDev 2.0OpenBMB | 76.1 |
| #65 | HebbiaHebbia, Inc. | 76.1 |
| #66 | LangGraphLangChain | 76 |
| #67 | Onyxonyx-dot-app | 76 |
| #68 | MemOSMemTensor (Li, Zhang, et al.) | 75.8 |
| #69 | Pickle AISoul Computer (YC-backed) | 75.8 |
| #70 | Stack AIStack AI Inc. (YC W23) | 75.8 |
| #71 | LangSmith LangGraph CloudLangChain Inc. | 75.7 |
| #72 | Pi InflectionInflection AI | 75.7 |
| #73 | BabyAGIYohei Nakajima | 75.6 |
| #74 | ArcMemoUC Berkeley / Stanford (Ho et al.) | 75.5 |
| #75 | Kore AIKore.ai Inc. | 75.5 |
| #76 | Limitless PendantLimitless AI (acquired by Meta Dec 2025) | 75.5 |
| #77 | ReplikaLuka, Inc. | 75.5 |
| #78 | Think-in-MemoryAnt Group / Alibaba (Liu et al.) | 75.5 |
| #79 | GPTeam101dotxyz | 75.4 |
| #80 | MCP Memory ServerAnthropic / Model Context Protocol | 75.3 |
| #81 | MoTFudan University (Li & Qiu) | 75.3 |
| #82 | Open InterpreterOpenInterpreter | 75.3 |
| #83 | AgentVerseOpenBMB (Tsinghua) | 75.1 |
| #84 | Memoripycaspianmoon | 75.1 |
| #85 | AgentScopeModelScope (Alibaba) | 75 |
| #86 | Granola AIGranola | 74.9 |
| #87 | VectorShiftVectorShift Inc. (YC S23) | 74.9 |
| #88 | HippoRAG 2OSU NLP Group | 74.8 |
| #89 | LagentInternLM (Shanghai AI Lab) | 74.6 |
| #90 | RAGFlowInfiniFlow | 74.6 |
| #91 | Vellum AIVellum AI Inc. (YC W23) | 74.6 |
| #92 | CradleBAAI-Agents | 74.5 |
| #93 | AutoWebGLMTHUDM | 74.4 |
| #94 | Maxim AIMaxim AI Inc. | 74.4 |
| #95 | HippoRAGOSU NLP Group (Ohio State University) | 74.1 |
| #96 | HuggingGPT / JARVISMicrosoft Research | 74 |
| #97 | Lindy AILindy AI | 74 |
| #98 | Mobile-AgentAlibaba Tongyi Lab (X-PLUG) | 74 |
| #99 | FlowiseFlowiseAI | 73.9 |
| #100 | KnowAgentzjunlp (Zhejiang University) | 73.7 |
| #101 | MoTFudan (Li, Qiu) | 73.5 |
| #102 | AppAgentTencent / mnotgod96 | 73.1 |
| #103 | Bee ComputerBee (acquired by Amazon 2026) | 73 |
| #104 | ReMeModelScope (Alibaba) | 73 |
| #105 | RMMGoogle / UCSB (2025) | 72.7 |
| #106 | ChatDBTsinghua University (Hu et al.) | 72.5 |
| #107 | MemoChatUniversity of Warwick / Alibaba | 72.5 |
| #108 | AriGraphAIRI Institute / Moscow | 72.3 |
| #109 | Personal AIPersonal AI | 72.3 |
| #110 | Generative AgentsStanford / Google | 72 |
| #111 | Second MeMindverse (Shang, Li, et al.) | 71.9 |
| #112 | SynapseNTU / Salesforce (Zheng et al.) | 71.8 |
| #113 | MemaryKingjulio8238 | 71.7 |
| #114 | Character AICharacter.AI (Google investment) | 71.7 |
| #115 | JARVIS-1CraftJarvis | 71.7 |
| #116 | Tab AITab (Avi Schiffmann) | 71.7 |
| #117 | EM-LLMem-llm (academic consortium) | 71.6 |
| #118 | ZepZep AI | 71.3 |
| #119 | Neo4j AuraDBNeo4j Inc. | 70.5 |
| #120 | Friend AIFriend | 70 |
| #121 | Charlie MnemonicGoodAI | 69.9 |
| #122 | HybridAGISynaLinks | 69.3 |
| #123 | Nomic AtlasNomic AI Inc. | 69.3 |
| #124 | Memorizing TransformerGoogle Research (Wu, Rabe, Hutchins, Szegedy) | 68.9 |
| #125 | Titanslucidrains (community) / paper by Google Research | 68.6 |
| #126 | Mem0Mem0 | 68.5 |
| #127 | GleanGlean Technologies | 68 |
| #128 | MemoryScopeAlibaba ModelScope | 66.6 |
| #129 | Claude ProjectsAnthropic | 66.6 |
| #130 | LarimarIBM Research | 66.4 |
| #131 | Gemini MemoryGoogle | 65.8 |
| #132 | LongMemUCSB / Microsoft Research | 65.8 |
| #133 | MemformerUC Santa Barbara / Amazon (Wu, Lan, Liu, et al.) | 65.7 |
| #134 | kNN-LMStanford / Facebook AI Research (Khandelwal et al.) | 64.9 |
| #135 | LettaLetta (formerly MemGPT) | 64.2 |
| #136 | Haystack Memorydeepset | 62.8 |
| #137 | MemoryBankInstitute of Software, Chinese Academy of Sciences | 62.3 |
| #138 | R3MemHKUST (2025) | 62.3 |
| #139 | LangMemLangChain | 61 |
| #140 | Copilot MemoryMicrosoft | 60.7 |
| #141 | ChatGPT MemoryOpenAI | 60.3 |
| #142 | Couchbase VectorCouchbase Inc. | 60 |
| #143 | Redis VectorRedis Ltd. | 59.2 |
| #144 | MemoryBankHarbin Institute of Technology / SenseTime | 59.1 |
| #145 | Notion AINotion Labs | 58.7 |
| #146 | KDB AIKX Systems | 58.2 |
| #147 | RagieRagie Inc. | 57.8 |
| #148 | Saner AISaner.AI | 57.7 |
| #149 | AnythingLLMMintplex Labs | 57.1 |
| #150 | MemGPT ClassicBerkeley / Letta | 56.8 |
| #151 | LlamaIndex MemoryLlamaIndex | 56.5 |
| #152 | Mem AIMem Labs | 56.3 |
| #153 | MongoDB Atlas VectorMongoDB Inc. | 55.6 |
| #154 | QuivrQuivrHQ | 55.3 |
| #155 | MemoroMIT Media Lab | 55.1 |
| #156 | Heyday AIHeyday (shut down 2025) | 54.9 |
| #157 | Sana AISana Labs | 54.9 |
| #158 | MemoriGibsonAI | 54.8 |
| #159 | Mnemosyneindependent | 54.5 |
| #160 | MnemosyneJohns Hopkins / independent (2025) | 54.5 |
| #161 | EpsillaEpsilla Inc. (YC S23) | 53.3 |