Back to Benchmarks

LoCoMo

LoCoMo: Long-Term Conversational Memory Benchmark

Benchmark Metadata

PublisherSnap Research
VenueACL 2024
Evaluation Typehybrid
Dimensions4
Test Prompts300
ScoringHigher is better
Update Frequencyannual
PaperView Paper
LeaderboardView Leaderboard

What It Measures

  • Single-hop conversational QA
  • Multi-hop conversational QA
  • Temporal reasoning over dialogue
  • Open-domain knowledge updates

What It Does Not Measure

  • Document QA
  • Code understanding
  • Numerical reasoning beyond dialogue

All Systems Evaluated(161 systems)

16 self-reported145 estimated
RankSystemScore
#1memUNevaMind-AI92.1
#2Backboard IOBackboard.io90
#3MemPalaceBen Sigman / Milla Jovovich (independent open-source)88.9
#4MemR32025 (December submission)86.8
#5A-MEMAGI Research / Rutgers86.4
#6MIRIXMIRIX AI (Wang, Chen)85.4
#7xmemoryxmemory Inc.85.1
#8VoyagerNVIDIA / Caltech / UT Austin / Stanford / ASU / UW (Wang et al.)83.6
#9SupermemorySupermemory83.5
#10ExpeLTsinghua University (Zhao et al.)83.1
#11SCMBeihang / NLPR (Wang et al.)82.9
#12CrewAI EnterpriseCrewAI Inc.81.8
#13Talkie AIMiniMax81.8
#14ReflexionNortheastern / MIT / Princeton (Shinn et al.)81.2
#15Agent Workflow MemoryCMU (Wang, Mao, Fried, Neubig)81
#16CognigyCognigy GmbH (acquired by NICE, July 2025)80.9
#17HiMemZhu et al. (JD.com, 2026)80.7
#18Nabla CopilotNabla80.7
#19DB-GPTeosphoros-ai80.6
#20BrowserGymServiceNow Research80.3
#21FastGPTlabring80.1
#22OS-Copilot / FRIDAYShanghai AI Lab / MMLab (Wu et al.)80
#23Nomi AIGlimpse AI, Inc.79.9
#24AutoGPT PlatformSignificant Gravitas79.7
#25AutoGen StudioMicrosoft Research79.6
#26Qwen-AgentQwenLM (Alibaba)79.5
#27AbridgeAbridge79.4
#28NemoriNemori AI (independent)79.4
#29Swarmskyegomez / Swarms Corp79.4
#30CrewAICrewAI Inc. (Joao Moura)79.3
#31Bishengdataelement78.9
#32DifyLangGenius78.9
#33Adept AIAdept AI Labs (acquired by Amazon 2024)78.6
#34SID AISID (YC)78.5
#35D-MemYou et al. (2025)78.4
#36Plaud NotePLAUD78.4
#37RecallMCisco Research / independent (Kynoch & Latapie)78.2
#38VoiceflowVoiceflow Inc.78.2
#39BotpressBotpress Inc.78.1
#40HoneyHiveHoneyHive Inc.77.9
#41MetaGPTDeepWisdom / geekan77.9
#42GAMVectorSpaceLab (BAAI-related)77.8
#43KindroidKindroid77.8
#44Self-RAGUniversity of Washington / Allen AI (Asai et al.)77.6
#45Dust ttDust (formerly XP1)77.5
#46Generative AgentsStanford University / Google Research77.5
#47SynapseNanyang Technological University (Zheng et al.)77.5
#48AutoGen Core MemoryMicrosoft77.4
#49MultiOnMultiOn (now AGI Inc.)77.4
#50Athina AIAthina AI (YC W23)77.2
#51CAMELCAMEL-AI.org77.2
#52AGiXTJosh-XT76.9
#53Galileo AIGalileo Technologies Inc.76.8
#54Suki AISuki (formerly Robin AI)76.8
#55HEMAindependent (Ahn et al.)76.7
#56MempZhejiang University (Fang et al.)76.6
#57WebVoyagerMinorJerry et al.76.6
#58LangflowLangflow-ai (DataStax)76.4
#59ParadotWithFeeling.AI76.4
#60MemoryLLMUCSD / Apple (Wang et al.)76.3
#61Nuance DAXNuance Communications (Microsoft)76.2
#62SuperAGITransformerOptimus76.2
#63ChatDev 2.0OpenBMB76.1
#64HebbiaHebbia, Inc.76.1
#65LangGraphLangChain76
#66Onyxonyx-dot-app76
#67MemOSMemTensor (Li, Zhang, et al.)75.8
#68Pickle AISoul Computer (YC-backed)75.8
#69Stack AIStack AI Inc. (YC W23)75.8
#70LangSmith LangGraph CloudLangChain Inc.75.7
#71Pi InflectionInflection AI75.7
#72BabyAGIYohei Nakajima75.6
#73ArcMemoUC Berkeley / Stanford (Ho et al.)75.5
#74Kore AIKore.ai Inc.75.5
#75Limitless PendantLimitless AI (acquired by Meta Dec 2025)75.5
#76ReplikaLuka, Inc.75.5
#77Think-in-MemoryAnt Group / Alibaba (Liu et al.)75.5
#78GPTeam101dotxyz75.4
#79MCP Memory ServerAnthropic / Model Context Protocol75.3
#80MoTFudan University (Li & Qiu)75.3
#81Open InterpreterOpenInterpreter75.3
#82AgentVerseOpenBMB (Tsinghua)75.1
#83Memoripycaspianmoon75.1
#84AgentScopeModelScope (Alibaba)75
#85Granola AIGranola74.9
#86VectorShiftVectorShift Inc. (YC S23)74.9
#87HippoRAG 2OSU NLP Group74.8
#88LagentInternLM (Shanghai AI Lab)74.6
#89RAGFlowInfiniFlow74.6
#90Vellum AIVellum AI Inc. (YC W23)74.6
#91CradleBAAI-Agents74.5
#92AutoWebGLMTHUDM74.4
#93Maxim AIMaxim AI Inc.74.4
#94HippoRAGOSU NLP Group (Ohio State University)74.1
#95HuggingGPT / JARVISMicrosoft Research74
#96Lindy AILindy AI74
#97Mobile-AgentAlibaba Tongyi Lab (X-PLUG)74
#98FlowiseFlowiseAI73.9
#99KnowAgentzjunlp (Zhejiang University)73.7
#100MoTFudan (Li, Qiu)73.5
#101AppAgentTencent / mnotgod9673.1
#102Bee ComputerBee (acquired by Amazon 2026)73
#103ReMeModelScope (Alibaba)73
#104RMMGoogle / UCSB (2025)72.7
#105ChatDBTsinghua University (Hu et al.)72.5
#106MemoChatUniversity of Warwick / Alibaba72.5
#107AriGraphAIRI Institute / Moscow72.3
#108Personal AIPersonal AI72.3
#109Generative AgentsStanford / Google72
#110Second MeMindverse (Shang, Li, et al.)71.9
#111SynapseNTU / Salesforce (Zheng et al.)71.8
#112MemaryKingjulio823871.7
#113Character AICharacter.AI (Google investment)71.7
#114JARVIS-1CraftJarvis71.7
#115Tab AITab (Avi Schiffmann)71.7
#116EM-LLMem-llm (academic consortium)71.6
#117Lyzr CognisLyzr AI71.5
#118ZepZep AI71.3
#119Neo4j AuraDBNeo4j Inc.70.5
#120Friend AIFriend70
#121Charlie MnemonicGoodAI69.9
#122HybridAGISynaLinks69.3
#123Nomic AtlasNomic AI Inc.69.3
#124Memorizing TransformerGoogle Research (Wu, Rabe, Hutchins, Szegedy)68.9
#125Titanslucidrains (community) / paper by Google Research68.6
#126GleanGlean Technologies68
#127Mem0Mem066.9
#128MemoryScopeAlibaba ModelScope66.6
#129Claude ProjectsAnthropic66.6
#130LarimarIBM Research66.4
#131Gemini MemoryGoogle65.8
#132LongMemUCSB / Microsoft Research65.8
#133MemformerUC Santa Barbara / Amazon (Wu, Lan, Liu, et al.)65.7
#134kNN-LMStanford / Facebook AI Research (Khandelwal et al.)64.9
#135LettaLetta (formerly MemGPT)64.2
#136Haystack Memorydeepset62.8
#137MemoryBankInstitute of Software, Chinese Academy of Sciences62.3
#138R3MemHKUST (2025)62.3
#139LangMemLangChain61
#140Copilot MemoryMicrosoft60.7
#141ChatGPT MemoryOpenAI60.3
#142Couchbase VectorCouchbase Inc.60
#143Redis VectorRedis Ltd.59.2
#144MemoryBankHarbin Institute of Technology / SenseTime59.1
#145Notion AINotion Labs58.7
#146KDB AIKX Systems58.2
#147RagieRagie Inc.57.8
#148Saner AISaner.AI57.7
#149AnythingLLMMintplex Labs57.1
#150MemGPT ClassicBerkeley / Letta56.8
#151LlamaIndex MemoryLlamaIndex56.5
#152Mem AIMem Labs56.3
#153MongoDB Atlas VectorMongoDB Inc.55.6
#154QuivrQuivrHQ55.3
#155MemoroMIT Media Lab55.1
#156Heyday AIHeyday (shut down 2025)54.9
#157Sana AISana Labs54.9
#158MemoriGibsonAI54.8
#159Mnemosyneindependent54.5
#160MnemosyneJohns Hopkins / independent (2025)54.5
#161EpsillaEpsilla Inc. (YC S23)53.3