# LoCoMo benchmark — Hanzo AI > Hanzo Context on LoCoMo retrieval: multi-hop ALL@20 jumps from 18.8 to 37.5 on held-out splits, outperforming standard cosine search. [Benchmarks](https://hanzo.ai/benchmarks) / LoCoMo measured # LoCoMo Ten long conversations, 1,540 questions with annotated evidence turns. Hanzo Context measured against cosine over turns and against the annotated gold turns, on the same split, embedder, reader and k. dataset [LoCoMo](https://github.com/snap-research/locomo) licence CC BY-NC 4.0 introduced by [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://arxiv.org/abs/2402.17753) — Maharana, Lee, Tulyakov, Bansal, Barbieri, Fang, ACL 2024 harness hanzoai/benchmarks · brain our papers [Hops Do Not Fix Multi-Hop Retrieval](https://github.com/hanzoai/papers/blob/main/site/public/pdfs/hanzo-multi-hop-retrieval.pdf), [Linkage, Not Similarity](https://github.com/hanzoai/papers/blob/main/site/public/pdfs/hanzo-context-memory.pdf) `node results.mjs`regenerates every table on this page from the raw runs ## What is measured Ten conversations, 5,882 turns, 1,540 questions, each annotated with the turns that answer it. Retrieval is scored against those turns. Configurations are chosen on conversations 0–2 and frozen at a commit before conversations 3–9 run once. Adversarial questions are excluded, as Mem0 and Zep exclude them. ## Retrieval The engine adds candidate generators to cosine — BM25, the atomic fact index, canonical entities, the timeline, adjacency, one hop of typed expansion, then a second retrieval hop asked with what the first resolved. On the held-out split multi-hop ALL@20 goes from 18.8 to 37.5 and R@20 from 70.9 to 80.7, on the same embedder and the same k. ### all ten conversations · zen-embedding-0.6b · k=20 facts LoCoMo’s own observation annotations · weights frozen on dev at 33d0f8749bd1 · rows are cumulative: each carries every generator before it, from BM25 on · ALL = every annotated turn inside the top 20; ANY = at least one; R@20 = |retrieved ∩ gold| / |gold| configuration multi-hop ALL multi-hop ANY single-hop ALL temporal ALL open-domain ALL R@k nDCG@k supported semantic only 22.7 79.8 78.2 77.3 33.7 71.9 49.6 81.0 +facts 36.9 88.3 82.3 82.6 42.4 78.2 59.6 86.4 +adjacency 39.0 88.7 84.2 83.5 44.6 79.7 61.4 87.8 +iterative hops 39.0 87.9 86.8 84.4 45.7 80.9 62.6 88.0 ### all ten conversations · all-MiniLM-L6-v2 · k=10 facts LoCoMo’s own observation annotations · weights frozen on dev at 33d0f8749bd1 · rows are cumulative: each carries every generator before it, from BM25 on · ALL = every annotated turn inside the top 10; ANY = at least one; R@10 = |retrieved ∩ gold| / |gold| configuration multi-hop ALL multi-hop ANY single-hop ALL temporal ALL open-domain ALL R@k nDCG@k supported semantic only 9.6 49.6 51.4 46.1 16.3 45.3 31.1 53.1 +lexical 13.8 58.9 67.2 62.0 23.9 59.2 43.7 67.6 +facts 20.9 73.4 73.1 72.0 25.0 66.7 52.0 75.6 +adjacency 21.6 74.5 74.6 72.6 27.2 67.7 53.3 76.3 +iterative hops 24.1 78.0 77.5 73.5 27.2 70.1 55.8 78.5 ### test split, conversations 3–9 · zen-embedding-0.6b · k=20 facts LoCoMo’s own observation annotations · weights frozen on dev at 33d0f8749bd1 · rows are cumulative: each carries every generator before it, from BM25 on · ALL = every annotated turn inside the top 20; ANY = at least one; R@20 = |retrieved ∩ gold| / |gold| configuration multi-hop ALL multi-hop ANY single-hop ALL temporal ALL open-domain ALL R@k nDCG@k supported semantic only 18.8 80.3 78.3 72.3 35.6 70.9 48.8 80.5 +facts 35.1 88.5 83.0 78.8 43.8 77.9 58.7 86.4 +entities 35.1 88.5 83.0 78.8 43.8 77.9 58.7 86.4 +timeline 34.6 88.5 83.3 79.7 45.2 78.2 59.9 86.8 +adjacency 37.5 88.5 84.6 80.1 45.2 79.1 60.5 87.5 +typed graph 37.5 88.5 84.6 80.1 45.2 79.1 60.5 87.5 +iterative hops 37.5 88.0 87.7 81.4 45.2 80.7 61.6 87.8 ## The other LoCoMo harness [hanzoai/semantic · bench/locomo](https://hanzo.ai/benchmarks/locomo-subject-scope) measures the same dataset under a different protocol: lexical retrieval with no embedding model, a fixed extractive reader with no language model anywhere, and all 1,986 questions including the adversarial quarter, reported apart. Its numbers measure a different reader and are not on the scale of these, so it has its own page and its own paper. Hanzo AI Cloud ### Build on Hanzo Cloud [Start on Hanzo Cloud](https://cloud.hanzo.ai)[View Models & Pricing](https://hanzo.ai/pricing) [PreviousGPQA-Diamond](https://hanzo.ai/benchmarks/gpqa) [All Benchmarks Board](https://hanzo.ai/benchmarks) [NextMemoryAgentBench](https://hanzo.ai/benchmarks/memoryagentbench) ### Related in memory [MemoryAgentBench](https://hanzo.ai/benchmarks/memoryagentbench)[LongMemEval](https://hanzo.ai/benchmarks/longmemeval)[LoCoMo · subject scope](https://hanzo.ai/benchmarks/locomo-subject-scope)[LoCoMo-Conv](https://hanzo.ai/benchmarks/locomo-conv) Other benchmarks[CronQuestions](https://hanzo.ai/benchmarks/cronquestions) · [RepoBench-R](https://hanzo.ai/benchmarks/repobench-r) · [Fleet residency](https://hanzo.ai/benchmarks/fleet) · [Live agent footprint](https://hanzo.ai/benchmarks/goroutine) · [Sandbox cold start](https://hanzo.ai/benchmarks/sandbox) · [Engine prefill on CUDA vs llama.cpp](https://hanzo.ai/benchmarks/inference) · [GPQA-Diamond](https://hanzo.ai/benchmarks/gpqa) raw runs, traces, prompts and scoring: hanzoai/benchmarks · brain · protocol in `METHOD.md` · every table above regenerated by `node results.mjs`