Research
How we measure memory recall, and the benchmark results once a full run is in.
Method
How we measure recall
- Every change to how memories are ranked is scored on a held-out set of queries, with nDCG@8: did the right memories come back, near the top of the first eight.
- Ranking turns on in production only after a full run scores at least 0.70. That bar does not move down.
- Each run is a committed report, so a result can be traced to the code and the data that produced it.
Benchmarks
Public benchmarks
- Next: LoCoMo and LongMemEval-S, run against the same memory API your agents use.
- Results go on this page with the judge prompt, the configuration and the cost of the run.
- Until a full run is in, we don't publish numbers. The method, judge and budget are on Benchmarks.
Proof
Check it yourself
- Every memory write is a receipt, anchored to a public record every 30 minutes.
- Paste one into Check a receipt to verify it without trusting us.
- What shipped, and when, is in the changelog.