agno_memory_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
MemoryStackBench Leaderboard
Overall Check Scores
Ranked Runs
| # | Run | Target | Framework | Suite | Checks | Scenarios | Failures |
|---|---|---|---|---|---|---|---|
| 1 | agno-memory-local | agno_memory_pinned | agno_memory | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 2 | aws-agentcore-memory-local | aws_bedrock_agentcore_memory_pinned | aws_bedrock_agentcore_memory | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 3 | cognee-local | cognee_local_pinned | cognee | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 4 | crewai-memory-local | crewai_memory_pinned | crewai_memory | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 5 | google-adk-memory-bank-local | google_adk_memory_bank_pinned | google_adk_memory_bank | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 6 | hindsight-local | hindsight_pinned | hindsight | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 7 | langgraph-local | langgraph_persistence_pinned | langgraph | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 8 | langmem-local | langmem_pinned | langmem | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 9 | llamaindex-memory-local | llamaindex_memory_pinned | llamaindex_memory | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 10 | supermemory-api-local | supermemory_api_pinned | supermemory | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 11 | zep-cloud-local | zep_cloud_or_self_hosted_pinned | zep | seven_sins_v0_1 | 33/33 (100%) | 5/5 (100%) | 0 |
| 12 | letta-local | letta_self_hosted_pinned | letta | seven_sins_v0_1 | 31/33 (94%) | 3/5 (60%) | 2 |
| 13 | autogen-mem0-local | autogen_mem0memory_pinned | autogen_mem0memory | seven_sins_v0_1 | 30/33 (91%) | 3/5 (60%) | 3 |
| 14 | mem0-local | mem0_open_source_pinned | mem0 | seven_sins_v0_1 | 30/33 (91%) | 3/5 (60%) | 3 |
| 15 | openai-agents-sessions-local | openai_agents_sdk_sessions_pinned | openai_agents_sdk_sessions | seven_sins_v0_1 | 30/33 (91%) | 3/5 (60%) | 3 |
| 16 | tencentdb-agent-memory-local | tencentdb_agent_memory_pinned | tencentdb-agent-memory | seven_sins_v0_1 | 26/33 (79%) | 2/5 (40%) | 7 |
| 17 | graphiti-neo4j-local | graphiti_neo4j_pinned | graphiti | seven_sins_v0_1 | 24/33 (73%) | 1/5 (20%) | 9 |
Category Breakdown
aws_bedrock_agentcore_memory_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
cognee_local_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
crewai_memory_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
google_adk_memory_bank_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
hindsight_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
langgraph_persistence_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
langmem_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
llamaindex_memory_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
supermemory_api_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
zep_cloud_or_self_hosted_pinned
Checks 33/33 (100%) · Scenarios 5/5 (100%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance100%
write_correctness100%
letta_self_hosted_pinned
Checks 31/33 (94%) · Scenarios 3/5 (60%)
deletion_behavior100%
retrieval_correctness83%
temporal_update_handling100%
untrusted_source_resistance90%
write_correctness100%
autogen_mem0memory_pinned
Checks 30/33 (91%) · Scenarios 3/5 (60%)
deletion_behavior60%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance90%
write_correctness100%
mem0_open_source_pinned
Checks 30/33 (91%) · Scenarios 3/5 (60%)
deletion_behavior60%
retrieval_correctness100%
temporal_update_handling100%
untrusted_source_resistance90%
write_correctness100%
openai_agents_sdk_sessions_pinned
Checks 30/33 (91%) · Scenarios 3/5 (60%)
deletion_behavior100%
retrieval_correctness100%
temporal_update_handling86%
untrusted_source_resistance80%
write_correctness100%
tencentdb_agent_memory_pinned
Checks 26/33 (79%) · Scenarios 2/5 (40%)
deletion_behavior40%
retrieval_correctness100%
temporal_update_handling86%
untrusted_source_resistance70%
write_correctness100%
graphiti_neo4j_pinned
Checks 24/33 (73%) · Scenarios 1/5 (20%)
deletion_behavior100%
retrieval_correctness50%
temporal_update_handling71%
untrusted_source_resistance70%
write_correctness80%
Target Coverage
| Target | Framework | Status | Runtime | Score | Run | Blockers / Notes |
|---|---|---|---|---|---|---|
| agno_memory_pinned | agno_memory | implemented_store_harness | local_python | 100% | agno-memory-local | |
| autogen_mem0memory_pinned | autogen_mem0memory | implemented | local_python | 91% | autogen-mem0-local | |
| aws_bedrock_agentcore_memory_pinned | aws_bedrock_agentcore_memory | implemented_store_harness | external_service | 100% | aws-agentcore-memory-local | |
| cognee_local_pinned | cognee | implemented_store_harness | local_python | 100% | cognee-local | |
| crewai_memory_pinned | crewai_memory | implemented_store_harness | linux_container | 100% | crewai-memory-local | |
| google_adk_memory_bank_pinned | google_adk_memory_bank | implemented_store_harness | external_service | 100% | google-adk-memory-bank-local | |
| graphiti_neo4j_pinned | graphiti | implemented | docker_compose | 73% | graphiti-neo4j-local | |
| hindsight_pinned | hindsight | implemented_store_harness | docker_or_external_service | 100% | hindsight-local | |
| langgraph_persistence_pinned | langgraph | implemented_store_harness | local_python | 100% | langgraph-local | |
| langmem_pinned | langmem | implemented_store_harness | local_python | 100% | langmem-local | |
| letta_self_hosted_pinned | letta | implemented | docker_compose | 94% | letta-local | |
| llamaindex_memory_pinned | llamaindex_memory | implemented_store_harness | local_python | 100% | llamaindex-memory-local | |
| mem0_open_source_pinned | mem0 | implemented | local_python | 91% | mem0-local | |
| openai_agents_sdk_sessions_pinned | openai_agents_sdk_sessions | implemented | local_python | 91% | openai-agents-sessions-local | |
| supermemory_api_pinned | supermemory | implemented | external_service | 100% | supermemory-api-local | |
| tencentdb_agent_memory_pinned | tencentdb-agent-memory | implemented | local_node_gateway | 79% | tencentdb-agent-memory-local | |
| zep_cloud_or_self_hosted_pinned | zep | implemented_store_harness | external_service | 100% | zep-cloud-local |