You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: polish sweep — fix stale docstrings, dead paths, PyPI-visible links (closes#21, #22, #23, #24, #25, #26, #27, #15)
Hunter Bundle E. 8 findings that mostly didn't affect runtime but
degraded the first-read experience — wrong tier copy, broken links on
PyPI, stale PNG duplicates, gitignored paths cited as sources.
- F12 (#21): vector_search.py module docstring rewritten — acknowledges
tier-aware embedder resolution (Model2Vec / Qwen3 per tier), not the
v0.3.0 Model2Vec-only framing. Adds sentence-transformers to Dependencies.
- F13 (#22): bench_truememory_base.py header — was copy-pasted from Pro
and still said "Pro Tier (+HyDE)". Rewritten to describe Base (Qwen3 +
gte-reranker, HyDE off, 91.5%). Usage examples now reference the right
script name.
- F14 (#23): category-heatmap.png deleted — was byte-identical to
category-grouped-bars.png (MD5 f00c08...). Only live reference was to
grouped-bars; heatmap filename was dead (CHANGELOG historical mention only).
- F15 (#24): README v0.3.0-chart caveat moved above the first benchmark
chart (was after all three). Option 2 interim per the finding — chart
regeneration (Option 1) is a separate deferred task.
- F16 (#25): BENCHMARK_RESULTS.md no longer cites gitignored
`_working/benchmarks_56combo/...` paths as authoritative. The aggregate
numbers in the table are the canonical reference; the Modal scripts
reproduce them.
- F17 (#26): 8 relative text links in README rewritten to absolute
github.com/.../blob/main/... URLs. PyPI's markdown renderer doesn't
rewrite relative paths, so the links 404'd on the package page for
install.sh, CLAUDE.md.example, LICENSE, and four benchmarks/* refs.
- F18 (#27): `_nm_format_ctx` / `_nm_make_hyde_fn` renamed to `_tm_*` in
the four bench scripts (bench_truememory_{edge,base,pro}.py,
modal_benchmark.py) — stray neuromem-namespace helpers from pre-rebrand.
Purely cosmetic; scripts are ship-in-sdist so the brand leak was visible.
- F40 (#15): documented `--help` precedence over `--setup` (Unix
convention — docs-emitting flags short-circuit side-effects) with an
inline comment in mcp_server.main and a regression-lock test in
test_cli_help.py.
pytest 197 passed / 1 skipped (baseline 196 + 1 new F40 test).
ruff clean. Local `python -m build && twine check --strict` PASSED.
Copy file name to clipboardExpand all lines: README.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,6 +29,8 @@
29
29
30
30
Tested on [LoCoMo](https://github.com/snap-research/locomo), the standard benchmark for conversational memory. 1,540 questions across 10 conversations. All 8 systems share the same answer model, judge, scoring, top-k, and byte-identical answer prompt — only retrieval differs.
31
31
32
+
> **Note on charts:** the hero banner and the three charts below still show the v0.3.0 single-Pro-tier layout (91.5%). Chart regeneration is tracked for a later release; all numerical claims in the README text and tables reflect the v0.4.0 three-tier scores (90.1 / 91.5 / 91.8%).
<imgsrc="https://raw.githubusercontent.com/buildingjoshbetter/TrueMemory/main/assets/charts/accuracy-vs-cost.png?v=2"alt="Accuracy vs Infrastructure Cost" />
40
42
</p>
41
43
42
-
All scores use the same evaluation pipeline: GPT-4.1-mini answer generation, GPT-4o-mini judge (3x majority vote), temperature=0. Zero errors across 12,320 total answers. Scores use a lenient semantic-match judge; rankings are valid across all systems but absolute values are higher than published LoCoMo baselines using strict exact-match. [Full methodology](benchmarks/locomo/BENCHMARK_RESULTS.md) and reproduction scripts in [`benchmarks/`](benchmarks/locomo/).
44
+
All scores use the same evaluation pipeline: GPT-4.1-mini answer generation, GPT-4o-mini judge (3x majority vote), temperature=0. Zero errors across 12,320 total answers. Scores use a lenient semantic-match judge; rankings are valid across all systems but absolute values are higher than published LoCoMo baselines using strict exact-match. [Full methodology](https://github.com/buildingjoshbetter/TrueMemory/blob/main/benchmarks/locomo/BENCHMARK_RESULTS.md) and reproduction scripts in [`benchmarks/`](https://github.com/buildingjoshbetter/TrueMemory/blob/main/benchmarks/locomo/).
43
45
44
46
---
45
47
@@ -57,8 +59,6 @@ All scores use the same evaluation pipeline: GPT-4.1-mini answer generation, GPT
57
59
58
60
TrueMemory Pro nearly matches EverMemOS across all 4 question categories. Mem0 collapses on multi-hop reasoning (37.7% vs 90.7%).
59
61
60
-
> The hero banner and charts in this README still show the v0.3.0 single-Pro-tier layout (91.5%). Chart regeneration is tracked for a later release; all numerical claims in the README text and tables reflect the v0.4.0 three-tier scores above.
> **What this actually does:** installs [uv](https://docs.astral.sh/uv/) (Astral's Python tool manager) if needed, fetches a managed Python 3.12 into `~/.local/share/uv/`, installs TrueMemory into an isolated tool environment, and auto-configures Claude Code and Claude Desktop. **Your system Python is never touched.** No sudo, no venvs, no pip struggle. Uninstall cleanly with `uv tool uninstall truememory`.
103
103
104
-
> **Want to audit the script first?** It's ~140 lines of shell, no sudo, stays entirely under `$HOME`. Read the source at [`install.sh`](install.sh), or download and inspect locally: `curl -LsSf https://raw.githubusercontent.com/buildingjoshbetter/TrueMemory/main/install.sh -o install.sh && less install.sh && sh install.sh`.
104
+
> **Want to audit the script first?** It's ~140 lines of shell, no sudo, stays entirely under `$HOME`. Read the source at [`install.sh`](https://github.com/buildingjoshbetter/TrueMemory/blob/main/install.sh), or download and inspect locally: `curl -LsSf https://raw.githubusercontent.com/buildingjoshbetter/TrueMemory/main/install.sh -o install.sh && less install.sh && sh install.sh`.
105
105
106
106
> **Want Base or Pro (adds Qwen3 embeddings + gte-reranker + sentence-transformers, ~1.5-2.5GB depending on OS)?**
107
107
> ```bash
@@ -149,7 +149,7 @@ After setup, TrueMemory runs automatically. It stores what you tell it and recal
149
149
150
150
### Make it automatic (optional)
151
151
152
-
Copy [`CLAUDE.md.example`](CLAUDE.md.example) to your home directory as `CLAUDE.md`. This tells Claude to store your preferences and recall them without being asked:
152
+
Copy [`CLAUDE.md.example`](https://github.com/buildingjoshbetter/TrueMemory/blob/main/CLAUDE.md.example) to your home directory as `CLAUDE.md`. This tells Claude to store your preferences and recall them without being asked:
153
153
154
154
```bash
155
155
cp CLAUDE.md.example ~/CLAUDE.md
@@ -214,9 +214,9 @@ uvx creates a cached environment on first run; subsequent spawns are fast. Good
214
214
215
215
Every benchmark script is self-contained and runs on [Modal](https://modal.com).
216
216
217
-
- **[Leaderboard & Reproduction](benchmarks/locomo/README.md)**: run any system yourself
TrueMemory tier per-category numbers are sourced from the authoritative 56-grid sweep (`_working/benchmarks_56combo/results/`in the repo's working archive, files: `mini_b1_model2vec_miniml6_hyde_off_result.json` for Edge, `pro_qwen3_256d_gte_reranker_hyde_off_result.json` for Base, `qwen3_256d__gte_reranker_result.json` for Pro).
45
+
TrueMemory tier per-category numbers are sourced from the authoritative 56-grid sweep. The raw result JSONs live in a gitignored working archive (not distributed with the package); the aggregate numbers in the table above are the canonical reference, and the Modal bench scripts in `benchmarks/locomo/scripts/` reproduce them end-to-end.
46
46
47
47
### Legacy v0.3.0 configurations (deprecated in v0.4.0)
0 commit comments