Add latex-vs-slogan retrieval experiment

[Vilin97]

What

Adds experiments/latex_vs_slogan/ — a controlled test of whether we should index a
theorem's verbatim raw-LaTeX statement or an LLM slogan for natural-language
retrieval. Corroborates the "raw LaTeX underperforms" decision and explains the apparent
counterexamples.

Setup

• 100 theorems from 10 papers, one per arXiv math tag (AG, AT, NT, CO, PR, DG, FA, LO, RT, AP); statements extracted from arXiv LaTeX source (frozen in data/papers.json).
• Per theorem: a slogan and an independent NL query (searcher's register, no formulas), both from Qwen3-235B-A22B-Instruct-2507.
• Embed with Qwen3-Embedding-8B via Nebius/TokenFactory; score gap = cos(query, slogan) − cos(query, raw) under the deployed asymmetric prompts and a symmetric-query variant.

Result

config	slogan closer	mean gap	mean cos slo / raw
asym (deployed)	87 / 100	+0.061	0.806 / 0.745
symq (symmetric)	90 / 100	+0.067	0.854 / 0.787

• Every tag favors the slogan on average (per-tag 60–100%).
• Mechanism: the slogan advantage grows with the notation density of the raw statement (Pearson r = +0.28) — symbol-dense statements embed poorly against NL.
• Raw LaTeX wins only a 13% minority: statements that are already prose (nothing to translate) or where the slogan drifted into higher-abstraction jargon.

Full write-up, per-tag tables, minority-case breakdown, and caveats (chiefly: query and slogan share a source, so the +0.06 is an upper-ish bound) are in experiments/latex_vs_slogan/README.md.

Notes

• Scripts read the embedding key from NEBIUS_API_KEY / TOKENFACTORY_API_KEY (or a repo-root .env); no secrets are committed.
• .gitignore gains two ! allow-rules for this experiment's data/*.json and figure, matching the existing per-experiment convention.
• Reproduce: pip install -r requirements.txt then run_experiment.py / analyze.py / plot.py (corpus is committed; fetch_theorems.py re-fetches).

🤖 Generated with Claude Code

https://claude.ai/code/session_01HqhLKdyxACFbErUP4vDpM1