Official SOTA Submission: VAC Memory System Reaching 80.1% on LoCoMo (Generous Judge)

[vac-architector]

Hello LoCoMo Team,

I am submitting the results for the VAC Memory System, which reached 80.1% average accuracy on the LoCoMo benchmark (Generous Judge). This result sets a new SOTA for RAG systems evaluated on this task.

The system was developed as a hybrid RAG architecture (FAISS + BM25 + Custom MCA Gate).

All results are 100% reproducible out-of-the-box.

Details for Verification:

Repository: https://github.com/vac-architector/VAC-Memory-System

Report File: baseline_100 result LoCoMo

Reproducibility Steps: Run the System

🐧 Linux
git clone https://github.com/vac-architector/VAC-Memory-System.git
cd VAC-Memory-System
export OPENAI_API_KEY="sk-..."
./run_test.sh

🪟 Windows
git clone https://github.com/vac-architector/VAC-Memory-System.git
cd VAC-Memory-System
set OPENAI_API_KEY=sk-...
run_test.bat

Verify Results

Run the official judge

python3 Core/gpt_official_generous_judge_from_mem0.py results/vac_v1_*.json

Check accuracy

cat results/*_generous_judged.json | grep "accuracy".

Author: Viktor Kuznetsov (Solo Founder)

We look forward to your verification and appreciate your work on this crucial benchmark. Thank you.