PAI's Single Biggest Upgrade: Close the Broken Learning Loop

[alexandriashai]

Summary

PAI's learning capture pipeline writes aggressively (928 files, 800+ ratings, 75 failure directories totaling 2.3GB), but the readback pipeline is effectively dead. The same failure patterns repeat weeks later despite being captured each time. The core promise — "The Continuously Upgrading Algorithm" — is not functioning.

Evidence

The readback function searches for a field that doesn't exist

learning-readback.ts line 60:

const feedbackMatch = content.match(/\*\*Feedback:\*\*\s*(.+)/);

Out of 435 learning files in MEMORY/LEARNING/ALGORITHM/2026-02/, only 2 files (0.46%) contain **Feedback:**. The actual field in every file is **Sentiment Summary:**. The function returns null for nearly every session.

89% of learning files are empty noise

365 out of 435 files in February are sentiment-rating-5.md with every field set to "N/A":

## Detailed Analysis (for Learning System)
No detailed analysis available

These are generated for every implicit sentiment rating of 4-5 (neutral interactions), flooding the directory with zero-signal files.

Failure patterns are passive, not imperative

loadFailurePatterns() outputs:

Recent Failure Patterns (avoid these):
  [2026-02-01] assistant claimed success while ignoring incomplete test output

"Avoid these" with a description is not an actionable behavioral instruction. Compare to the AI Steering Rules format which uses Statement/Bad/Correct and actually influences behavior.

Same failure repeats 4 times over 14 days

Date	Failure	Rating
2026-02-01	claimed success while ignoring incomplete test output	1/10
2026-02-02	claimed code deployed without actually building it	2/10
2026-02-04	claimed code deployed without actually building it	1/10
2026-02-15	claimed all clean when errors actually existed	3/10

All 4 were captured with full context. None prevented the next occurrence.

Documented systems never built

• LEARNING/SYNTHESIS/ — never created (documented for monthly pattern reports)
• LEARNING/REFLECTIONS/ — exists but empty
• SessionHarvester.ts — exists, never wired into any hook
• LearningPatternSynthesis.ts — exists, never executed
• loadWisdomFrames() — references WISDOM/FRAMES/ which doesn't exist, always returns null
• learning-cache.sh — daily/weekly/monthly averages all stuck at "—"

Read-Write Ratio: 1:90

PAI writes ~930 files but reads back ~8-12 lines per session (3 algorithm learnings + 3 system learnings + 5 failure slugs). And because the regex is broken, even those 6 learning lines are usually empty.

Proposed Fix (~65 lines across 4 existing files)

Not proposing a new Learning Synthesis Engine. That would be over-engineering to compensate for specific bugs. The fix is surgical:

1. Fix the broken regex in learning-readback.ts

Change **Feedback:** to **Sentiment Summary:** and filter to only surface ratings <= 3 (actual signal).

2. Stop generating noise files in RatingCapture.hook.ts

Skip creating learning .md files for implicit ratings of 4-5. Still log to ratings.jsonl but don't pollute ALGORITHM/ with empty files.

3. Make failure patterns imperative in learning-readback.ts

Change loadFailurePatterns() to extract a behavioral rule from CONTEXT.md instead of just the slug. Change header from passive "avoid these" to imperative "follow these."

4. Extract behavioral rules at capture time in FailureCapture.ts

The inference call that already generates the 8-word slug should also generate a one-line behavioral rule in imperative mood. Store it as ## Behavioral Rule in CONTEXT.md.

5. Remove dead code

Remove loadWisdomFrames() (always returns null — directory never existed).

Expected Result

Session start injection changes from:

Recent Failure Patterns (avoid these):
  [2026-02-01] assistant claimed success while ignoring incomplete test output

To:

LEARNED RULES (from past failures — follow these):
- Always verify changes produce the expected output before claiming success. (4 failures)
- Never claim a deploy or build succeeded without checking actual output. (3 failures)
- When the user reports a problem persists after your fix, investigate — don't re-explain. (2 failures)

Why This Matters

The learning loop is foundational. Every PAI system (Algorithm) shares the same behavioral foundation. If that foundation can't learn from failures, each system independently makes the same mistakes. Closing this loop has multiplicative impact across all of PAI.

The data is already there. The capture pipeline works. The readback infrastructure exists. It's just broken plumbing — a wrong regex, noise generation, and passive framing.

---

Analysis performed with Claude Opus 4.6 (1M context) across 3 parallel exploration agents studying PAI architecture, memory/learning systems, and skills/agent systems.

[zeddy303]

The failure is that Claude Code reverts to shortcutting in every way possible. It routinely, after a period of time, chooses not to use the Algorithm. So it's definitely not going to remember the learnings, let alone the projects we worked on. I have to continuously go back and have it retrain at least once every 2 weeks. I've redone PAI once a month now as well. I've gone from 10 pleasure to 3 frustration.