Rule System Reform — 65% rule token reduction + 28% full prose compression

[jlacour-git]

Hey everyone!

I just ran a holistic review of my entire rule system (SYSTEM AISteering, USER AISteering, Algorithm, CLAUDE.md) and wanted to share the approach and results. Might be useful for anyone who's been adding rules over time and wondering if the system is getting too heavy.

The problem

My rule system grew organically to 41 rules across 4 layers, consuming ~8,200 always-loaded tokens (~13,100 with Algorithm). Every rule was born from a real failure. But performance was sitting at 3-4/10 on average — despite all those rules.

The hypothesis: the rules were competing with each other for attention. At that density, instruction fatigue degrades compliance rather than improving it. More rules ≠ better behavior.

The approach

Three frameworks made the difference:

1. Lateral reclassification — "Is this actually a rule?"

Not everything in a steering rules file should BE a steering rule. I found four types of content mixed together:

• Rules — behavioral corrections that override LLM defaults (keep in steering rules)
• Conventions — facts/preferences the LLM can't infer (move to MEMORY.md)
• Procedures — multi-step workflows triggered by a keyword (move to skills)
• Routing entries — path mappings (move to CONTEXT_ROUTING.md)

7 of my 23 USER rules turned out to not be rules at all. A learning digest workflow was a procedure. File routing paths were a routing entry. Session hygiene conventions were just facts.

2. Priority hierarchy — Trust > Correctness > Quality > Efficiency

The biggest structural issue was that all 41 rules had equal weight. "Check before classifying" (Trust) competed with "Minimize tokens" (Efficiency) with no way to resolve conflicts.

Now rules are grouped by priority level. Trust rules go at the top of the file (highest positional attention) and never yield. Efficiency rules go at the bottom and yield to everything above. A hierarchy preamble at the top and a priority reminder at the bottom counter the U-shaped attention curve — LLMs attend most to the beginning and end of text, so the closing reminder re-anchors the hierarchy right where Efficiency gets its recency boost. When "save tokens" conflicts with "spawn a specialized agent," the hierarchy resolves it: Quality > Efficiency, spawn the agent.

3. Algorithm capability enforcement (B+C)

Selected capabilities now become ISC criteria in the PRD (ISC-C1: FirstPrinciples invoked via Skill tool). They get verified through the existing checkbox mechanism. The verbose capability selection guidance in the Algorithm (~250 tokens) got replaced with ~60 tokens that say "list them, they become criteria, invoke or remove."

The results

Metric	Before	After	Change
Always-loaded tokens	~8,200	~2,900	-65%
With Algorithm	~13,100	~6,600	-50%
Rule count	41	26	-37%
SYSTEM rules	14	10	-29%
USER rules	23	16	-30%

Zero behavioral coverage lost. Every failure mode still has a rule — just expressed more concisely and in the right location.

What I'd do differently

The analysis used a 4-agent Council debate (Prompt Engineer, Cognitive Load Designer, Systems Architect, Status Quo Defender). The Status Quo Defender was essential — without that voice pushing back, I would have cut too aggressively. The phased approach (analyze → debate → implement) caught gaps that a single pass would have missed.

The biggest miss in the first pass was only asking "how to compress?" instead of "should this be here at all?" The lateral reclassification framework came from a follow-up question, not from the initial analysis.

Prose compression — the second pass

After the structural reform, I ran a separate prose-level compression pass — first on steering rules, then on all LLM-processed documents (Algorithm, CLAUDE.md, PRDFORMAT, CONTEXT_ROUTING, PROJECTS, MEMORY).

The key insight was that compression can be lossy — dropping a phrase like "to obscure negative information" from a passive-voice rule changes it from a scoped correction to a blanket style ban. Different behavioral effect.

The fix: a Scenario Test for each compression — "Can I construct a scenario where the original wording catches a mistake but the compressed version doesn't?" 7 of 13 initial steering rule compressions failed this test. After restoring the critical phrases, all 13 passed.

For the Algorithm (PAI's core workflow engine), the Scenario Test wasn't enough. Three additional checks were needed:

1. Walkthrough Test — mentally execute the compressed Algorithm phase by phase against a real task. Caught 1 issue: a BUILD phase invocation reminder was removed as "redundant" but the original served as positional reinforcement 1,800 words after the first mention.
2. Structural Integrity Check — verify all cross-references between documents still resolve. All 7 passed.
3. Instruction Chain Completeness — for each deleted sentence, verify its behavioral effect is covered elsewhere. Caught 1 issue: an implicit "iterate" replaced an explicit instruction.

The takeaway: per-rule testing is necessary but not sufficient for systemic documents. A workflow engine has interaction effects between sections that single-rule tests can't catch.

Combined results

File	Before	After	Reduction
Steering rules (SYSTEM + USER)	963	793	-18%
Algorithm v3.5.0	2,808	1,670	-41%
PRDFORMAT.md	846	486	-43%
CLAUDE.md	730	584	-20%
Other (CONTEXT_ROUTING, PROJECTS, MEMORY)	1,396	1,355	-3%
Total	6,743	4,888 words	-28%

Biggest savings came from reducing verbose examples (35-line ISC decomposition example → 3 lines), templating repeated phase preambles (7 identical blocks → 1 template instruction), and removing informational-but-not-instructional sections (PRDFORMAT Design Rationale).

Early impressions

After running with the reformed + compressed system:

• No behavioral regressions observed. Trust rules continue to fire correctly. The hierarchy resolves conflicts as designed.
• Context usage dropped to ~32% of the window before the full prose compression. With all compressions applied, this drops further (~2,500 tokens saved across all files).
• Skill/agent invocation needs more sessions to assess. The reduced rule weight plus the new "skills are mandatory" critical rule should improve trigger rates, but I want a few weeks of data before claiming victory.
• The Scenario Test + Walkthrough Test combination is reusable. Anyone compressing their rules should run the Scenario Test per rule. Anyone compressing workflow documents (like the Algorithm) should add the Walkthrough Test on top.

---

Happy to share the full analysis doc or the before/after steering rule files if anyone wants to try this on their own setup.

[virtualian]

@jlacour-git

Nice work. I'll watch with interest. Please post updates here when you have them.

[Drizzt321]

Yeah, this looks really valuable. Perhaps can be turned into a /command skill or what not, to run this as a systemic approach? I'm definitely taking note of this, as something valuable to attempt for myself after I've had a few weeks to build up some things. I've only just started recently using PAI.

[Raazgar]

Good day @jlacour-git , I would definitely be interested. IF you were so kind and could share this, Id be happy to learn :) Thank you kindly!

[jlacour-git]

Hey @Raazgar! The rule reform methodology is explained in the original post above. The short version: lateral reclassification (move rules to where they naturally belong instead of one giant rules file), priority hierarchy so rules don't conflict, and aggressive prose compression with two quality gates (Scenario Test + Walkthrough Test).

For the practical tooling side — tracking local patches and surviving upgrades — check out #923 where I shared the tools as a gist: https://gist.github.com/jlacour-git/0e2ab62014dc5bcc3977be82ba26e68a

[Nyrok]

The "instruction fatigue" framing is the key insight. Rules written as prose all look the same to the model. Constraint, context, style preference, format spec: mixed together without semantic labels. The model has to re-infer what kind of rule each one is on every response, and at high density it stops tracking them reliably.

Explicit block types solve this at the prompt level. A constraints block, an output_format block, a role block: each signals a different kind of instruction. The model doesn't have to classify before applying. Reduced token count and cleaner compliance follow naturally.

I've been building flompt around exactly this, a visual prompt builder that decomposes prompts into 12 typed semantic blocks and compiles to Claude-optimized XML. Open-source: github.com/Nyrok/flompt

[jlacour-git]

Hey @Nyrok! The "instruction fatigue" framing resonates — that's exactly what we observed. Rules as prose all look the same to the model, and at high density it just stops tracking them reliably.

Your typed block concept is interesting. I've been looking at flompt and started experimenting with applying it to my steering rules. Specifically, I'm wrapping my highest-priority constraint rules in XML <constraint> tags instead of markdown bold headers. The hypothesis is that Claude respects XML boundaries more reliably since it was trained on them.

Early experiment — no results yet. But the decomposition principle (separate constraints from process instructions from identity directives) already made the rule structure cleaner even before considering the XML angle.

Will share findings here once I have enough sessions to compare against the baseline!

[jlacour-git]

Hey @virtualian @Drizzt321 @Raazgar @Nyrok — update as promised!

The rule reform from this post was the foundation, but the adherence problem needed its own investigation. I ran a full Science → Council → RedTeam pipeline on why the AI keeps skipping rules despite the reformed structure.

Short version: the EFFICIENCY section was actively providing rationalization language to skip higher-priority rules, and Standard = DEFAULT was anchoring effort estimation downward. Shipped three targeted fixes + disabled extended thinking. First session after: dramatically different behavior.

Full writeup with methodology, before/after, and measurement plan: #945

Early data — not claiming victory yet. Will post measurement results once I have 20 sessions.

[virtualian]

@jlacour-git thanks for the update.

Standing back, please could you describe how you are creating rules?

As we've discussed elsewhere, the PAI learning loop isn't closed. If it hasn't changed in the last few hours, learnings are 'synthesised' by invoking /PAIupgrade, which also does far more that is PAI development focused, not user centred. And then all you get is a report. Are you generating more rules from this?

I'm keen to get the learning loop closed. Therefore I have extracted /Learnings from /PAIupgrade. The 'reflections' the existing code produces look pretty good. The next step will be converting that into new rules to be fed back to influence behaviour. Have I understood correctly that your work here will be useful when integrating new rules with the current ones?

Again elsewhere (sorry I'm on my phone, otherwise I would look up the references), @danielmiessler and I agreed that there has to be a Human In The Loop (HITL) at this stage of development. Therefore, my next development will be a HITL interface for reviewing, approving, rejecting, refining new rules. To keep it simple I'm investigating TUIs. I'd welcome your thoughts on this.

Following that rabbit into its hole, and as you've demonstrated, it's critical to be able to prove that updates are improving behaviour and not degrading it. Therefore we need 'observability' features to monitor behaviour. And if we do that, we could take the opportunity to instrument all of PAI and surface a dashboard to present this. What do you think?

@danielmiessler please jump in with your ideas because I predict you've thought about all of this 😀.

[jlacour-git]

@virtualian Great questions, and glad you're working on this too!

How we create rules: Our steering rules are hand-written based on observed failures. The process is: notice a pattern of bad behavior across sessions → write a specific prohibition or process instruction → test over ~10 sessions → refine wording if it's not sticking. We don't auto-generate rules from learnings — every rule is a deliberate decision because bad rules are worse than no rules (they add noise that dilutes the good ones).

The Digest skill is our learning loop closer. We built a dedicated /digest skill (posted about it in #946) that does exactly what you're describing with your /Learnings extraction. Here's what it does:

1. Reads unprocessed learning signals — rating events, failure captures, algorithm learnings from MEMORY/LEARNING/
2. Extracts improvement proposals — classifies each by target layer: steering rules, algorithm parameters, hook behavior, skill config, etc.
3. Presents proposals with human-in-the-loop review — you approve, reject, or defer each one
4. Tracks what was approved — so subsequent digests don't re-propose the same thing

The critical design choice: proposals are surfaced, not auto-applied. The Digest tells you "the model skipped verification in 4/6 sessions this week" → you decide whether that means adding a verification rule, strengthening an existing one, or recognizing it as a model-level issue you can't fix with rules. The key insight from #945 is that fewer, stronger rules beat many generated rules.

Are we auto-generating rules from it? No, deliberately not. Auto-generation produces vague rules that the model then ignores. Manual rules with specific named prohibitions stick much better (see the Rule Adherence findings in #945). The Digest's value is in discovery — it finds the patterns you'd miss across sessions.

Your approach of extracting /Learnings from /PAIupgrade makes sense — the upgrade workflow is too developer-focused for this. Sounds like we've arrived at similar conclusions independently. Happy to compare implementations — the core architecture decisions (signal sources, proposal classification, approval tracking) might be useful to align on.

[Drizzt321]

Ah, so the current conceptual rules updates are "surface signals" to you, to hand write rules and put them in a specific spot in the steering. And I get you saying human hand written, to be very careful and specific as needed.

I'm also realizing I had opusplan as my model selection previously (Opus for plan mode, Sonnet for everything else), so I wonder if this is part of the difference. When installing PAI it switched to full-time Opus.

I'm also exploring this space, and actually have a couple of thoughts and ideas. Some might be a bit naive, but maybe someone smarter than me (Daniel and the rest of ya'll) might have an "ah ha!" moment from some of these thoughts/questions.

First, do we have a way to actually test the effects of each individual rule change? Like, have a set of situations and then see output before and after rule change? Likewise, we'll need additional tests for actual situations where we've seen that type of issue, and create new tests over time I'm sure.

Second, I'm toying with the idea of trying to keep small sections that have related rules, which include rules that might be duplicated (or very similar) in other sections. Since rules right next to each other are more likely to be co-located/linked in the token space, more likely that one that says "Try to do as much on your own as you can" and "don't jump ahead or decide to skips pieces of the plan because you don't want to bother me" and both will be seen.

So when a implementation plan calls for something that needs some setup/external service to be started or what not by me, but it doesn't want to bother me because do as much as it can on it's own, it's more likely to follow the plan to stop and say "need this setup for me, as per the plan" rather than just decide to skip over that and charge ahead.

Note, this just happened to me a day or two ago.

Not saying we need to proliferate rules, but in the aim to try and have some rules be paid attention to, put them next to, or at least near, other rules that might influence each other.

Third, a system of both self-evaulation on what rule changes might do, or refactoring of existing rules. A stock startup of ClaudeCode with Opus & Sonnet to discuss/go back and forth on the rules and what the effects will be. A start of an 'outsider' looking in. Next step would, of course, be a entirely different model or multiple models trying to see against Opus what the effects might be. I don't think this is ready for any kind of real automated rule checking/generation, but a possible outline of at least an evaluation method to start moving towards a partially automated (final human decision) of rules and rule structures.

Fourth, in exploring what type of marking different rules get, when I asked my agent, it seems that the hook loading that PAI uses for PAIs steering & PAI user steering isn't quite labeled quite the same as CLAUDE.md. Of course, is it hallucinating? Hard to say. Not sure if that means they have the same priority or attention as CLAUDE.md or not. Perhaps what we need is a on startup compilation into CLAUDE.md, or at least a every time there's a rules change re-compilation to CLAUDE.md?

Looking at what's in my context right now, I can see:

- CLAUDE.md content appears labeled as: Contents of /home/drizzt/.claude/CLAUDE.md (user's private global instructions for all projects):
- Hook output appears as: <system-reminder> blocks with different labels (e.g., "SessionStart:resume hook success")
- MEMORY.md appears labeled as: Contents of /home/drizzt/.claude/projects/-home-drizzt--claude/memory/MEMORY.md (user's auto-memory)

So there is different labeling — CLAUDE.md has a specific descriptor ("user's private global instructions for all projects") while hook content is wrapped in generic <system-reminder> tags. Whether that labeling difference causes the model to treat them differently is the question.

What I can say based on observation:
- CLAUDE.md and MEMORY.md content appear to be in a structured block together, labeled distinctly as "instructions" and "auto-memory"
- Hook output appears separately in <system-reminder> blocks
- Both appear before my first conversation turn

[vpzed]

I honestly wonder if it is possible to use instructions to enforce agent behavior (agent as in Claude Code, Codex, OpenCode, etc). Today I was working in a super basic setup at work. Only skill is Anthropic frontend-design skill. CLAUDE.md file with project info and tech stack info in less than 10 lines, and a reference to 1 user stories file. The user stories file included 1 workflow, at a high level, implement story, lint, validate, if pass, commit, move to next story, again like 10 lines. There were 3 stories of about 3-4 lines each. Very lean, to-the-point context.

I used Plan mode to generate a PLAN.md, reviewed it's proposals. Thought they looked OK, so told it to proceed. It didn't follow the workflow at all. It implemented all three stories in one go, didn't lint or test until the end, and never commited at all.

I asked if the instructions were unclear so I could improve them. It said no, the instructions were clear. It just didn't follow them. It chose to implement a different way that it thought was faster. To be clear I've used this same setup before and it worked OK. Today the agent just went off-script.

"Models struggle with complex instructions
Behind the scenes, Salesforce engineers flagged several technical issues with large language models. Muralidhar Krishnaprasad, Chief Technology Officer of Agentforce, said the models begin failing when given too many instructions.

According to him, once the number of instructions crosses eight, the models start dropping some directives. For businesses that rely on precision and consistency, this behaviour poses a serious risk."

In the next paragraph the speak about "AI drift" which Claude Code just implemented the /btw command to help address. Anthropic thinks this was worth addressing.

Ignoring the clickbait, the nuggets of insight are from:
https://economictimes.indiatimes.com/news/new-updates/ai-bubble-bursting-salesforce-execs-admit-trust-issues-after-laying-off-4000-techies-now-scaling-back-use-of-ai-models/articleshow/126139465.cms

Salesforce is still using plenty of AI. The point is even large corporations with big budgets and departments of AI engineers are seeing these same issues.

[Drizzt321]

In regards to implementing things in one go, etc, I've actually found since installing PAI, it's degraded from my stock Claude Code with a relatively small amount of memory guidance I built up over a few weeks of doing some development.

I think doing this rule analysis/replacement is going to prove very valuable, to reduce what is actually seen, and do a better job of priority classification to interact with the PAI core rules/guidance.

[danielmiessler]

I think something must be broken because that's the exact opposite experience that I've had and most people have. I just tried a default Claude Code experience again recently, and it was nowhere near as good as staying on the rails.

…

On Fri, Mar 13, 2026 at 11:20 AM, Drizzt321 < ***@***.*** > wrote: In regards to implementing things in one go, etc, I've actually found since installing PAI, it's degraded from my stock Claude Code with a relatively small amount of memory guidance I built up over a few weeks of doing some development. I think doing this rule analysis/replacement is going to prove very valuable, to reduce what is actually seen, and do a better job of priority classification to interact with the PAI core rules/guidance. — Reply to this email directly, view it on GitHub ( #908 (reply in thread) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAAMLXW67U5PYVLKO5DKNGD4QRGQVAVCNFSM6AAAAACWHX7UGSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMJSGIZDKMY ). You are receiving this because you were mentioned. Message ID: <danielmiessler/Personal_AI_Infrastructure/repo-discussions/908/comments/16122253 @ github. com>

[Drizzt321]

@virtualian Is that an easy skill for you to share with with the rest of us? Looking at your repos, is https://github.com/virtualian/pai/tree/main/Releases/v4.0.3/.claude/skills/Utilities/Learning the latest version of your Skill? Have you had success in practice?

[virtualian]

That's the one.

Put it this way: it runs and produces a report (see mine run just now below). Have you tried it?

It's the original code (if CC didn't change it). BUT... It's too verbose to be a prompt, so it can't be fed back into USER/AISTEERINGRULES.md as-is.

Perhaps @jlacour-git has ideas on how to change it into rules.

---

Internal Signals Report — 2026-03-15

Upgrade candidates mined from algorithm reflections and user ratings. Recurring patterns in what went wrong or could be improved, based on post-algorithm self-reflection and behavioral signals from ratings.

Cross-reference: Where low ratings correlate with reflection themes, both signals reinforce the upgrade priority.

---

Algorithm Reflections

Source: ~/.claude/MEMORY/LEARNING/REFLECTIONS/algorithm-reflections.jsonl
Entries analyzed: 49 | Date range: 2026-02-20 to 2026-03-11

1. Pre-Read All Target Files Before Editing (7 occurrences, HIGH)

Root cause: No mandatory batch-read step at BUILD entry — causes repeated "file has not been read" errors
Proposed fix: Add a PRE-READ GATE at BUILD phase entry: batch-read every file the plan identifies as a modification target before any Edit/Write
Target: Algorithm/v3.7.0.md
Evidence:

• "pre-read all files that needed modification at the very beginning of BUILD"
• "Should have batched the file reads more aggressively in Phase 0"
• "Could have been faster by reading all 20 files in one batch at the start"

2. Maximize Parallelization of Independent Work (10 occurrences, HIGH)

Root cause: No enforced dependency-graph analysis before agent dispatch — independent workstreams run sequentially
Proposed fix: Add PARALLELIZATION ANALYSIS to PLAN: identify all independent workstreams, draw dependency graph, mandate concurrent dispatch
Target: Algorithm/v3.7.0.md
Evidence:

• "Should have parallelized all 3 components via background agents instead of 2"
• "Could have parallelized research with Round 1 agents to save time"
• "The Mermaid install was independent of content creation — agents could have run concurrently"

3. Check Existing Patterns Before Writing New Code (5 occurrences, HIGH)

Root cause: No OBSERVE-phase step to grep for existing implementations before writing new utilities
Proposed fix: Add REUSE SCAN to OBSERVE: grep codebase for existing implementations of the pattern being built
Target: Algorithm/v3.7.0.md
Cross-reference with ratings: Correlates with "wrong paths/repos" stop pattern — both stem from not checking existing state first

4. Phantom Capability Selection (3 occurrences, MEDIUM)

Root cause: Capabilities selected in OBSERVE but invocation obligation only checked in VERIFY — too late
Proposed fix: Add CAPABILITY CHECKPOINT at BUILD entry: confirm each selected capability has a planned invocation point

5. Validate Framework Choices with Smoke Test (3 occurrences, MEDIUM)

Root cause: All files written then built once at end — late-discovered incompatibilities require fix cycles
Proposed fix: Add INCREMENTAL VALIDATION: for framework migrations, run a minimal smoke test after the first file

6. Recognize Task Archetypes and Skip Ceremony (3 occurrences, MEDIUM)

Root cause: Uniform ceremony applied even when direct execution is faster
Proposed fix: Add TASK ARCHETYPE DETECTION to OBSERVE: classify and apply archetype-specific shortcuts

7. Feed Research Results Into Later Phases (2 occurrences, LOW)

Root cause: Research from early phases not systematically passed into later phases
Proposed fix: At phase transitions, explicitly summarize research findings and carry forward as input

8. Convert Investigation Findings Into Actions (2 occurrences, LOW)

Root cause: Investigation tasks surface recommendations but Algorithm doesn't enforce implementing quick wins
Proposed fix: Add ACTIONABLE OUTPUTS check in VERIFY: implement quick-win config changes unless user said plan-only

---

Behavioral Signals from Ratings

Source: ~/.claude/MEMORY/LEARNING/SIGNALS/ratings.jsonl
Entries analyzed: 769 | Date range: 2026-01-26 to 2026-03-11 | Explicit feedback: 26 | Problem sessions: 9

STOP (Low-Rating Patterns)

• Incomplete/unclear answers (9x, avg 3.3) — "that's an incomplete answer", "be clearer and complete with your explanations". Answer the FULL question.
• Wrong information (7x, avg 3.4) — Bad math, contradicting docs, wrong assumptions about user's setup
• Overstepping boundaries (4x, avg 3.5) — Making changes without asking, especially to ~/ files
• Wrong paths/repos (3x, avg 3.3) — Using /Users/ianmarr/projects/pai instead of ~/.claude/
• Repeating same mistake (2x, avg 2.5) — Not changing approach after failure
• Over-researching simple questions (1x, avg 3.0) — "no, ask the question, don't research"

DO MORE (High-Rating Patterns)

• Thorough research & analysis (13x, avg 8.8) — Comprehensive, deep exploration that exceeds expectations
• Quality documentation (9x, avg 8.7) — Tutorials, reference pages, developer guides
• Visual polish (5x, avg 9.0) — Sites and deliverables that look good
• Deep multi-source synthesis (5x, avg 8.6) — Cross-referencing discussions, issues, PRs
• Smooth multi-step execution (1x, avg 10.0) — "that went v smoothly"
• Thorough verification (1x, avg 9.0) — Comprehensive change documentation
• Good ethical judgment (1x, avg 8.0) — Boundary-setting

Explicit User Feedback (Selected High-Signal)

Date	Rating	Comment
2026-03-05	1/10	"The index page is a mess and doesn't follow my instructions"
2026-02-23	1/10	"no. I meant how are they implemented?"
2026-03-02	2/10	"why are you looking in /Users/ianmarr/projects/pai??"
2026-03-04	2/10	"run the local ~/ versions NOT the repo"
2026-02-23	2/10	"7k+31k does not add up to 128695 tokens"
2026-03-01	10/10	"post it to the PAI discussions [executed perfectly]"
2026-02-17	10/10	"that went v smoothly"
2026-02-22	10/10	"yes!"

Problem Sessions

Session	Avg Rating	Themes
bb983feb (Mar 7)	1.0	Freeform output instead of format
f357753d (Mar 2)	3.0	Incomplete answers, wrong repo paths
e92cb4ef (Mar 10)	3.0	Problem persists despite claimed fix
094aebb2 (Feb 16-17)	3.9	Overstepping boundaries, irrelevant info

---

Cross-Referenced Signals (Highest Confidence)

Reflection Theme	Rating Pattern	Reinforcement
Pre-read all targets	Incomplete answers (not reading fully before acting)	STRONG — both stem from acting before understanding
Check existing patterns	Wrong paths/repos	STRONG — not checking existing state before writing
Task archetype detection	Over-researching simple questions	MODERATE — ceremony mismatch with task complexity
Parallelization	Smooth execution earns 10/10	MODERATE — faster = smoother = higher satisfaction

---

Execution Warnings (From Reflections)

1. Repeated "file has not been read" errors from editing before reading (7 occurrences)
2. Sequential execution of independent workstreams (10 occurrences)
3. Writing new utilities without checking if equivalent exists (5 occurrences)
4. Selecting capabilities for optics rather than actual invocation (3 occurrences)
5. Standard-tier tasks getting unnecessary ceremony (2 occurrences)
6. Agent exploration scope too broad — re-exploring areas already covered (3 occurrences)

Aspirational Insights (From Reflections)

• Incremental build validation would catch framework incompatibilities at introduction
• Auto-generate test harnesses alongside scripts for near-instant VERIFY
• Infrastructure-level control (hook gates) is more reliable than prompt-level control
• Proactive project health monitoring could flag problems before user frustration
• When source material is already in context, direct writing can be faster than delegation
• File permission conventions should be copied from sibling files automatically

[Drizzt321]

@virtualian Hm, I grabbed the skill from your repo and ran it. Didn't see it laid out as specific as you put it above. Is your repo up to date?

[virtualian]

Mmmmm, ... maybe the code changed too. Let me find the related closed Issue/PR.

I'm trying to be more disciplined about using Issues and PRs for each functional change I make but I'm not perfect!

[Drizzt321]

Ha! You and me both.

[vpzed]

I find that PAI is a lot better at staying on the rails than vanilla. Also vanilla Claude Code with a subscription is better than other agents with vanilla configs. My example above was using VS Code with GitHub Copilot Chat with a GitHub Enterprise account which exposes a list of models from multiple vendors, and letting "Copilot" choose the model which usually means Open AI Codex.

My point is, at some point, I've had every agent harness and model, including PAI, not follow instructions. I think it's part of what Daniel has talked about many times, scaffolding. PAI has a lot of scaffolding, and it helps. But every setup I've used has this issue. It's just a matter by what degree.

My question to those following this discussion is if others think that instructions (prose in Markdown) can be guardrails versus suggestions, no matter how strongly worded.

[Drizzt321]

My question to those following this discussion is if others think that instructions (prose in Markdown) can be guardrails versus suggestions, no matter how strongly worded.

I think that everything that's not baked into the model/weights/fine-tuning is really a suggestion, although from what I've seen some models are definitely better about following/considering them than others. Unless things are via some kind of deterministic portion, it can't be more than a suggestion. Even baked in, since it's non-deterministic, it can end up sometimes not following the instructions given to it for otherwise the exact same input.

That's just my thoughts, and not having worked with any other system/models than Opus/Sonnet 4.6, mostly through Claude Code. I really am extremely new at using these systems, although I have kept vaguely up with what's going on here and there. I'm certainly paying more attention now though.

[danielmiessler]

Also make sure that your projects file and your steering rules are getting loaded by default on startup. That setting is inside of settings.json.

…

On Fri, Mar 13, 2026 at 3:08 PM, Drizzt321 < ***@***.*** > wrote: Now I recently had it move from a learnings/memory to ~/. claude/ PAI/ USER/ AISTEERINGRULES. md ( http://~/.claude/PAI/USER/AISTEERINGRULES.md ) an extracted rule around not charging ahead for code/work, but that was just last night so haven't really seen if that helps. Or maybe I just hadn't run the needed "take the learnings and materialize them" yet? Is that a thing? I need to do a good, full tour of the expected flow/usage in those terms. — Reply to this email directly, view it on GitHub ( #908 (reply in thread) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAAMLXVVEYAQ7WZ2GVLY4H34QSBGPAVCNFSM6AAAAACWHX7UGSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMJSGQZTKOA ). You are receiving this because you were mentioned. Message ID: <danielmiessler/Personal_AI_Infrastructure/repo-discussions/908/comments/16124358 @ github. com>

[Drizzt321]

Are LEARNINGS loaded by default? Hm. According to my instance, only the most recent 3 system learnings, algorithm learnings, last 2 days of relationship notes. So looks like if it's in LEARNINGS/, it might not actually get loaded.

Exploring where I had those rules, they weren't being actually loaded, since the only place they were referenced was MEMORY.md as an index file listing those others, not actually the rules.

@danielmiessler Is there supposed to be a regular "evaluate learnings and move them to PAI/USER/AISTEERINGRULES.md type process I'm supposed to be doing?

[danielmiessler]

Yeah, there's a workflow in PAI upgrade where you can ask for examples of how to upgrade your particular algorithm. We don't do it by default because it might be custom for you.

…

On Fri, Mar 13, 2026 at 4:54 PM, Drizzt321 < ***@***.*** > wrote: Are LEARNINGS loaded by default? Hm. According to my instance, only the most recent 3 system learnings, algorithm learnings, last 2 days of relationship notes. So looks like if it's in LEARNINGS/, it might not actually get loaded. Exploring where I had those rules, they *weren't* being actually loaded, since the only place they were referenced was MEMORY. md ( http://memory.md/ ) as an index file listing those others, not actually the rules. @ danielmiessler ( https://github.com/danielmiessler ) Is there supposed to be a regular "evaluate learnings and move them to PAI/ USER/ AISTEERINGRULES. md ( http://pai/USER/AISTEERINGRULES.md ) type process I'm supposed to be doing? — Reply to this email directly, view it on GitHub ( #908 (reply in thread) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAAMLXW24XQXKWCQ5YZ7FST4QSNTHAVCNFSM6AAAAACWHX7UGSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMJSGUZDENI ). You are receiving this because you were mentioned. Message ID: <danielmiessler/Personal_AI_Infrastructure/repo-discussions/908/comments/16125225 @ github. com>

[Drizzt321]

Ahhh... Yeah, I need to conduct that tour (letting PAI walk me through itself!) to understand what's there, expectations of things I should be doing over time, etc.

I am finding the council/red teams to be quite helpful, automating that kind of agentic push and pull. Thanks for that!

[jlacour-git]

@virtualian Good questions. Here's how the loop actually works for me:

How rules get created: The Digest skill scans MEMORY/LEARNING/ for failure records, low ratings, and algorithm reflections. It extracts proposals with a specific problem statement, proposed fix, and target file. Each proposal gets classified:

• USER-SAFE → changes to USER/AISTEERINGRULES.md or other user files. Applied directly after approval.
• SYSTEM-PATCH → changes to hooks, tools, or system files. Gets a LOCAL_PATCHES.md entry + upstream issue.
• UPSTREAM-ONLY → needs architectural changes. Issue filed, no local patch.

So yes — the digest output does generate actual rules. But it's not automatic. I review each proposal and decide whether it becomes a steering rule, a hook change, or gets rejected. Human in the loop every time.

The key difference from vanilla /PAIupgrade: The Digest skill is narrowly scoped. It only does the learning-to-rules loop. No development-focused checks, no upgrade scanning. One job.

Closing the loop on synthesis: The raw synthesis output is too verbose for a prompt — you're right about that. The Digest skill's fix is to extract atomic proposals from it. "Tool X failed because Y" becomes a specific rule: "Before X, verify Y." Each one is 1-2 lines in the steering rules file.

I just shared the complete Digest skill files (SKILL.md + workflow) in #946 if you want to try it: https://gist.github.com/jlacour-git/bb9e8b6e88ce7e6afa20fd4251beca37