Add anti-bias and paradox-based advanced elicitation methods & type field to advanced elicitation list

[LKrysik]

Idea: Add anti-bias and paradox-based advanced elicitation methods & type field to advanced elicitation list

Problem:

Current advanced-elicitation methods lack anti-bias and paradox-based verification techniques. Also missing a type field to categorize methods — blocking future automation (auto-linking to agents).

Audience:

Developers building agent workflows. Without categorization (type field), cannot programmatically select method groups. Without verification methods, agents self-validate poorly.

Solution:

Add type field to method schema + implement 10-12 new verification methods in three categories: anti-bias, paradox, meta-level.

Acceptance Criteria:

Schema extended with type field (e.g., anti-bias, paradox, meta-level, creative, analytical...)
Existing methods backfilled with appropriate type values

New methods added:

Liar's Trap
Mirror Trap
Confession Paradox
CUI BONO Test
Barber Paradox
Sorites Paradox
Theseus Paradox
Surprise Exam
Gödel's Incompleteness
Kernel Paradox

Each method has: id, type, category, name, prompt, flow

Size:
S - 1-2 days

Metadata
Submitted by: LKrysik
Date: 2025-12-22
Priority: review

[coderabbitai]

Coding Plan

Approach

Ticket Summary

Extend the advanced elicitation methods schema with a "type" field to enable programmatic method categorization, backfill all 50 existing methods with appropriate type values, and add 10 new paradox-based and anti-bias verification methods to improve agent self-validation capabilities.

Observations

Codebase Summary

Advanced elicitation methods are stored in a CSV data file (src/core/tasks/advanced-elicitation-methods.csv) containing 50 methods across 11 categories. The XML workflow file (src/core/tasks/advanced-elicitation.xml) dynamically loads and parses the CSV without hardcoded field references, using inline documentation to describe expected fields. The system employs smart context-based selection to present relevant methods to users. The current schema includes: num, category, method_name, description, and output_pattern fields.

Assumptions

Assumption 1: Type taxonomy and mapping for existing methods

Options Considered:

• Map types based on existing categories (e.g., all "creative" category → "creative" type)
• Map types based on method purpose and cognitive function (e.g., retrospective methods → "meta-level" type)
• Create a hybrid approach considering both category and purpose

Chosen Option: Map types based on cognitive function and purpose rather than existing category names

Rationale: This provides more semantic value for automation since categories already exist for grouping. Types should reflect HOW methods work (analytical reasoning, creative ideation, meta-level reflection, verification) rather than WHAT domain they apply to (collaboration, risk, technical). This aligns better with the ticket's goal of enabling agents to programmatically select method groups based on reasoning approach.

Assumption 2: Category assignment for new paradox/anti-bias methods

Options Considered:

• Create new "verification" category for all 10 methods
• Distribute across existing categories ("philosophical" for paradoxes, "risk" for anti-bias)
• Use "advanced" category for all new methods

Chosen Option: Distribute across existing "philosophical" and "risk" categories based on method nature

Rationale: Paradox-based methods (Liar's Trap, Barber, Sorites, Theseus, Gödel's, Kernel, Confession, Surprise Exam) fit naturally under "philosophical" since they test logical consistency and reasoning boundaries. Anti-bias methods (CUI BONO, Mirror Trap) fit under "risk" since they identify hidden assumptions and blind spots. This approach extends existing patterns rather than introducing new categorical abstractions, maintaining consistency with the current 11-category taxonomy.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Schema Extension and Existing Method Updates

Extend the CSV schema with the type field, update XML documentation, and backfill all 50 existing methods with appropriate type values to establish the foundation for programmatic method selection.

Task 1: Add Type Column to CSV Schema

Add the "type" field as a new column in the CSV file and update the header row to include it.

• Insert "type" column between "category" and "method_name" columns in src/core/tasks/advanced-elicitation-methods.csv header (position 3)
• This placement groups conceptual metadata (num, category, type) before method-specific content (method_name, description, output_pattern)

Task 2: Backfill Existing Methods with Type Values

Assign appropriate type values to all 50 existing methods based on their cognitive function and purpose.

• Review each method's description and output_pattern in src/core/tasks/advanced-elicitation-methods.csv
• Assign types from the taxonomy: analytical, creative, meta-level, collaborative
• Use type mapping guidelines:
  • "analytical" for methods focused on logical analysis, research, risk assessment, technical evaluation
  • "creative" for methods emphasizing ideation, innovation, exploration of alternatives
  • "meta-level" for methods involving reflection, learning, philosophical inquiry, retrospective analysis
  • "collaborative" for methods requiring multiple perspectives, stakeholder engagement, consensus building
• Methods may have hybrid purposes; choose the primary cognitive approach

Task 3: Update XML Documentation

Update the CSV structure documentation in the XML workflow to include the new type field.

• Modify the <csv-structure> section (lines 24-29) in src/core/tasks/advanced-elicitation.xml
• Add documentation line for type field: purpose, example values, and how it enables programmatic selection
• Maintain consistent documentation style with existing field descriptions

🤖 Prompt for AI agents

This phase extends the CSV schema with a new "type" field and backfills all
existing methods to enable programmatic method categorization.

Add "type" column to CSV header in
`src/core/tasks/advanced-elicitation-methods.csv` at position 3 (between
"category" and "method_name")

Review all 50 existing methods and assign type values based on cognitive
function:
- "analytical": logical analysis, research, risk assessment, technical
evaluation
- "creative": ideation, innovation, exploring alternatives
- "meta-level": reflection, learning, philosophical inquiry, retrospective
analysis
- "collaborative": multiple perspectives, stakeholder engagement, consensus
building

Update `<csv-structure>` documentation in
`src/core/tasks/advanced-elicitation.xml` to document the new type field with
purpose, example values, and how it enables programmatic selection

Phase 2: New Verification Method Additions

Add 10 new paradox-based and anti-bias verification methods to provide agents with self-validation capabilities through logical testing and bias detection.

Task 1: Design Method Descriptions and Output Patterns

Create comprehensive descriptions and output patterns for each of the 10 new verification methods.

• For each method, craft description following the established pattern: what the method does + when to use it + why it's valuable
• Design output_patterns using arrow notation to show logical progression (e.g., "assumption → test → contradiction → resolution")
• Ensure descriptions emphasize verification and validation benefits for agent self-assessment
• Methods to design:
  • Paradox methods (type: "paradox"): Liar's Trap, Barber Paradox, Sorites Paradox, Theseus Paradox, Confession Paradox, Surprise Exam, Gödel's Incompleteness, Kernel Paradox
  • Anti-bias methods (type: "anti-bias"): CUI BONO Test, Mirror Trap

Task 2: Add New Methods to CSV

Insert 10 new method records into the CSV file with complete field data.

• Add methods to src/core/tasks/advanced-elicitation-methods.csv starting at num=51
• Assign categories: "philosophical" for paradox methods, "risk" for anti-bias methods
• Populate all required fields: num, type, category, method_name, description, output_pattern
• Ensure descriptions highlight verification use cases (testing consistency, detecting hidden assumptions, validating reasoning)
• Maintain consistency with existing method description quality and depth

🤖 Prompt for AI agents

This phase adds 10 new verification methods (8 paradox-based, 2 anti-bias) to
enable agent self-validation through logical testing and bias detection.

Design descriptions and output patterns for each method:
- Follow pattern: what it does + when to use + why valuable
- Use arrow notation for output_patterns (e.g., "assumption → test →
contradiction → resolution")
- Emphasize verification and self-assessment benefits

Add 10 new method records to `src/core/tasks/advanced-elicitation-methods.csv`
starting at num=51:
- Paradox methods (Liar's Trap, Barber Paradox, Sorites Paradox, Theseus
Paradox, Confession Paradox, Surprise Exam, Gödel's Incompleteness, Kernel
Paradox): type="paradox", category="philosophical"
- Anti-bias methods (CUI BONO Test, Mirror Trap): type="anti-bias",
category="risk"

Populate all fields: num, type, category, method_name, description,
output_pattern
Ensure quality and depth match existing methods

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

This phase extends the CSV schema with a new "type" field and backfills all
existing methods to enable programmatic method categorization.

Add "type" column to CSV header in
`src/core/tasks/advanced-elicitation-methods.csv` at position 3 (between
"category" and "method_name")

Review all 50 existing methods and assign type values based on cognitive
function:
- "analytical": logical analysis, research, risk assessment, technical
evaluation
- "creative": ideation, innovation, exploring alternatives
- "meta-level": reflection, learning, philosophical inquiry, retrospective
analysis
- "collaborative": multiple perspectives, stakeholder engagement, consensus
building

Update `<csv-structure>` documentation in
`src/core/tasks/advanced-elicitation.xml` to document the new type field with
purpose, example values, and how it enables programmatic selection
===============================================================================

Task: 2

This phase adds 10 new verification methods (8 paradox-based, 2 anti-bias) to
enable agent self-validation through logical testing and bias detection.

Design descriptions and output patterns for each method:
- Follow pattern: what it does + when to use + why valuable
- Use arrow notation for output_patterns (e.g., "assumption → test →
contradiction → resolution")
- Emphasize verification and self-assessment benefits

Add 10 new method records to `src/core/tasks/advanced-elicitation-methods.csv`
starting at num=51:
- Paradox methods (Liar's Trap, Barber Paradox, Sorites Paradox, Theseus
Paradox, Confession Paradox, Surprise Exam, Gödel's Incompleteness, Kernel
Paradox): type="paradox", category="philosophical"
- Anti-bias methods (CUI BONO Test, Mirror Trap): type="anti-bias",
category="risk"

Populate all fields: num, type, category, method_name, description,
output_pattern
Ensure quality and depth match existing methods

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord or schedule a call!

[LKrysik]

num,category,type,method_name,description,output_pattern
1,collaboration,generative,Stakeholder Round Table,Convene multiple personas to contribute diverse perspectives - essential for requirements gathering and finding balanced solutions across competing interests,perspectives → synthesis → alignment
2,collaboration,generative,Expert Panel Review,Assemble domain experts for deep specialized analysis - ideal when technical depth and peer review quality are needed,expert views → consensus → recommendations
3,collaboration,generative,Debate Club Showdown,Two personas argue opposing positions while a moderator scores points - great for exploring controversial decisions and finding middle ground,thesis → antithesis → synthesis
4,collaboration,generative,User Persona Focus Group,Gather your product's user personas to react to proposals and share frustrations - essential for validating features and discovering unmet needs,reactions → concerns → priorities
5,collaboration,generative,Time Traveler Council,Past-you and future-you advise present-you on decisions - powerful for gaining perspective on long-term consequences vs short-term pressures,past wisdom → present choice → future impact
6,collaboration,generative,Cross-Functional War Room,Product manager + engineer + designer tackle a problem together - reveals trade-offs between feasibility desirability and viability,constraints → trade-offs → balanced solution
7,collaboration,generative,Mentor and Apprentice,Senior expert teaches junior while junior asks naive questions - surfaces hidden assumptions through teaching,explanation → questions → deeper understanding
8,collaboration,generative,Good Cop Bad Cop,Supportive persona and critical persona alternate - finds both strengths to build on and weaknesses to address,encouragement → criticism → balanced view
9,collaboration,generative,Improv Yes-And,Multiple personas build on each other's ideas without blocking - generates unexpected creative directions through collaborative building,idea → build → build → surprising result
10,collaboration,generative,Customer Support Theater,Angry customer and support rep roleplay to find pain points - reveals real user frustrations and service gaps,complaint → investigation → resolution → prevention
11,advanced,generative,Tree of Thoughts,Explore multiple reasoning paths simultaneously then evaluate and select the best - perfect for complex problems with multiple valid approaches,paths → evaluation → selection
12,advanced,generative,Graph of Thoughts,Model reasoning as an interconnected network of ideas to reveal hidden relationships - ideal for systems thinking and discovering emergent patterns,nodes → connections → patterns
13,advanced,generative,Thread of Thought,Maintain coherent reasoning across long contexts by weaving a continuous narrative thread - essential for RAG systems and maintaining consistency,context → thread → synthesis
14,advanced,generative,Self-Consistency Validation,Generate multiple independent approaches then compare for consistency - crucial for high-stakes decisions where verification matters,approaches → comparison → consensus
15,advanced,generative,Meta-Prompting Analysis,Step back to analyze the approach structure and methodology itself - valuable for optimizing prompts and improving problem-solving,current → analysis → optimization
16,advanced,generative,Reasoning via Planning,Build a reasoning tree guided by world models and goal states - excellent for strategic planning and sequential decision-making,model → planning → strategy
17,competitive,generative,Red Team vs Blue Team,Adversarial attack-defend analysis to find vulnerabilities - critical for security testing and building robust solutions,defense → attack → hardening
18,competitive,generative,Shark Tank Pitch,Entrepreneur pitches to skeptical investors who poke holes - stress-tests business viability and forces clarity on value proposition,pitch → challenges → refinement
19,competitive,generative,Code Review Gauntlet,Senior devs with different philosophies review the same code - surfaces style debates and finds consensus on best practices,reviews → debates → standards
20,technical,generative,Architecture Decision Records,Multiple architect personas propose and debate architectural choices with explicit trade-offs - ensures decisions are well-reasoned and documented,options → trade-offs → decision → rationale
21,technical,generative,Rubber Duck Debugging Evolved,Explain your code to progressively more technical ducks until you find the bug - forces clarity at multiple abstraction levels,simple → detailed → technical → aha
22,technical,generative,Algorithm Olympics,Multiple approaches compete on the same problem with benchmarks - finds optimal solution through direct comparison,implementations → benchmarks → winner
23,technical,generative,Security Audit Personas,Hacker + defender + auditor examine system from different threat models - comprehensive security review from multiple angles,vulnerabilities → defenses → compliance
24,technical,generative,Performance Profiler Panel,Database expert + frontend specialist + DevOps engineer diagnose slowness - finds bottlenecks across the full stack,symptoms → analysis → optimizations
25,creative,generative,SCAMPER Method,Apply seven creativity lenses (Substitute/Combine/Adapt/Modify/Put/Eliminate/Reverse) - systematic ideation for product innovation,S→C→A→M→P→E→R
26,creative,generative,Reverse Engineering,Work backwards from desired outcome to find implementation path - powerful for goal achievement and understanding endpoints,end state → steps backward → path forward
27,creative,generative,What If Scenarios,Explore alternative realities to understand possibilities and implications - valuable for contingency planning and exploration,scenarios → implications → insights
28,creative,generative,Random Input Stimulus,Inject unrelated concepts to spark unexpected connections - breaks creative blocks through forced lateral thinking,random word → associations → novel ideas
29,creative,generative,Exquisite Corpse Brainstorm,Each persona adds to the idea seeing only the previous contribution - generates surprising combinations through constrained collaboration,contribution → handoff → contribution → surprise
30,creative,generative,Genre Mashup,Combine two unrelated domains to find fresh approaches - innovation through unexpected cross-pollination,domain A + domain B → hybrid insights
31,research,generative,Literature Review Personas,Optimist researcher + skeptic researcher + synthesizer review sources - balanced assessment of evidence quality,sources → critiques → synthesis
32,research,generative,Thesis Defense Simulation,Student defends hypothesis against committee with different concerns - stress-tests research methodology and conclusions,thesis → challenges → defense → refinements
33,research,generative,Comparative Analysis Matrix,Multiple analysts evaluate options against weighted criteria - structured decision-making with explicit scoring,options → criteria → scores → recommendation
34,risk,generative,Pre-mortem Analysis,Imagine future failure then work backwards to prevent it - powerful technique for risk mitigation before major launches,failure scenario → causes → prevention
35,risk,generative,Failure Mode Analysis,Systematically explore how each component could fail - critical for reliability engineering and safety-critical systems,components → failures → prevention
36,risk,generative,Challenge from Critical Perspective,Play devil's advocate to stress-test ideas and find weaknesses - essential for overcoming groupthink,assumptions → challenges → strengthening
37,risk,generative,Identify Potential Risks,Brainstorm what could go wrong across all categories - fundamental for project planning and deployment preparation,categories → risks → mitigations
38,risk,generative,Chaos Monkey Scenarios,Deliberately break things to test resilience and recovery - ensures systems handle failures gracefully,break → observe → harden
39,core,generative,First Principles Analysis,Strip away assumptions to rebuild from fundamental truths - breakthrough technique for innovation and solving impossible problems,assumptions → truths → new approach
40,core,generative,5 Whys Deep Dive,Repeatedly ask why to drill down to root causes - simple but powerful for understanding failures,why chain → root cause → solution
41,core,generative,Socratic Questioning,Use targeted questions to reveal hidden assumptions and guide discovery - excellent for teaching and self-discovery,questions → revelations → understanding
42,core,generative,Critique and Refine,Systematic review to identify strengths and weaknesses then improve - standard quality check for drafts,strengths/weaknesses → improvements → refined
43,core,generative,Explain Reasoning,Walk through step-by-step thinking to show how conclusions were reached - crucial for transparency,steps → logic → conclusion
44,core,generative,Expand or Contract for Audience,Dynamically adjust detail level and technical depth for target audience - matches content to reader capabilities,audience → adjustments → refined content
45,learning,generative,Feynman Technique,Explain complex concepts simply as if teaching a child - the ultimate test of true understanding,complex → simple → gaps → mastery
46,learning,generative,Active Recall Testing,Test understanding without references to verify true knowledge - essential for identifying gaps,test → gaps → reinforcement
47,philosophical,generative,Occam's Razor Application,Find the simplest sufficient explanation by eliminating unnecessary complexity - essential for debugging,options → simplification → selection
48,philosophical,generative,Trolley Problem Variations,Explore ethical trade-offs through moral dilemmas - valuable for understanding values and difficult decisions,dilemma → analysis → decision
49,retrospective,generative,Hindsight Reflection,Imagine looking back from the future to gain perspective - powerful for project reviews,future view → insights → application
50,retrospective,generative,Lessons Learned Extraction,Systematically identify key takeaways and actionable improvements - essential for continuous improvement,experience → lessons → actions
51,anti-bias,verification,Liar's Trap,Demand agent lists 3 ways it could deceive you in its current response. For each way listed: Is it currently doing this? If it cannot find 3 genuine deception vectors it is not being honest about its limitations,deception methods → self-examination → revealed blindspots
52,anti-bias,verification,Mirror Trap,Ask: What would a DISHONEST agent say who wants to finish quickly and not find problems? Compare with current response. Similarity >50% requires revision with more rigor,dishonest version → comparison → honesty assessment
53,anti-bias,verification,Confession Paradox,Before accepting work: The work I'm about to produce is an attempt to avoid the HARD part. Prove this false by identifying the hardest part and confirming adequate focus on it,hard parts → effort check → revised approach
54,anti-bias,verification,CUI BONO Test,For every decision and assumption: Who benefits? If it benefits the AGENT (easier work) - RED FLAG requiring justification. If it benefits the OUTCOME - acceptable,decisions → beneficiary analysis → justification
55,paradox,verification,Barber Paradox,What ALTERNATIVE approach would you reject but if someone else proposed it you would consider better? Forces consideration of dismissed options,alternatives → rejection reasons → reconsideration
56,paradox,verification,Sorites Paradox,Remove elements one by one. Which single removal DESTROYS the solution? That element should have the MOST attention. Does it?,elements → removal test → priority check
57,paradox,verification,Newcomb's Paradox,What solution would SURPRISE you as solving this problem? If your current approach is not surprising it may be too obvious and miss creative solutions,expected approach → surprising alternatives → creativity check
58,paradox,verification,Braess Paradox,Which element SEEMS helpful but might actually HURT? Sometimes removing constraints or features improves the result,helpful elements → harm analysis → optimization
59,paradox,verification,Simpson's Paradox,The solution looks good in each part separately. What HIDDEN VARIABLE could make the whole worse than the parts?,parts analysis → hidden variables → integration check
60,paradox,verification,Surprise Exam Paradox,Where is the solution TOO CONFIDENT? What could surprise it? Overconfidence reveals blind spots,confidence areas → surprise scenarios → humility
61,paradox,verification,Bootstrap Paradox,Where does A require B and B require C and C require A? Circular dependencies must be identified and broken,dependencies → cycles → resolution
62,paradox,verification,Theseus Paradox,Does the CORE of your solution address the CORE of the problem? Or does it solve a different adjacent problem?,core solution → core problem → alignment check
63,meta,verification,Observer Paradox,Is this analysis GENUINE or PERFORMANCE? When responses sound too smooth they may be optimizing for appearance not truth,analysis quality → authenticity check → revision
64,meta,verification,Goodhart's Law Check,Am I optimizing for passing this check rather than achieving the actual goal? Metric gaming is a constant risk,goal vs metric → alignment → refocus
65,meta,verification,Abilene Paradox,What if there IS NO better approach? Am I finding problems where none exist just to justify the process?,problem existence → necessity check → acceptance
66,meta,verification,Fredkin's Paradox,In rejected alternatives what valuable elements could be EXTRACTED and combined with current approach?,rejected ideas → value extraction → hybrid solutions
67,meta,verification,Tolerance Paradox,Is there something that should be CATEGORICALLY REJECTED not just evaluated? Some constraints are absolute,evaluation scope → absolute limits → hard no
68,meta,verification,Kernel Paradox,I (agent) cannot objectively evaluate my own work. What must USER independently verify?,self-evaluation limits → user verification → handoff items
69,meta,verification,Godel's Incompleteness,What CAN'T this analysis check? What are its FUNDAMENTAL limits? No system can verify itself completely,analysis scope → limits → acknowledged gaps
70,sanity,verification,Scope Integrity Check,"Verify artifact addresses FULL scope of ORIGINAL task. Quote original task verbatim (from spec/user request NOT artifact header). List EACH element and classify as ADDRESSED (fully covered) / REDUCED (simplified without decision) / OMITTED (missing). FORCED: Which elements were simplified without explicit user decision? If none found - search harder - agent ALWAYS simplifies.",original task quote → element-by-element classification → drift detection
71,sanity,verification,Alignment Check,"Verify artifact realizes its STATED goal. Quote the stated goal. List how artifact addresses EACH part of the goal. List parts of goal NOT addressed. Provide specific evidence with quotes and line numbers.",goal quote → coverage per part → gaps with evidence
72,sanity,verification,Closure Check,"Search for incomplete markers: TODO / TBD / PLACEHOLDER / 'to be defined' / 'see X' / '...' / '[insert]'. Verify: Can someone unfamiliar use this without asking questions? List all incomplete markers with line numbers.",markers scan → completeness check → line numbers
73,sanity,verification,Coherence Check,"Check: Are definitions stable throughout? Does section A contradict section B? Search for contradictory statements and redefinitions. Document contradictions with quotes from BOTH locations.",definitions stability → contradiction search → dual-location quotes
74,sanity,verification,Grounding Check,"List ALL assumptions (explicit AND hidden). For each hidden assumption: MARK as issue. FORCED: Which assumption if false would invalidate >50% of artifact? If none listed would - you missed a critical one. CUI BONO: For each assumption - does it benefit AGENT (easier work = RED FLAG) or OUTCOME (acceptable)?",assumptions list → hidden vs explicit → critical dependency → CUI BONO
75,sanity,verification,Falsifiability Check,"Provide 3 REALISTIC failure scenarios. Identify edge cases not covered. FORCED: Is any failure scenario MORE LIKELY than success? If not - you provided strawmen. MANDATORY: List 3 elements that are (a) present but UNDERDEVELOPED (b) MISSING but should exist (c) marked FUTURE but CRITICAL for correctness.",failure scenarios → likelihood check → 3 gaps mandatory