feat: Cross-Session Learning & Knowledge Accumulation

[lsm]

Purpose

Enable NeoKai to accumulate expertise across sessions, learning from each interaction and building a persistent knowledge base that improves future performance. This is essential for AGI-level autonomy because:

• Compound learning: Each session makes NeoKai more capable
• Project expertise: Deep knowledge of specific codebases
• Skill acquisition: Learning new capabilities from experience
• Best practices: Capturing and applying learned patterns

Without cross-session learning, NeoKai starts fresh each time and cannot develop expertise.

---

Current State

NeoKai has:

• Per-room memory storage
• No systematic knowledge extraction
• No cross-project learning
• No skill acquisition mechanism

Each session operates largely independently, with limited carryover of learned knowledge.

---

Proposed Approach

Phase 1: Knowledge Base Architecture

1. Knowledge Layers

   interface KnowledgeBase {
     // Global knowledge (all projects)
     global: GlobalKnowledge;
     
     // Project-specific knowledge
     projects: Map<ProjectId, ProjectKnowledge>;
     
     // User preferences
     userPreferences: UserPreferences;
     
     // Skills and capabilities
     skills: SkillRegistry;
   }
   
   interface GlobalKnowledge {
     patterns: Pattern[];
     antiPatterns: AntiPattern[];
     bestPractices: BestPractice[];
     toolUsage: ToolUsageKnowledge[];
     domainKnowledge: DomainKnowledge[];
   }
   
   interface ProjectKnowledge {
     architecture: ArchitectureUnderstanding;
     conventions: CodeConvention[];
     dependencies: DependencyKnowledge[];
     hotspots: CodeHotspot[];  // Areas with frequent changes
     gotchas: ProjectGotcha[];
     testingPatterns: TestingPattern[];
   }

2. Knowledge Storage

   interface KnowledgeEntry {
     id: string;
     type: KnowledgeType;
     content: string;
     
     // Provenance
     source: 'extracted' | 'user_provided' | 'inferred';
     sessionOrigin?: SessionId;
     projectOrigin?: ProjectId;
     
     // Metadata
     confidence: number;
     applicability: string[];  // Tags for when this applies
     lastVerified: Date;
     usageCount: number;
     
     // Relationships
     relatedEntries: string[];
     supersedes?: string[];  // If this replaces older entries
   }

Phase 2: Knowledge Extraction Pipeline

1. Session Analysis

   interface KnowledgeExtractor {
     // Analyze completed session
     analyzeSession(session: SessionLog): Promise<ExtractedKnowledge>;
     
     // Identify patterns worth remembering
     identifyPatterns(session: SessionLog): Promise<Pattern[]>;
     
     // Extract best practices
     extractBestPractices(session: SessionLog): Promise<BestPractice[]>;
     
     // Identify anti-patterns
     identifyAntiPatterns(session: SessionLog): Promise<AntiPattern[]>;
   }

2. Extraction Triggers

   const extractionTriggers = {
     // Successful patterns
     taskSuccess: {
       extract: ['approach', 'techniques', 'tools_used'],
       confidence: 0.8
     },
     
     // Failed patterns (anti-patterns)
     taskFailure: {
       extract: ['what_failed', 'why_failed', 'what_worked_instead'],
       confidence: 0.7
     },
     
     // User corrections
     userCorrection: {
       extract: ['correction', 'reason'],
       confidence: 0.95
     },
     
     // Novel discoveries
     newInsight: {
       extract: ['insight', 'context'],
       confidence: 0.6
     }
   };

3. LLM-Assisted Extraction

   const extractionPrompt = `
   Analyze this session and extract knowledge worth remembering:
   
   Session summary: {summary}
   Key actions taken: {actions}
   Outcomes: {outcomes}
   
   Extract:
   1. Patterns that worked well (could be reused)
   2. Patterns that didn't work (should be avoided)
   3. Project-specific conventions discovered
   4. General insights applicable to other sessions
   
   For each, provide:
   - The knowledge content
   - When it applies (context tags)
   - Confidence level (how certain)
   - Why it's worth remembering
   `;

Phase 3: Knowledge Application

1. Context Injection

   interface KnowledgeInjector {
     // Inject relevant knowledge into prompts
     injectKnowledge(
       prompt: string,
       context: SessionContext
     ): Promise<EnrichedPrompt>;
     
     // Find applicable knowledge
     findApplicableKnowledge(
       context: SessionContext
     ): Promise<KnowledgeEntry[]>;
   }

2. Prompt Enhancement

   // Example enhanced prompt:
   const enhancedPrompt = `
   ${basePrompt}
   
   ## Relevant Project Knowledge
   - This project uses tab indentation (detected from 47 files)
   - The auth module requires special handling (learned from session #123)
   - Tests must be run before commits (user preference from session #89)
   
   ## Known Gotchas
   - The database migration script occasionally fails; retry works
   - E2E tests are flaky on the login flow; check for race conditions
   
   ## Applicable Best Practices
   - When modifying shared types, update all consumers
   - Run 'bun run check' after any significant changes
   `;

3. Knowledge Retrieval

   interface KnowledgeRetrieval {
     // Semantic search across knowledge base
     semanticSearch(query: string): Promise<KnowledgeEntry[]>;
     
     // Get knowledge by context tags
     getByTags(tags: string[]): Promise<KnowledgeEntry[]>;
     
     // Get most relevant knowledge for current context
     getRelevant(context: SessionContext): Promise<KnowledgeEntry[]>;
   }

Phase 4: Skill Acquisition System

1. Skill Definition

   interface Skill {
     id: string;
     name: string;
     description: string;
     
     // When this skill is applicable
     triggers: SkillTrigger[];
     
     // How to execute this skill
     procedure: SkillStep[];
     
     // Prerequisites
     prerequisites: string[];
     
     // Proficiency tracking
     proficiency: number;  // 0-100
     successfulApplications: number;
     failedApplications: number;
   }

2. Skill Learning

   interface SkillLearner {
     // Learn skill from successful execution
     learnFromSuccess(
       task: Task,
       execution: ExecutionTrace
     ): Promise<Skill>;
     
     // Improve existing skill
     improveSkill(
       skill: Skill,
       execution: ExecutionTrace
     ): Promise<Skill>;
     
     // Identify skill gaps
     identifyGaps(): Promise<string[]>;
   }

3. Skill Application

   interface SkillApplicator {
     // Check if any skills apply to current task
     findApplicableSkills(task: Task): Promise<Skill[]>;
     
     // Execute skill procedure
     executeSkill(skill: Skill, context: Context): Promise<Result>;
   }

Phase 5: Knowledge Maintenance

1. Knowledge Lifecycle

   interface KnowledgeLifecycle {
     // Verify knowledge is still accurate
     verify(entry: KnowledgeEntry): Promise<boolean>;
     
     // Update outdated knowledge
     update(entry: KnowledgeEntry): Promise<KnowledgeEntry>;
     
     // Archive deprecated knowledge
     archive(entry: KnowledgeEntry): Promise<void>;
     
     // Merge duplicate knowledge
     deduplicate(): Promise<void>;
   }

2. Conflict Resolution

   interface ConflictResolver {
     // Detect conflicting knowledge
     detectConflicts(): Promise<KnowledgeConflict[]>;
     
     // Resolve conflicts
     resolve(conflict: KnowledgeConflict): Promise<Resolution>;
   }

---

Technical Considerations

Storage Architecture

• Centralized vs distributed knowledge storage
• Indexing for efficient retrieval
• Handling large knowledge bases

Knowledge Quality

• Preventing accumulation of low-quality knowledge
• Handling contradictory knowledge
• Measuring knowledge utility

Privacy & Security

• What knowledge should be shared vs private
• Handling sensitive information
• User control over knowledge accumulation

Performance

• Retrieval speed for large knowledge bases
• Incremental updates vs full rebuilds
• Caching strategies

---

Success Metrics

1. Knowledge Utilization: % of sessions that use accumulated knowledge
2. Learning Velocity: Rate of knowledge accumulation over time
3. Quality Improvement: Correlation between knowledge usage and success rate
4. Skill Proficiency: Improvement in skill success rates over time

---

Implementation Roadmap

1. Phase 1: Basic knowledge storage and retrieval
2. Phase 2: Session analysis and extraction
3. Phase 3: Context injection and application
4. Phase 4: Skill acquisition system
5. Phase 5: Knowledge maintenance lifecycle

---

Questions for Discussion

1. Should knowledge be shared across all users or per-user?
2. How to handle knowledge that becomes outdated?
3. What's the right granularity for knowledge entries?
4. Should users be able to manually add/edit knowledge?

---

Part of the AGI-Level Autonomy initiative