Implement Hippo AI-Generated Insights Memory System

[nikomatsakis]

Hippo AI-Generated Insights Memory System

Status: Active Day Tracking Complete - Ready for Real-World Testing

Current Understanding

Hippo is an AI-generated insights memory system with reinforcement learning. Core hypothesis: AI-generated insights + user reinforcement > manual curation. System generates insights automatically during consolidation moments, applies temporal decay, and uses user feedback to reinforce valuable insights.

Major Update: Implemented research-based temporal scoring with active day tracking for vacation-proof frequency and recency calculations.

Next Steps

• Test with real conversation workflows and insight generation
• Tune relevance formula parameters based on search result quality
• Add basic automated tests for core functionality
• Document usage patterns and insight evolution over time
• Consider adding insight modification/deletion capabilities

Recently Completed

• ✅ Active Day Tracking: Implemented vacation-proof temporal scoring
• ✅ Research-Based Formula: 30% recency + 20% frequency + 35% importance + 15% context
• ✅ Bounded Storage: Max 90 access entries per insight, adapts to usage patterns
• ✅ Automatic Access Recording: Search automatically tracks insight usage
• ✅ Updated Documentation: Design doc reflects new temporal approach
• ✅ MCP Server Fix: Server starts correctly with proper initialization

Open Questions

• Optimal decay rate tuning based on real usage (currently 0.05 per active day)
• Frequency normalization factor (currently /10.0 for max reasonable frequency)
• Search ranking algorithm fine-tuning based on result quality

Context

Memory systems for AI collaboration need to surface relevant insights at the right moments. Traditional approaches require manual curation. Hippo tests whether AI can generate insights automatically and reinforcement learning can identify what's truly valuable.

Key Innovation: Active day counter ensures insights don't decay during vacation periods when the system isn't used - temporal calculations based on actual usage days, not calendar time.

Design documents completed in mdbook:

• Design Document - Updated with active day tracking architecture
• LLM Prompts - LLM usage guidance
• Example Dialog - Realistic workflow demonstration
• Delegate Experiment - Validation test
• Temporal Scoring Research - Research foundation

Ready for real-world validation of the core hypothesis with production-quality temporal scoring.

[nikomatsakis]

🎯 Status Update: MVP Implementation Complete

Date: January 2025
Phase: Core implementation complete, refinement in progress

✅ Major Milestones Achieved

Core Infrastructure

• MCP Server: Full Python implementation with all 4 tools
• JSON Storage: Atomic writes, corruption backup, async I/O
• Type Safety: Complete Pydantic models with Field descriptions
• CLI: Command-line interface with --hippo-file argument

Enhanced Features Beyond MVP

• Semantic Search: Upgraded from substring to sentence transformers (all-MiniLM-L6-v2)
• Local Inference: ~90MB model, completely offline after download
• Composite Relevance: Multi-factor ranking algorithm

🔧 Current Architecture

# Core Models
Insight: votes + situation + importance + temporal_decay
JsonStorage: atomic writes + backup + async
InsightSearcher: semantic matching + fuzzy situation filtering
HippoServer: 4 MCP tools (record/search/modify/reinforce)

MCP Tools Status

1. ✅ hippo_record_insight - Create insights with situation context
2. ✅ hippo_search_insights - Semantic search with relevance ranking
3. ✅ hippo_modify_insight - Update content/situation/importance
4. ✅ hippo_reinforce_insight - Batch upvote/downvote

📊 Key Design Decisions

Original Plan	Implementation Choice	Rationale
"Points"	"Votes"	More intuitive (upvote/downvote)
"Context"	"Situation"	Less overloaded term
Substring matching	Sentence transformers	Semantic understanding
Simple scoring	Composite relevance	Votes + content + situation + importance

🧮 Current Relevance Formula

final_relevance = (
    0.5 * normalized_votes +      # Community validation (50%)
    0.3 * content_relevance +     # Semantic match to query (30%) 
    0.1 * situation_relevance +   # Situational match (10%)
    0.1 * importance             # AI assessment boost (10%)
)

🤔 Active Design Questions

1. Recency vs Frequency Tracking

• Issue: Research formula separates recency/frequency, our model conflates them
• Current: Only track votes_last_modified_at (recency)
• Missing: reinforcement_count field for frequency tracking
• Decision needed: Update model to match research formula?

2. Votes vs Importance Relationship

• Issue: In single-user system, votes and importance measure similar things
• Current: Both influence relevance (potential double-counting)
• Question: Should votes be the definitive importance signal over time?

3. Relevance Formula Weights

• Current weights: Estimated, not research-backed
• Original research: 30% recency + 20% frequency + 35% importance + 15% context
• Decision needed: Align with research or tune based on usage?

🚀 Ready for Testing

The core hypothesis can now be validated:

"AI-generated insights + reinforcement learning > manual curation"

Test Scenarios Ready

• Recording insights during "make it so" moments
• Searching with semantic understanding
• Reinforcement through upvotes/downvotes
• Temporal decay over time

Installation & Usage

uv sync                                    # Install dependencies  
uv run hippo-server --hippo-file ~/hippo.json  # Start MCP server

📋 Next Steps

1. Resolve recency/frequency tracking - Update model or keep current approach?
2. Test with real usage - Generate insights from actual conversations
3. Tune relevance formula - Based on search result quality
4. Add basic tests - Validate core functionality
5. Document usage patterns - How insights evolve over time

[nikomatsakis]

Session summary:

• Completed major model refactoring: unified votes and importance into single importance field
• Simplified conceptual model eliminating vote/importance duality
• Updated reinforcement multipliers to 1.5x upvote, 0.5x downvote (more moderate)
• Adjusted search relevance formula to 60% importance, 30% content, 10% situation

Impact on approach:

• Resolved one of the three active design questions (votes vs importance relationship)
• Identified need for frequency and recency tracking to align with research formula
• Research formula: 30% recency + 20% frequency + 35% importance + 15% context
• Current formula missing separate recency/frequency tracking

Progress: Model refactoring complete and pushed to GitHub. Next: research standard temporal decay approaches for frequency scoring, then implement last_accessed_at and access counting fields.

[nikomatsakis]

Session summary:

• Implemented active day tracking for frequency and recency scoring
• Added daily_access_counts field with max 90 entries, adapts to usage patterns
• Added active_day_counter that only increments on actual usage days (vacation-proof)
• Updated search relevance formula to research-based: 30% recency + 20% frequency + 35% importance + 15% context
• Search now automatically records access for returned insights
• Updated mdbook documentation to reflect new temporal scoring approach

Impact on approach:

• Resolved the frequency/recency tracking design question from the tracking issue
• System is now truly vacation-proof - no decay during inactive periods
• Frequency calculation adapts naturally: frequent use = fine-grained recent history, rare use = coarse-grained long history
• Storage remains bounded (max 90 entries per insight) regardless of usage patterns

Progress: Active day tracking implementation complete and tested. Core temporal scoring hypothesis can now be validated with real usage data. Next: test with actual conversation workflows and tune parameters based on search result quality.

[nikomatsakis]

Session summary:

• Reviewed complete insight lifecycle from creation through frequency/recency calculation
• Completed constants refactoring by updating server.py to use UPVOTE_MULTIPLIER/DOWNVOTE_MULTIPLIER
• Verified the "funny frequency behavior" fix using 30-day sliding window instead of full history
• Confirmed all temporal scoring components work together: active day tracking, windowed frequency, exponential recency decay, and research-based relevance formula

Impact on approach:

• Constants refactoring now complete across entire codebase - all hardcoded values centralized
• System ready for real-world testing with production-quality temporal scoring
• Frequency calculation properly handles long gaps without dilution effects

Progress: Completed constants refactoring. System architecture is solid and ready for validation with actual usage patterns.

[nikomatsakis]

Session summary:

• Fixed subtle bug in search distribution calculation
• Changed from importance_distribution to relevance_distribution in SearchResults
• Distribution now uses same composite relevance formula as search filtering (30% recency + 20% frequency + 35% importance + 15% context)

Impact on approach:

• Search results now accurately reflect what clients see - distribution shows relevance scores that match filtering criteria
• Helps clients understand what additional data exists beyond their filtered results
• More meaningful for understanding search result quality and tuning

Progress: Fixed search distribution bug. System now provides accurate metadata about relevance distribution across all insights for any given query/filter combination.

[nikomatsakis]

Session summary:

• Restructured search logic to separate scoring from filtering
• Renamed _apply_filters to _compute_relevance_scores (no filtering, just scoring)
• Added _calculate_distribution_from_results to avoid duplicate relevance computation
• Distribution now computed from all scored results before any filtering
• Cleaner separation of concerns: scoring → distribution → filtering → pagination

Impact on approach:

• Search flow is now more logical and easier to understand
• Distribution accurately represents what's available for given query/situation filters
• No duplicate relevance calculations - more efficient
• Easier to reason about what distribution represents for clients

Progress: Search architecture restructured. Ready to add unit tests to validate the scoring and filtering logic work correctly.

[nikomatsakis]

Next Phase: Testing Infrastructure

Created subtask issue #2 for unit testing infrastructure. The search system restructuring is complete and ready for validation through integration tests.

Testing approach:

• Integration testing through stable MCP interfaces rather than micro-unit tests
• In-memory storage to avoid disk I/O during tests
• Controllable time advancement to test temporal scoring behavior
• Test scenarios that create insights, advance time, and verify search relevance

This will validate the temporal scoring system we've built and ensure the frequency/recency calculations work as expected over time.