Paper: Understanding Is Getting the Context Right — An OS for Language Models

[MikeyBeez]

Sharing a design paper that proposes treating context management as an operating system concern — directly relevant to how LlamaIndex handles retrieval and context assembly.

Core argument: Context selection, not context length, is the dominant factor in reasoning quality. The paper proposes a two-agent architecture:

• A curator agent (lightweight local model) that continuously manages what enters the reasoning agent's context window
• Threaded conversation history (DAG structure) instead of flat sequential logs — preserving reasoning trajectory rather than retrieving isolated chunks
• Two manifests per turn: a compact topic index for scope awareness, and a curated active context payload
• Provenance-aware metadata distinguishing user-stated facts from search results from model inferences
• Exponential decay with current theory marking — information fades by default, working hypotheses are protected
• A persistent repository that accumulates into an emergent knowledge graph across sessions

The thread-based retrieval approach is a direct contrast to embedding-based chunk retrieval: instead of finding semantically similar fragments, you retrieve the full reasoning chain within a topic. The paper argues this preserves context that chunk retrieval loses.

Paper and PDF: github.com/MikeyBeez/fuzzyOS
DOI: 10.5281/zenodo.18571717

Interested in thoughts from people building retrieval and context assembly systems.