Refactor: System Message Injection for Pruning Context #87
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove brittle 'synthetic injection' logic that modified user messages. - Introduce 'injectSystemMessage' to FormatDescriptor interface. - Implement system message injection for OpenAI, Bedrock, and Gemini. - Update 'prune' tool prompt to focus on distillation workflow. - Remove obsolete prompt files (synthetic.txt, system-reminder.txt). - Simplify handler.ts to treat prunable lists as system instructions.
- Implement dedicated 'anthropic' format adapter in lib/fetch-wrapper/formats/anthropic.ts. - Handle top-level 'system' field injection (supports string and array). - Handle 'tool_result' content blocks in user messages. - Update fetch wrapper detection order to prioritize Anthropic over OpenAI-compatible. - Fixes incompatibility where OpenAI adapter would inject invalid 'system' role messages into Anthropic requests.
- Apply code formatting standards. - Add documentation files (ANTHROPIC_IMPLEMENTATION.md, IMPLEMENTATION_SUMMARY.md). - Include test-anthropic-format.js for verification.
Removed temporary documentation files ANTHROPIC_IMPLEMENTATION.md and IMPLEMENTATION_SUMMARY.md as requested.
Changed 'SYSTEM ROLE' to 'SYSTEM PROMPT' in tool.txt to be provider-agnostic, as not all providers use the term 'role' for system instructions.
- Add baseline synthetic.txt with context management protocol always present - Enhance nudge.txt with urgent, actionable context warnings - Update tool.txt with three distinct pruning scenarios (Task Completion, Noise Removal, Context Conservation) - Refactor prunable-list.ts to always inject synthetic instruction, optionally add nudge - Clarify when distillation is required vs forbidden for each scenario
- Split context management injection: core protocol remains in system message, but dynamic tool list moves to user message for better adherence - Add injectUserMessage support to all provider formats - Update synthetic prompt with system-reminder to prevent model from referencing invisible injected content
- Centralize formatting logic in display-utils.ts to share between tool output and notifications - Update pruning tool to return a detailed, human-readable summary of actions taken - Simplify notification logic by reusing shared formatting functions
…ions Allows LLM to specify why it's pruning (completion, noise, consolidation) as the first element of the ids array. Reason is displayed in UI notifications but hidden from tool output to avoid redundancy.
… new user message This change addresses models consistently responding to the injected prunable list even when instructed not to. By appending to the last assistant message instead of creating a new user message, the list appears as part of the model's own context state awareness rather than user input to respond to. - Rename injectUserMessage -> appendToLastAssistantMessage across all formats - Each format handler now finds and appends to the last assistant/model message - Update synthetic.txt prompt to reflect the new assistant message framing - Rename buildUserInjection -> buildAssistantInjection
- Streamline synthetic.txt system-reminder to single sentence - Rename 'CONTEXT CONSERVATION' to 'CONTEXT CONSOLIDATION' for clarity - Update tool.txt to reference assistant messages (matching new injection method) - Set nudge_freq default to 0 (disabled)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR refactors the plugin to use System Message Injection instead of "synthetic user message injection" for communicating pruning instructions to the model.
Key Changes
injectSynthandinjectPrunableListinFormatDescriptorwith a singleinjectSystemMessagemethod.synthetic.txtandsystem-reminder.txtas they are no longer needed.tool.txt(Prune tool definition) to explicitly enforce the "Distill before Pruning" workflow using strict instructions.systemarray.systemInstruction.parts.Motivation
The previous approach of hiding instructions inside user messages (synthetic injection) was brittle and complex to maintain. Moving these instructions to the System Prompt is cleaner, more robust, and leverages the model's native understanding of system-level constraints.
Impact
handler.tslogic significantly.Status: Experimental. Needs testing to ensure models respect dynamic system prompt updates.