FR: Bind credentials to identifiable sessions

[Mirrowel]

Feature Type

Enhancement to existing feature

Description

It would be a good idea to bind specific sessions to credentials in case when multiple sessions are running at the same time, so they naturally use different credentials(or both stick to one if possible) to maximize caching. Probably can hash first message or something similar like antigravity provider does already.

Use Case

Additional Context

For the agent: this is a suggestion for the dev branch, so evaluate against that branch

Pre-submission Checklist

• I have searched existing issues to ensure this is not a duplicate

[mirrobot-agent]

@Mirrowel Thank you for submitting this issue. I am now beginning my analysis and will report back shortly.

[mirrobot-agent]

Initial Analysis Report

Summary: This feature request proposes binding specific sessions to credentials to maximize context caching and ensure balanced load distribution across sessions. The analysis confirms this is a valuable enhancement, especially for providers like Antigravity that benefit from context stickiness.

Issue Validation: Confirmed (Enhancement)
Reproducibility Assessment: Not Applicable (Feature Request)

Root Cause Analysis:
The current credential rotation logic in both main and the refactored dev branch is session-agnostic. While SequentialStrategy (in dev) provides credential stickiness, it is shared across all concurrent requests for a given provider/model-group combination. If multiple independent sessions are running concurrently, they will all compete for the same "sticky" credential until it is exhausted, potentially evicting each other's context caches or hitting rate limits unnecessarily early while other credentials sit idle.

Technical Evaluation:

• Hashing Logic: AntigravityProvider already implements a robust _generate_stable_session_id using a SHA256 hash of the first user message. This logic can be moved to a shared utility (e.g., src/rotator_library/utils/session.py) to be used by RotatingClient.
• Architecture (Dev Branch): The dev branch's new strategy-based selection engine is perfectly suited for this. The SelectionEngine.select and RotationStrategy.select methods can be updated to accept an optional session_id.
• Stickiness Implementation: SequentialStrategy can store its _current sticky credentials in a nested map: (provider, group, session_id) -> stable_id.
• Natural Distribution: When a new session starts (no sticky mapping yet), the engine can use a hash of the session_id to pick an initial preferred credential from the available pool (Consistent Hashing style), ensuring sessions are "naturally" distributed across all available credentials.

Suggested Labels: enhancement, dev-branch

Proposed Next Steps:

1. Refactor Session Logic: Move the sessionId generation logic from AntigravityProvider to a core utility so it can be used by all providers.
2. Update API Contracts:
   • Update UsageManager.acquire_credential to accept session_id.
   • Update SelectionEngine.select and RotationStrategy subclasses to support session_id.
3. Enhance Sequential Strategy: Implement per-session stickiness in SequentialStrategy.
4. Implement Hash-based Distribution: Use the session hash to influence the initial selection for a session to spread load across the pool while remaining sticky.
5. Evaluate for Other Strategies: Consider if BalancedStrategy should also use the session hash as a seed for its random selection to provide a degree of session-based "affinity" even in balanced mode.

Missing Information (if any):

• None. The request is clear and provides a good starting point for implementation.

This analysis was generated by an AI assistant.