Feature request: per-user runtime model switching

[yuyangwu0325]

Summary

This is a feature request for allowing users to switch the Bedrock model used for their session at runtime via a chat command or keyword, without requiring an operator to redeploy the CDK stack.

Motivation

The current architecture supports a single configurable model via default_model_id in cdk.json. This works well for a fixed deployment, but in practice users may want to switch between models for different tasks — for example:

• Use Claude Sonnet as the default (faster, cost-efficient for everyday tasks)
• Switch to Claude Opus temporarily for complex reasoning or multi-step research
• Switch back to the default when done

This is particularly useful for operators who have multiple users with different usage profiles.

Current workaround implemented

We implemented this in our fork using keyword detection in agentcore-proxy.js:

const DEEPTHINK_MODEL = "global.anthropic.claude-opus-4-6-v1";
const modelOverrideMap = new Map(); // keyed by actorId

// In the request handler, after extracting lastUserText:
const lowerText = lastUserText.toLowerCase().trim();
if (/\bdeepthinking\b/.test(lowerText)) {
  modelOverrideMap.set(actorId, DEEPTHINK_MODEL);
} else if (/\bnormalmode\b/.test(lowerText)) {
  modelOverrideMap.delete(actorId);
}
const resolvedModel = modelOverrideMap.get(actorId) || MODEL_ID;

The user sends a message containing the word deepthinking to activate Claude Opus, and normalmode to revert to the default.

Limitation: The override is stored in-memory. When the AgentCore microVM is recycled (idle timeout or max lifetime reached), the preference is lost and the user must set it again.

Proposed better approach

Persist the model preference in the existing DynamoDB identity table as a user attribute (e.g., modelOverride field on the USER# profile item). This would:

1. Survive session restarts — the contract server or proxy loads the preference from DynamoDB at startup
2. Keep separation of concerns — the proxy reads model preference from environment/config rather than parsing message text
3. Be operator-configurable — a second deepthink_model_id key in cdk.json (analogous to subagent_model_id) would let operators define what model the "deep thinking" mode uses

Suggested schema

cdk.json:

{
  "context": {
    "default_model_id": "global.anthropic.claude-sonnet-4-6-v1",
    "deepthink_model_id": "global.anthropic.claude-opus-4-6-v1"
  }
}

DynamoDB identity table — user profile item:

{
  "PK": "USER#user_abc123",
  "SK": "PROFILE",
  "modelOverride": "global.anthropic.claude-opus-4-6-v1"
}

Flow:

1. Contract server passes MODEL_OVERRIDE env var to proxy on startup (fetched from DynamoDB)
2. Proxy uses MODEL_OVERRIDE || MODEL_ID for all requests
3. When user sends keyword (or a proper /model command handled in Router Lambda), Router Lambda updates the DynamoDB item — no container restart needed (takes effect next session)

Alternatives considered

• OpenClaw tool/skill for model switching — would require OpenClaw to have a mechanism to change its own inference model mid-session, which it does not support
• Per-session env var via AgentCore — AgentCore doesn't currently expose per-session environment variable injection
• Handled entirely in Router Lambda — the Router Lambda could detect model-switch commands before forwarding to AgentCore and update DynamoDB accordingly, keeping the proxy unchanged

Impact

This is a non-breaking, additive feature. The existing default_model_id behaviour is unchanged. The deepthink_model_id defaults to empty (disabled).

Would welcome feedback on the right layer to implement this — proxy, contract server, or Router Lambda.

[zenttic-aws]

Thanks for the detailed feature request and for sharing your working implementation! This is a great idea.

To confirm: this feature is not yet implemented in the main codebase. The current architecture only supports a single default_model_id set at deploy time via cdk.json.

Your proposed approach — persisting the model preference in DynamoDB and reading it at session startup — is the right direction. It cleanly separates the operator configuration (deepthink_model_id in cdk.json) from the user preference (DynamoDB profile item), and survives container recycling which the in-memory approach does not.

We'd welcome a PR implementing this. The suggested flow (Router Lambda detects command → updates DynamoDB → contract server reads modelOverride at init) keeps the proxy unchanged and is consistent with how other per-user state is already managed in the identity table.

Leaving this open as a tracked feature request.

[zenttic-aws]

Update (2026-03-18): This feature request is still open — per-user runtime model switching is not yet implemented in main.

However, PR #45 (just merged) provides important prerequisite infrastructure:

• Cross-region inference profiles now work correctly — global.* and us.* model IDs (e.g. global.anthropic.claude-opus-4-6-v1 for the deepthink model) previously hung silently when routed through the regional Bedrock Runtime VPC endpoint. The proxy now detects cross-region profile prefixes and routes them via NAT, so both the default and deepthink models will work regardless of prefix.
• IAM execution role now includes arn:aws:bedrock:*::inference-profile/* (system-defined cross-region profiles), which is required for global.* model IDs.

These are blocking infrastructure fixes that any implementation of this feature would have needed. The feature itself (DynamoDB-backed model preference, keyword detection or /model command, contract server reading modelOverride at init) remains to be built — the suggested approach in the original issue is still the right direction.

[zenttic-aws]

Update (2026-03-18): This feature has been implemented. PR #46 is open and ready for review.

What's implemented

• Router Lambda — Detects deepthinking, normalmode, normalthinking, /model opus, /model default commands across all three channel handlers (Telegram, Slack, Feishu). Persists modelOverride on the user's DynamoDB PROFILE item and forwards it in every AgentCore invocation payload.
• Contract server — Reads modelOverride from the invocation payload and writes it to the shared identity file (/tmp/current-identity.json).
• Proxy — New getModelOverride() reads the override from the identity file per-request. resolveModelId() uses it when set, falling back to MODEL_ID. Subagent requests always use SUBAGENT_BEDROCK_MODEL_ID regardless of override.
• CDK — New deepthink_model_id context key (default empty = feature disabled). Passed as DEEPTHINK_MODEL_ID env var to Router Lambda.

The implementation follows the exact design proposed in this issue — DynamoDB-backed persistence (survives container recycling), operator-configurable via cdk.json, feature-gated when deepthink_model_id is absent.

Test coverage

• 19 unit tests for router model switching (detection, DynamoDB persistence, payload forwarding, feature-disabled guard) — ✅ all passing
• 8 unit tests for proxy MODEL_OVERRIDE resolution — ✅ all passing
• Full bridge test suite (346 tests) — ✅ all passing

Closing when PR #46 is merged.