multimodal: add screenshot quality + budget controls

[Ayush7614]

Problem

When screenshots are used for vision, users have no control over image size/detail or how many screenshots can be captured/sent per turn. This can increase costs and can overwhelm smaller endpoints.

Proposal

Add an "Image budget" section under Settings → Multimodal with controls such as:

• Image detail: low / high (provider-dependent)
• Max screenshots per turn (e.g. 0–5)
• Max image dimension or size cap (e.g. 1024px / 1536px)

Acceptance criteria

• Settings persisted and applied to screenshot capture + vision request formatting.
• Defaults preserve existing behavior.
• UI shows a short explanation of how these settings affect cost/latency.

Test plan

• Manual: set max screenshots to 0/1/2 and verify behavior.
• Manual: change detail/size and verify request payload reflects it.

[esokullu]

it's already quite optimized, but this may be an advanced setting (though I believe very few people would use it tbh)

[Ayush7614]

Fair point — I dug into agent.js and there's already a hardcoded IMAGE_BUDGET (1568px/token caps, ~1.4 MB base64 ceiling, iterative JPEG quality reduction). So the optimization work is largely done; what's missing is user-facing controls.

I'll reframe this as advanced settings that expose existing knobs rather than building new capture logic:

• max dimension / token budget (maps to IMAGE_BUDGET)
• max screenshots per turn (new cap on auto-screenshot injection)
• optional detail preset where the provider supports it

Defaults would stay as-is so current behavior is unchanged. Happy to keep this low-priority / advanced-only if that matches your expectation.

[esokullu]

i think we may only need one configurable param here and that is max screenshots per turn, because the size is really optimum, larger doesn't help much, smaller doesn't help at all. perhaps for very large configurations (like local multiple H200 confs that don't care about wasting tokens, with a vision enabled model) we could enable 2x the ceiling but that would be a hidden setting behind Advanced.

[Ayush7614]

@esokullu great point, and largely matches what the code already does today. I checked the current capture path before replying.

On size/detail: there's intentionally no manual size knob right now — captureAutoScreenshot already auto-fits every image under a hard budget (Agent.IMAGE_BUDGET): each dimension capped at 1568px, a ~1568-token ceiling, and a ~1.4 MB base64 cap (Anthropic's limit), starting at JPEG q=0.75 and iteratively downgrading to q=0.10 only if a high-DPR capture would otherwise overflow. So the "size is already optimal" observation holds — manual size/detail sliders would mostly just let users opt into overflowing a provider's cap, with little real benefit. I'd lean toward keeping size automatic and not surfacing detail/size controls.

On "max screenshots per turn": this is the one genuinely missing lever. Today the only throttle is a 500ms debounce plus the autoScreenshot mode filter (off / on navigation / on state change (default) / every step) in Settings → Display. There's no explicit cap on how many get captured/sent in a single turn, so a long every_step run can still stack several images. A single Max screenshots per turn control (0–5, default ~3) is the cleanest addition and directly addresses the cost/overwhelm concern.

On the 2× ceiling for big local setups: agreed it's best hidden behind an Advanced sub-section rather than cluttering the main control. For local vision models on large GPUs that don't care about token cost, raising the per-turn ceiling (or the size budget) is reasonable, but it shouldn't be a default-exposed toggle.

So my proposed scope, trimmed from the original issue:

• Add Max screenshots per turn (persisted, default preserves current behavior) under Settings → Multimodal, with a one-line cost/latency explanation.
• Keep size/detail automatic (no manual control) — matches your reasoning.
• Optional Advanced subsection to raise the ceiling for high-resource local setups.

I'll update the acceptance criteria to reflect this narrower scope. Want me to open a PR implementing the per-turn cap?