fix: subscription probe no longer filters accessible models (#1448)

[Fixxu]

Problem

When a user connects with a Claude.ai subscription token (OAuth sk-ant-oat-*), the model discovery probe incorrectly filters out sonnet and opus model families from the model picker. Users with valid Claude Max/Pro subscriptions can only see haiku models.

Reported in: #1448

Root Cause

The probeModel() function in anthropic-subscription-probe.ts treats all HTTP 400 responses as "subscription tier doesn't include this model family":

if (res.status === 400) return false;  // ← treats format errors as tier blocks

But Anthropic returns 400/invalid_request_error for probe format issues — not tier restrictions. The probe never reads the error response body to distinguish between the two.

Fix

• Parse the error response body instead of blindly treating HTTP 400 as blocked
• Only filter models for genuine tier errors: authentication_error, permission_error, not_found_error
• Treat invalid_request_error as a probe format issue → keep the model
• Handle 403 responses (future-proofing)
• Default to keeping models when error body can't be parsed
• Wrapped filterBySubscriptionAccess() in try/catch in model-discovery.service.ts so probe failures don't nuke the entire model list

Tests

• Updated existing mock to use permission_error (403) for genuine tier restrictions
• Added test: invalid_request_error keeps models (the core bug scenario)
• Added test: permission_error correctly removes models
• Added test: unparseable error body keeps models (graceful degradation)

Files Changed

• packages/backend/src/model-discovery/anthropic-subscription-probe.ts
• packages/backend/src/model-discovery/model-discovery.service.ts
• packages/backend/src/model-discovery/anthropic-subscription-probe.spec.ts

---

Summary by cubic

Fixes Anthropic subscription probing so Claude.ai subscription tokens no longer hide accessible sonnet and opus models in the picker. The probe now reads error types and only filters on real permission/auth errors; invalid_request_error keeps the model.

• Bug Fixes
  • Parse error body and only block on authentication_error, permission_error, or not_found_error (incl. 403).
  • Treat invalid_request_error and unparseable bodies as non-blocking; default to keep models.
  • Wrap filterBySubscriptionAccess() in model-discovery.service.ts with try/catch to avoid dropping all models on probe failure.
  • Tests updated to cover kept-on-invalid_request_error, removed-on-permission_error, and kept-on-bad-JSON scenarios.

^{Written for commit e532869. Summary will update on new commits.}

[cubic-dev-ai]

No issues found across 5 files

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.68%. Comparing base (63dc2aa) to head (e532869).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1451   +/-   ##
=======================================
  Coverage   98.68%   98.68%           
=======================================
  Files         117      117           
  Lines        7349     7349           
  Branches     2855     2855           
=======================================
  Hits         7252     7252           
  Misses         88       88           
  Partials        9        9           

Flag	Coverage Δ
frontend	98.76% <ø> (ø)
manifest	97.72% <ø> (ø)
plugin	94.66% <ø> (ø)
shared	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.