Skip to content

fix(qa): use the provider's own model for direct-API QA sessions#338

Merged
OBenner merged 1 commit into
developfrom
fix/qa-runtime-provider-model-coherence
Jun 13, 2026
Merged

fix(qa): use the provider's own model for direct-API QA sessions#338
OBenner merged 1 commit into
developfrom
fix/qa-runtime-provider-model-coherence

Conversation

@OBenner

@OBenner OBenner commented Jun 13, 2026

Copy link
Copy Markdown
Owner

Находка живой сборки №8 (QA-аналог #337)

Финальный прогон (спек 901) — первый, где QA-роли реально дошли до прямого провайдера (после #317, удалившего опт-ин). Маршрутизация сработала: qa_reviewer ушёл на openai через гейт. Но QA-сессия собралась с дефолтом QA-фазы (phase_config отдаёт claude-*) → OpenAI 404 на claude-sonnet-4-5-20250929.

Это точный аналог #337, но в QA-пути: как только маршрут корректно отправляет QA на прямого провайдера, Claude-центричный дефолт фазы протекает в сессию.

Фикс

Новый ProviderConfig.coherent_session_model() возвращает собственную модель провайдера, когда запрошенный id — claude-family, а провайдер не-Claude; иначе пропускает запрос как есть. Оба QA runtime-shim'а (run_qa_reviewer_runtime_session, _build_qa_fixer_runtime_session) прогоняют модель через него перед create_session. Recovery-цикл fixer'а уже использует get_fallback_model() (паритет #316), так что при корректной начальной модели чейн gpt-5.2 → gpt-5 → … остаётся когерентным.

Tests

5 кейсов хелпера (claude→провайдер, явная сохраняется, None, claude-провайдер без изменений, нет настроенной модели → запрос как есть) + существующие QA runtime / factory сьюты. 85 passed, ruff clean.

Восьмой фикс серии живой сборки (#332#337 → этот). После него QA-фаза на прямом провайдере не падает на старте по 404.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Enhanced model selection logic to prevent incompatibilities between AI providers and requested models, reducing errors during QA operations.
    • Improved provider configuration consistency across QA processing phases.
  • Tests

    • Added comprehensive tests for AI provider and model compatibility validation.

Live-build finding (2026-06-13, spec 901): with QA roles routed to a
direct provider (qa_reviewer/qa_fixer on openai via the promotion gate,
now that #317 dropped the opt-in), the QA runtime session was built with
the QA phase-model DEFAULT — which phase_config resolves to a claude-*
id — and OpenAI 404'd on claude-sonnet-4-5-20250929. This is the QA-path
analog of the coder fix in #337: once routing correctly sends QA to the
direct provider, the Claude-centric phase default leaks straight into
the session.

New ProviderConfig.coherent_session_model() returns the provider's own
configured model when the requested model is a Claude-family id and the
provider is non-Claude; otherwise it passes the request through
unchanged. Both QA runtime shims (run_qa_reviewer_runtime_session,
_build_qa_fixer_runtime_session) now route their model through it before
create_session. The qa_fixer recovery loop already uses
get_fallback_model() (#316 parity), so once the initial model is the
provider's, the fallback chain (gpt-5.2 -> gpt-5 -> ...) stays coherent.

Tests: 5 helper cases (claude->provider, explicit preserved, None,
claude-provider unchanged, no-configured-model keeps request) + existing
QA runtime / factory suites. 85 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Oleg Miagkov <mrobenner@gmail.com>
@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ae07b5ef-6ba8-46ca-9f24-e3c9426b5b33

📥 Commits

Reviewing files that changed from the base of the PR and between 83c5f49 and fb669cf.

📒 Files selected for processing (4)
  • apps/backend/core/providers/config.py
  • apps/backend/qa/fixer.py
  • apps/backend/qa/reviewer.py
  • tests/test_provider_coherent_model.py

📝 Walkthrough

Walkthrough

This PR adds provider-model coherence checking to prevent Claude-family model IDs from being used with non-Claude providers. A new coherent_session_model() method on ProviderConfig substitutes the configured provider's model when a Claude-family request arrives on a non-Claude provider. The method is integrated into QA fixer and reviewer runtime sessions, and comprehensive tests validate behavior across five scenarios.

Changes

Provider-model coherence for QA sessions

Layer / File(s) Summary
Coherent model selection in ProviderConfig
apps/backend/core/providers/config.py
ProviderConfig.coherent_session_model() method accepts a requested model ID and returns the configured provider's own model when the provider is non-Claude and the requested model is Claude-family; otherwise returns the requested model unchanged (or None if passed).
QA fixer and reviewer runtime session updates
apps/backend/qa/fixer.py, apps/backend/qa/reviewer.py
QA fixer and reviewer runtime sessions now route the requested model through coherent_session_model() when creating their provider sessions, ensuring direct-provider configurations do not attempt to use Claude-family models.
Coherent model method test suite
tests/test_provider_coherent_model.py
New test module with five test cases verifies that the method replaces Claude-family models for direct providers, preserves explicitly set provider models, passes through None, keeps Claude models when the provider is Claude, and falls back to requested models when the provider has no configured model.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • OBenner/Auto-Coding#316: Both PRs modify apps/backend/qa/fixer.py's runtime session model selection—main PR routes the requested model through ProviderConfig.coherent_session_model(...), while #316 rebuilds/adjusts the runtime session per recovery attempts (including model fallback).
  • OBenner/Auto-Coding#276: Main PR updates apps/backend/qa/reviewer.py's run_qa_reviewer_runtime_session() to pass model through ProviderConfig.coherent_session_model(), which directly impacts the same runtime QA reviewer execution path.

Suggested labels

area/backend, size/M

Poem

🐰 A model must find its provider's way,
No Claude-dreams when OpenAI's in play!
With coherence checked and QA tests passed,
Runtime sessions are aligned at last. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: adding coherent session model logic to prevent Claude-family model IDs from being used in direct-provider QA sessions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/qa-runtime-provider-model-coherence

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added area/backend bug Something isn't working size/S labels Jun 13, 2026
@sonarqubecloud

Copy link
Copy Markdown

@OBenner OBenner merged commit eb86b8f into develop Jun 13, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/backend bug Something isn't working size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant