Skip to content

feat(fim): add Mercury Edit FIM support with provider-based routing#1693

Merged
chrarnoldus merged 6 commits intomainfrom
session/agent_2c490b14-98de-41b6-93fd-37d60b9f47fd
Mar 30, 2026
Merged

feat(fim): add Mercury Edit FIM support with provider-based routing#1693
chrarnoldus merged 6 commits intomainfrom
session/agent_2c490b14-98de-41b6-93fd-37d60b9f47fd

Conversation

@kilo-code-bot
Copy link
Copy Markdown
Contributor

@kilo-code-bot kilo-code-bot bot commented Mar 28, 2026

Summary

Add Mercury Edit FIM model support alongside existing Codestral models. Implements provider-based routing (model name → provider) instead of a non-standard provider field. The changes include:

  • Added INCEPTION_API_KEY environment variable for Inception Labs API authentication
  • Removed provider field from FIM request schema; routing is now inferred from model name
  • Implemented resolveFimProvider() function to map model names to providers (mistral/inception) and upstream URLs
  • Updated BYOK lookup to use provider-specific keys ('codestral' for Mistral, 'inception' for Inception)
  • Added BYOK-required error handling when system API key is missing
  • Implemented provider-aware cost calculation: Inception uses $0.25/$0.75 pricing vs Mistral's $0.3/$0.9
  • Updated Sentry span names from mistral-specific to generic 'fim-*' labels

Verification

  • Modified files: src/lib/config.server.ts, src/app/api/fim/completions/route.ts, src/lib/llm-proxy-helpers.ts
  • Formatting applied with oxfmt

Visual Changes

N/A

Reviewer Notes

  • The inception/mercury-edit model name is normalized to mercury-edit when sent upstream to Inception API
  • Mistral models retain their prefix format (mistralai/codestral-2501codestral-2501)
  • Both Mistral and Inception APIs return OpenAI-compatible FIM responses with usage statistics
  • BYOK API keys take precedence over system keys per the existing pattern
  • Span names updated for consistency (e.g., fim.time_to_first_token_ms instead of mistral.time_to_first_token_ms)

@kilo-code-bot
Copy link
Copy Markdown
Contributor Author

kilo-code-bot bot commented Mar 28, 2026

Code Review Summary

Status: 4 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 2
WARNING 2
SUGGESTION 0
Issue Details (click to expand)

Resolved since last review: the previous ProviderId / inception type mismatch on src/app/api/fim/completions/route.ts:152 remains fixed.

No new issues were found on lines that appear in the latest incremental diff (50b236c5d9b169e01e02389bbd95bba31921b263), so the findings below are carried forward from changed code outside the diff.

Fix these issues in Kilo Cloud

Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

CRITICAL

File Line Issue
src/routers/kiloclaw-router.ts 1061 Stale running rows can be overwritten with a later run's output because controller status is matched only by user/current run state, not by the persisted run identity.
src/routers/kiloclaw-router.ts 1096 Cancel requests terminate the controller's current job without verifying it matches the requested runId, so an older stale row can cancel a newer run.

WARNING

File Line Issue
packages/db/src/schema.ts 3639 kiloclaw_cli_runs stores only user_id, so multi-instance users cannot disambiguate which instance a run belongs to and histories from multiple instances are merged together.
kiloclaw/src/routes/platform.ts 981 The new kilo-cli-run platform endpoints do not parse or forward instanceId, so they always target the legacy user-keyed DO instead of the selected instance.
Files Reviewed (1 file)
  • src/app/api/fim/completions/route.ts

Reviewed by gpt-5.4-20260305 · 324,065 tokens

@chrarnoldus chrarnoldus deleted the session/agent_2c490b14-98de-41b6-93fd-37d60b9f47fd branch March 30, 2026 09:56
@chrarnoldus chrarnoldus restored the session/agent_2c490b14-98de-41b6-93fd-37d60b9f47fd branch March 30, 2026 09:57
@chrarnoldus chrarnoldus reopened this Mar 30, 2026
{ status: 400 }
);
//NOTE: mistral does not do data collection on paid org accounts like ours.
//If we ever support OTHER providers, we need to either ensure they don't
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this note relevant?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we just remove it and collect now?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inception doesn't do collection either

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are real a gaps, we don't enforce the provider blocklist for example, but that's not a regression

// Extract properties for usage context
const promptInfo = extractFimPromptInfo(requestBody);

const byokProviderKey = fimProvider === 'mistral' ? 'codestral' : 'inception';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just make the fim provider be codestral in the first place?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's called mistral everywhere, but this hack is needed for the "not all API keys support all models" feature that mistral has.

@chrarnoldus chrarnoldus merged commit ce90707 into main Mar 30, 2026
15 checks passed
@chrarnoldus chrarnoldus deleted the session/agent_2c490b14-98de-41b6-93fd-37d60b9f47fd branch March 30, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants