Skip to content

feat: OpenAI Native service tier support (selection, API fallback, resolved-tier costing, tiered pricing UI) #7643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

feat: OpenAI Native service tier support (selection, API fallback, resolved-tier costing, tiered pricing UI) #7643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion webview-ui/src/components/settings/ApiOptions.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,11 @@ const ApiOptions = ({
)}

{selectedProvider === "openai-native" && (
<OpenAI apiConfiguration={apiConfiguration} setApiConfigurationField={setApiConfigurationField} />
<OpenAI
apiConfiguration={apiConfiguration}
setApiConfigurationField={setApiConfigurationField}
selectedModelInfo={selectedModelInfo}
Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this implementation complete? The UI is passing selectedModelInfo to the OpenAI component, but there's no backend implementation to actually use the service tier selection. The following files mentioned in the PR description are missing their implementations:

  • packages/types/src/providers/openai.ts - No service tier support
  • src/api/providers/openai-native.ts - No service tier handling
  • src/api/providers/__tests__/openai-native-service-tier.spec.ts - Test file doesn't exist

Without these backend changes, the UI will set the service tier but it won't actually be used in API requests.

/>
)}

{selectedProvider === "mistral" && (
Expand Down
94 changes: 81 additions & 13 deletions webview-ui/src/components/settings/ModelInfoView.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,14 @@ export const ModelInfoView = ({
}: ModelInfoViewProps) => {
const { t } = useAppTranslation()

const infoItems = [
// Show tiered pricing table for OpenAI Native when model supports non-standard tiers
const allowedTiers =
(modelInfo?.allowedServiceTiers || []).filter((tier) => tier === "flex" || tier === "priority") ?? []
Comment on lines +29 to +30
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nullish coalescing operator (??) is redundant since the logical OR (||) already provides an empty array as fallback. Remove the ?? [] at the end.

Suggested change
const allowedTiers =
(modelInfo?.allowedServiceTiers || []).filter((tier) => tier === "flex" || tier === "priority") ?? []
(modelInfo?.allowedServiceTiers || []).filter((tier) => tier === "flex" || tier === "priority")

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nullish coalescing operator (??) is redundant here since the logical OR (||) already provides an empty array as fallback.

Suggested change
(modelInfo?.allowedServiceTiers || []).filter((tier) => tier === "flex" || tier === "priority") ?? []
const allowedTiers =
(modelInfo?.allowedServiceTiers || []).filter((tier) => tier === "flex" || tier === "priority")

Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical issue: The properties allowedServiceTiers and serviceTierPricing don't exist in the ModelInfo type. This will cause TypeScript compilation errors. The type definitions need to be added to packages/types/src/model.ts first.

const tierPricing = modelInfo?.serviceTierPricing
const shouldShowTierPricingTable = apiProvider === "openai-native" && allowedTiers.length > 0 && !!tierPricing
const fmt = (n?: number) => (typeof n === "number" ? `${formatPrice(n)}` : "—")

const baseInfoItems = [
typeof modelInfo?.contextWindow === "number" && modelInfo.contextWindow > 0 && (
<>
<span className="font-medium">{t("settings:modelInfo.contextWindow")}</span>{" "}
Expand Down Expand Up @@ -53,6 +60,21 @@ export const ModelInfoView = ({
supportsLabel={t("settings:modelInfo.supportsPromptCache")}
doesNotSupportLabel={t("settings:modelInfo.noPromptCache")}
/>,
apiProvider === "gemini" && (
<span className="italic">
{selectedModelId.includes("pro-preview")
? t("settings:modelInfo.gemini.billingEstimate")
: t("settings:modelInfo.gemini.freeRequests", {
count: selectedModelId && selectedModelId.includes("flash") ? 15 : 2,
})}{" "}
<VSCodeLink href="https://ai.google.dev/pricing" className="text-sm">
{t("settings:modelInfo.gemini.pricingDetails")}
</VSCodeLink>
</span>
),
].filter(Boolean)

const priceInfoItems = [
modelInfo?.inputPrice !== undefined && modelInfo.inputPrice > 0 && (
<>
<span className="font-medium">{t("settings:modelInfo.inputPrice")}:</span>{" "}
Expand All @@ -77,20 +99,10 @@ export const ModelInfoView = ({
{formatPrice(modelInfo.cacheWritesPrice || 0)} / 1M tokens
</>
),
apiProvider === "gemini" && (
<span className="italic">
{selectedModelId.includes("pro-preview")
? t("settings:modelInfo.gemini.billingEstimate")
: t("settings:modelInfo.gemini.freeRequests", {
count: selectedModelId && selectedModelId.includes("flash") ? 15 : 2,
})}{" "}
<VSCodeLink href="https://ai.google.dev/pricing" className="text-sm">
{t("settings:modelInfo.gemini.pricingDetails")}
</VSCodeLink>
</span>
),
].filter(Boolean)

const infoItems = shouldShowTierPricingTable ? baseInfoItems : [...baseInfoItems, ...priceInfoItems]

return (
<>
{modelInfo?.description && (
Expand All @@ -106,6 +118,62 @@ export const ModelInfoView = ({
<div key={index}>{item}</div>
))}
</div>

{shouldShowTierPricingTable && (
<div className="mt-2">
<div className="text-xs text-vscode-descriptionForeground mb-1">
Pricing by service tier (price per 1M tokens)
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User‐facing text in the tiered pricing table header and column labels is hard-coded. Consider internationalizing these strings using t() for consistency.

Suggested change
Pricing by service tier (price per 1M tokens)
{t('settings:modelInfo.tierPricingHeader')}

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hardcoded text needs internationalization. Consider using:

Suggested change
Pricing by service tier (price per 1M tokens)
<div className="text-xs text-vscode-descriptionForeground mb-1">
{t('settings:modelInfo.tierPricingHeader')}
</div>

</div>
<div className="border border-vscode-dropdown-border rounded-xs overflow-hidden">
<table className="w-full text-sm">
<thead className="bg-vscode-dropdown-background">
<tr>
<th className="text-left px-3 py-1.5">Tier</th>
<th className="text-right px-3 py-1.5">Input</th>
<th className="text-right px-3 py-1.5">Output</th>
<th className="text-right px-3 py-1.5">Cache reads</th>
</tr>
</thead>
<tbody>
<tr className="border-t border-vscode-dropdown-border/60">
<td className="px-3 py-1.5">Standard</td>
<td className="px-3 py-1.5 text-right">{fmt(modelInfo?.inputPrice)}</td>
<td className="px-3 py-1.5 text-right">{fmt(modelInfo?.outputPrice)}</td>
<td className="px-3 py-1.5 text-right">{fmt(modelInfo?.cacheReadsPrice)}</td>
</tr>
{allowedTiers.includes("flex") && (
<tr className="border-t border-vscode-dropdown-border/60">
<td className="px-3 py-1.5">Flex</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.flex?.inputPrice ?? modelInfo?.inputPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated 6 times across this component. Could we extract this into a helper function? For example:

const getTierPrice = (tier: string, priceType: 'inputPrice' | 'outputPrice' | 'cacheReadsPrice') => {
  return tierPricing?.[tier]?.[priceType] ?? modelInfo?.[priceType]
}

Then use it like: {fmt(getTierPrice('flex', 'inputPrice'))}

</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.flex?.outputPrice ?? modelInfo?.outputPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.flex?.cacheReadsPrice ?? modelInfo?.cacheReadsPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
</td>
</tr>
)}
{allowedTiers.includes("priority") && (
<tr className="border-t border-vscode-dropdown-border/60">
<td className="px-3 py-1.5">Priority</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.priority?.inputPrice ?? modelInfo?.inputPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.priority?.outputPrice ?? modelInfo?.outputPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
</td>
<td className="px-3 py-1.5 text-right">
{fmt(tierPricing?.priority?.cacheReadsPrice ?? modelInfo?.cacheReadsPrice)}
Copy link

Copilot AI Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tier pricing fallback logic is duplicated across multiple rows. Consider extracting this into a helper function that takes the tier name and price type to reduce duplication.

Copilot uses AI. Check for mistakes.
</td>
</tr>
)}
</tbody>
</table>
</div>
</div>
)}
</>
)
}
Expand Down
44 changes: 42 additions & 2 deletions webview-ui/src/components/settings/providers/OpenAI.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,21 @@ import { useCallback, useState } from "react"
import { Checkbox } from "vscrui"
import { VSCodeTextField } from "@vscode/webview-ui-toolkit/react"

import type { ProviderSettings } from "@roo-code/types"
import type { ModelInfo, ProviderSettings } from "@roo-code/types"

import { useAppTranslation } from "@src/i18n/TranslationContext"
import { VSCodeButtonLink } from "@src/components/common/VSCodeButtonLink"
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue, StandardTooltip } from "@src/components/ui"

import { inputEventTransform } from "../transforms"

type OpenAIProps = {
apiConfiguration: ProviderSettings
setApiConfigurationField: (field: keyof ProviderSettings, value: ProviderSettings[keyof ProviderSettings]) => void
selectedModelInfo?: ModelInfo
}

export const OpenAI = ({ apiConfiguration, setApiConfigurationField }: OpenAIProps) => {
export const OpenAI = ({ apiConfiguration, setApiConfigurationField, selectedModelInfo }: OpenAIProps) => {
const { t } = useAppTranslation()

const [openAiNativeBaseUrlSelected, setOpenAiNativeBaseUrlSelected] = useState(
Expand Down Expand Up @@ -72,6 +74,44 @@ export const OpenAI = ({ apiConfiguration, setApiConfigurationField }: OpenAIPro
{t("settings:providers.getOpenAiApiKey")}
</VSCodeButtonLink>
)}

{(() => {
const allowedTiers = (selectedModelInfo?.allowedServiceTiers || []).filter(
(t) => t === "flex" || t === "priority",
)
if (allowedTiers.length === 0) return null

return (
<div className="flex flex-col gap-1 mt-2" data-testid="openai-service-tier">
<div className="flex items-center gap-1">
<label className="block font-medium mb-1">Service tier</label>
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The service tier dropdown label, tooltip content, and option texts ('Service tier', 'Standard', 'Flex', 'Priority') are hard-coded. Please use the translation function (t()) for these UI strings.

Suggested change
<label className="block font-medium mb-1">Service tier</label>
<label className="block font-medium mb-1">{t("settings:providers.serviceTier")}</label>

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These UI strings need internationalization. The label "Service tier" and the dropdown options "Standard", "Flex", "Priority" should use translation keys:

Suggested change
<label className="block font-medium mb-1">Service tier</label>
<label className="block font-medium mb-1">{t("settings:providers.serviceTier")}</label>

<StandardTooltip content="For faster processing of API requests, try the priority processing service tier. For lower prices with higher latency, try the flex processing tier.">
<i className="codicon codicon-info text-vscode-descriptionForeground text-xs" />
</StandardTooltip>
</div>

<Select
value={apiConfiguration.openAiNativeServiceTier || "default"}
Copy link
Contributor

@roomote roomote bot Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical issue: The property openAiNativeServiceTier doesn't exist in ProviderSettings type. This needs to be added to packages/types/src/provider-settings.ts in the openAiNativeSchema.

onValueChange={(value) =>
setApiConfigurationField(
"openAiNativeServiceTier",
value as ProviderSettings["openAiNativeServiceTier"],
)
}>
<SelectTrigger className="w-full">
<SelectValue placeholder={t("settings:common.select")} />
</SelectTrigger>
<SelectContent>
<SelectItem value="default">Standard</SelectItem>
{allowedTiers.includes("flex") && <SelectItem value="flex">Flex</SelectItem>}
{allowedTiers.includes("priority") && (
<SelectItem value="priority">Priority</SelectItem>
)}
</SelectContent>
</Select>
</div>
)
})()}
</>
)
}
Loading