Skip to content

Flex working1 #7045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Flex working1 #7045

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5338922
Flex tier
mechanicmuthu Aug 12, 2025
b0c1eed
basic changes not specific to flex tier
mechanicmuthu Aug 12, 2025
5e4c32e
flex with heavy console logs
mechanicmuthu Aug 12, 2025
7c16a04
Merge branch 'main' into flex-working1
mechanicmuthu Aug 13, 2025
f546851
working after merge aug13
mechanicmuthu Aug 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions apps/web-roo-code/src/components/homepage/features.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ export function Features() {
y: 0,
transition: {
duration: 0.6,
ease: [0.21, 0.45, 0.27, 0.9],
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These animation easing changes seem unrelated to the flex pricing feature. Could these be moved to a separate PR to keep this one focused on a single feature?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are in fact unrelated. These were causing compile errors for me. These cane be removed or ignored.

},
}
Expand All @@ -104,7 +103,7 @@ export function Features() {
opacity: 1,
transition: {
duration: 1.2,
ease: "easeOut",
// removed ease due to type constraints in motion-dom Easing typing
},
},
}
Expand Down
2 changes: 1 addition & 1 deletion apps/web-roo-code/src/components/homepage/install-section.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export function InstallSection({ downloads }: InstallSectionProps) {
opacity: 1,
transition: {
duration: 1.2,
ease: "easeOut",
// removed ease due to type constraints in motion-dom Easing typing
},
},
}
Expand Down
3 changes: 1 addition & 2 deletions apps/web-roo-code/src/components/homepage/testimonials.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ export function Testimonials() {
y: 0,
transition: {
duration: 0.6,
ease: [0.21, 0.45, 0.27, 0.9],
},
},
}
Expand All @@ -82,7 +81,7 @@ export function Testimonials() {
opacity: 1,
transition: {
duration: 1.2,
ease: "easeOut",
// removed ease due to type constraints in motion-dom Easing typing
},
},
}
Expand Down
9 changes: 9 additions & 0 deletions packages/types/src/model.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,15 @@ export const modelInfoSchema = z.object({
outputPrice: z.number().optional(),
cacheWritesPrice: z.number().optional(),
cacheReadsPrice: z.number().optional(),
// Optional discounted pricing for flex service tier
flexPrice: z
.object({
inputPrice: z.number().optional(),
outputPrice: z.number().optional(),
cacheWritesPrice: z.number().optional(),
cacheReadsPrice: z.number().optional(),
})
.optional(),
description: z.string().optional(),
reasoningEffort: reasoningEffortsSchema.optional(),
minTokensPerCachePoint: z.number().optional(),
Expand Down
3 changes: 3 additions & 0 deletions packages/types/src/provider-settings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,9 @@ const baseProviderSettingsSchema = z.object({

// Model verbosity.
verbosity: verbosityLevelsSchema.optional(),

// Service tier selection for providers that support tiered pricing (e.g. OpenAI flex tier)
serviceTier: z.enum(["auto", "default", "flex"]).optional(),
})

// Several of the providers share common model config properties.
Expand Down
25 changes: 25 additions & 0 deletions packages/types/src/providers/openai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ export const openAiNativeModels = {
inputPrice: 1.25,
outputPrice: 10.0,
cacheReadsPrice: 0.13,
flexPrice: {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flexPrice object is missing cacheWritesPrice. Is this intentional, or should it include all pricing fields for consistency?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenAI does not charged extra for CacheWrite. Check with the pricing of other OpenAI api model pricing, if cache write price is included, then inputPrice can be considered as cachWritesPrice, otherwise if these are not maintained for other openai models, then it can be safely ignored here.

inputPrice: 0.625,
outputPrice: 5.0,
cacheReadsPrice: 0.063,
},
description: "GPT-5: The best model for coding and agentic tasks across domains",
// supportsVerbosity is a new capability; ensure ModelInfo includes it
supportsVerbosity: true,
Expand All @@ -30,6 +35,11 @@ export const openAiNativeModels = {
inputPrice: 0.25,
outputPrice: 2.0,
cacheReadsPrice: 0.03,
flexPrice: {
inputPrice: 0.125,
outputPrice: 1.0,
cacheReadsPrice: 0.013,
},
description: "GPT-5 Mini: A faster, more cost-efficient version of GPT-5 for well-defined tasks",
supportsVerbosity: true,
},
Expand All @@ -43,6 +53,11 @@ export const openAiNativeModels = {
inputPrice: 0.05,
outputPrice: 0.4,
cacheReadsPrice: 0.01,
flexPrice: {
inputPrice: 0.025,
outputPrice: 0.2,
cacheReadsPrice: 0.003,
},
description: "GPT-5 Nano: Fastest, most cost-efficient version of GPT-5",
supportsVerbosity: true,
},
Expand Down Expand Up @@ -81,6 +96,11 @@ export const openAiNativeModels = {
inputPrice: 2.0,
outputPrice: 8.0,
cacheReadsPrice: 0.5,
flexPrice: {
inputPrice: 1.0,
outputPrice: 4.0,
cacheReadsPrice: 0.25,
},
supportsReasoningEffort: true,
reasoningEffort: "medium",
},
Expand Down Expand Up @@ -112,6 +132,11 @@ export const openAiNativeModels = {
inputPrice: 1.1,
outputPrice: 4.4,
cacheReadsPrice: 0.275,
flexPrice: {
inputPrice: 0.55,
outputPrice: 2.2,
cacheReadsPrice: 0.138,
},
supportsReasoningEffort: true,
reasoningEffort: "medium",
},
Expand Down
119 changes: 110 additions & 9 deletions src/api/providers/openai-native.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,11 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
this.options.enableGpt5ReasoningSummary = true
}
const apiKey = this.options.openAiNativeApiKey ?? "not-provided"
this.client = new OpenAI({ baseURL: this.options.openAiNativeBaseUrl, apiKey })
this.client = new OpenAI({
baseURL: this.options.openAiNativeBaseUrl,
apiKey,
timeout: 15 * 1000 * 60, // 15 minutes default timeout
})
}

private normalizeGpt5Usage(usage: any, model: OpenAiNativeModel): ApiStreamUsageChunk | undefined {
Expand All @@ -74,6 +78,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
totalOutputTokens,
cacheWriteTokens || 0,
cacheReadTokens || 0,
this.options.serviceTier,
)

return {
Expand Down Expand Up @@ -135,7 +140,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
const isOriginalO1 = model.id === "o1"
const { reasoning } = this.getModel()

const response = await this.client.chat.completions.create({
const params: any = {
model: model.id,
messages: [
{
Expand All @@ -147,9 +152,30 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
stream: true,
stream_options: { include_usage: true },
...(reasoning && reasoning),
})
}

// Add service_tier parameter if configured and not "auto"
if (this.options.serviceTier && this.options.serviceTier !== "auto") {
params.service_tier = this.options.serviceTier
console.log("[DEBUG] Setting service_tier parameter:", this.options.serviceTier)
console.log("[DEBUG] Full request params:", JSON.stringify(params, null, 2))
} else {
console.log("[DEBUG] Service tier not set or is 'auto'. Current value:", this.options.serviceTier)
}

const response = await this.client.chat.completions.create(params, { timeout: 15 * 1000 * 60 })
console.log("[DEBUG] OpenAI Chat Completions Response (O1Family):", response)

yield* this.handleStreamResponse(response, model)
if (typeof (response as any)[Symbol.asyncIterator] !== "function") {
throw new Error(
"OpenAI SDK did not return an AsyncIterable for streaming response. Please check SDK version and usage.",
)
}

yield* this.handleStreamResponse(
response as unknown as AsyncIterable<OpenAI.Chat.Completions.ChatCompletionChunk>,
model,
)
}

private async *handleReasonerMessage(
Expand All @@ -160,7 +186,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
): ApiStream {
const { reasoning } = this.getModel()

const stream = await this.client.chat.completions.create({
const params: any = {
model: family,
messages: [
{
Expand All @@ -172,9 +198,29 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
stream: true,
stream_options: { include_usage: true },
...(reasoning && reasoning),
})
}

// Add service_tier parameter if configured and not "auto"
if (this.options.serviceTier && this.options.serviceTier !== "auto") {
params.service_tier = this.options.serviceTier
console.log("[DEBUG] Setting service_tier parameter:", this.options.serviceTier)
console.log("[DEBUG] Full request params:", JSON.stringify(params, null, 2))
} else {
console.log("[DEBUG] Service tier not set or is 'auto'. Current value:", this.options.serviceTier)
}

const stream = await this.client.chat.completions.create(params, { timeout: 15 * 1000 * 60 })

yield* this.handleStreamResponse(stream, model)
if (typeof (stream as any)[Symbol.asyncIterator] !== "function") {
throw new Error(
"OpenAI SDK did not return an AsyncIterable for streaming response. Please check SDK version and usage.",
)
}

yield* this.handleStreamResponse(
stream as unknown as AsyncIterable<OpenAI.Chat.Completions.ChatCompletionChunk>,
model,
)
}

private async *handleDefaultModelMessage(
Expand All @@ -199,7 +245,16 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
params.verbosity = verbosity
}

const stream = await this.client.chat.completions.create(params)
// Add service_tier parameter if configured and not "auto"
if (this.options.serviceTier && this.options.serviceTier !== "auto") {
params.service_tier = this.options.serviceTier
console.log("[DEBUG] Setting service_tier parameter:", this.options.serviceTier)
console.log("[DEBUG] Full request params:", JSON.stringify(params, null, 2))
} else {
console.log("[DEBUG] Service tier not set or is 'auto'. Current value:", this.options.serviceTier)
}

const stream = await this.client.chat.completions.create(params, { timeout: 15 * 1000 * 60 })

if (typeof (stream as any)[Symbol.asyncIterator] !== "function") {
throw new Error(
Expand Down Expand Up @@ -276,6 +331,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
temperature?: number
max_output_tokens?: number
previous_response_id?: string
service_tier?: string
}

const requestBody: Gpt5RequestBody = {
Expand All @@ -296,9 +352,20 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
...(requestPreviousResponseId && { previous_response_id: requestPreviousResponseId }),
}

// Add service_tier parameter if configured and not "auto"
if (this.options.serviceTier && this.options.serviceTier !== "auto") {
requestBody.service_tier = this.options.serviceTier
console.log("[DEBUG] Setting service_tier parameter:", this.options.serviceTier)
console.log("[DEBUG] Full request body:", JSON.stringify(requestBody, null, 2))
} else {
console.log("[DEBUG] Service tier not set or is 'auto'. Current value:", this.options.serviceTier)
}

try {
// Use the official SDK
const stream = (await (this.client as any).responses.create(requestBody)) as AsyncIterable<any>
const stream = (await (this.client as any).responses.create(requestBody, {
timeout: 15 * 1000 * 60,
})) as AsyncIterable<any>

if (typeof (stream as any)[Symbol.asyncIterator] !== "function") {
throw new Error(
Expand All @@ -307,11 +374,13 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
}

for await (const event of stream) {
console.log("[DEBUG] GPT-5 Responses API Stream Event:", event) // Log each event
for await (const outChunk of this.processGpt5Event(event, model)) {
yield outChunk
}
}
} catch (sdkErr: any) {
console.error("[DEBUG] OpenAI Responses API SDK Error:", sdkErr) // Log SDK errors
// Check if this is a 400 error about previous_response_id not found
const errorMessage = sdkErr?.message || sdkErr?.error?.message || ""
const is400Error = sdkErr?.status === 400 || sdkErr?.response?.status === 400
Expand Down Expand Up @@ -418,6 +487,15 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
const baseUrl = this.options.openAiNativeBaseUrl || "https://api.openai.com"
const url = `${baseUrl}/v1/responses`

// Log the exact request being sent
console.log("[DEBUG] GPT-5 Responses API Request URL:", url)
console.log("[DEBUG] GPT-5 Responses API Request Headers:", {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey.substring(0, 8)}...`, // Log only first 8 chars of API key for security
Accept: "text/event-stream",
})
console.log("[DEBUG] GPT-5 Responses API Request Body:", JSON.stringify(requestBody, null, 2))

try {
const response = await fetch(url, {
method: "POST",
Expand All @@ -429,8 +507,18 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
body: JSON.stringify(requestBody),
})

// Log the response status
console.log("[DEBUG] GPT-5 Responses API Response Status:", response.status)
// Convert headers to a plain object for logging
const headersObj: Record<string, string> = {}
response.headers.forEach((value, key) => {
headersObj[key] = value
})
console.log("[DEBUG] GPT-5 Responses API Response Headers:", headersObj)

if (!response.ok) {
const errorText = await response.text()
console.log("[DEBUG] GPT-5 Responses API Error Response Body:", errorText)

let errorMessage = `GPT-5 API request failed (${response.status})`
let errorDetails = ""
Expand Down Expand Up @@ -470,6 +558,11 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
this.resolveResponseId(undefined)

// Retry the request without the previous_response_id
console.log(
"[DEBUG] GPT-5 Responses API Retry Request Body:",
JSON.stringify(retryRequestBody, null, 2),
)

const retryResponse = await fetch(url, {
method: "POST",
headers: {
Expand All @@ -480,7 +573,11 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
body: JSON.stringify(retryRequestBody),
})

console.log("[DEBUG] GPT-5 Responses API Retry Response Status:", retryResponse.status)

if (!retryResponse.ok) {
const retryErrorText = await retryResponse.text()
console.log("[DEBUG] GPT-5 Responses API Retry Error Response Body:", retryErrorText)
// If retry also fails, throw the original error
throw new Error(`GPT-5 API retry failed (${retryResponse.status})`)
}
Expand Down Expand Up @@ -536,6 +633,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
// Handle streaming response
yield* this.handleGpt5StreamResponse(response.body, model)
} catch (error) {
console.error("[DEBUG] GPT-5 Responses API Fetch Error:", error) // Log fetch errors
if (error instanceof Error) {
// Re-throw with the original error message if it's already formatted
if (error.message.includes("GPT-5")) {
Expand Down Expand Up @@ -1039,6 +1137,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
* Used by both the official SDK streaming path and (optionally) by the SSE fallback.
*/
private async *processGpt5Event(event: any, model: OpenAiNativeModel): ApiStream {
console.log("[DEBUG] processGpt5Event: Processing event type:", event?.type)
// Persist response id for conversation continuity when available
if (event?.response?.id) {
this.resolveResponseId(event.response.id)
Expand Down Expand Up @@ -1147,6 +1246,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
model: OpenAiNativeModel,
): ApiStream {
for await (const chunk of stream) {
console.log("[DEBUG] handleStreamResponse: OpenAI Chat Completions Stream Chunk:", chunk) // Log each chunk here
const delta = chunk.choices[0]?.delta

if (delta?.content) {
Expand Down Expand Up @@ -1180,6 +1280,7 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
outputTokens,
cacheWriteTokens || 0,
cacheReadTokens || 0,
this.options.serviceTier,
)

yield {
Expand Down
Loading
Loading