fix: token limits & Temperature Parameter Errors for Reasoning Models

[Stijnus]

Fix Token Limits & Temperature Parameter Errors for Reasoning Models

🐛 Problem

Reasoning models (GPT-5, o1, o3 series) were failing due to two critical parameter compatibility issues:

1. Token Limit Error — max_tokens is too large: 100000. This model supports at most 16384 completion tokens.
2. Temperature Parameter Error — Unsupported value: temperature does not support 0 with reasoning models. Only the default (1) value is supported.

🔍 Root Cause

OpenAI's reasoning models have different API parameter requirements than traditional models:

• Token Parameters: Require maxCompletionTokens instead of maxTokens
• Temperature: Only support temperature: 1 (no custom values)
• Unsupported Parameters: Many traditional parameters (topP, presencePenalty, etc.) are not supported

✅ Solution

1. Model Detection System

export function isReasoningModel(modelName: string): boolean {
  return /^(o1|o3|gpt-5)/i.test(modelName);
}

2. Provider-Specific Token Limits

export const PROVIDER_COMPLETION_LIMITS: Record<string, number> = {
  OpenAI: 16384,
  Anthropic: 128000,
  Google: 32768,
  // ... other providers
};

3. Parameter Compatibility Logic
Token Parameters:

const tokenParams = isReasoning
  ? { maxCompletionTokens: safeMaxTokens }
  : { maxTokens: safeMaxTokens };

Temperature Parameters:

const finalParams = isReasoning
  ? { ...baseParams, temperature: 1 }   // Required for reasoning models
  : { ...baseParams, temperature: 0 };  // Deterministic for traditional models

Parameter Filtering (Streaming):

const filteredOptions = isReasoning && options
  ? Object.fromEntries(
      Object.entries(options).filter(
        ([key]) => ![
          'temperature', 'topP', 'presencePenalty',
          'frequencyPenalty', 'logprobs', 'topLogprobs', 'logitBias'
        ].includes(key)
      )
    )
  : options || {};

📁 Files Changed

• app/lib/.server/llm/constants.ts → Added reasoning model detection & provider limits
• app/lib/.server/llm/stream-text.ts → Fixed streaming parameter compatibility
• app/routes/api.llmcall.ts → Fixed direct API call parameters
• app/lib/modules/llm/types.ts → Added maxCompletionTokens field
• app/lib/modules/llm/providers/* → Updated model definitions with completion token limits

🧪 Testing

Verified with multiple reasoning models:

• ✅ gpt-5 → Working with correct parameters
• ✅ gpt-5-chat-latest → Successful completion
• ✅ o1-preview → Parameter compatibility confirmed
• ✅ Traditional models (GPT-4, Claude) → Unchanged behavior maintained

🎯 Impact

• Before: Reasoning models were completely unusable due to parameter errors
• After: Full compatibility with OpenAI reasoning models, while preserving traditional model functionality
• Backward Compatible: No breaking changes for existing traditional model usage

🔧 Debug Logging

Added debug logging for troubleshooting:

logger.info(`DEBUG: Model "${modelName}" detected as reasoning model: ${isReasoning}`);
logger.info(`DEBUG: Final params:`, { hasTemperature, hasMaxTokens, hasMaxCompletionTokens });

✅ This fix ensures seamless integration with OpenAI's latest reasoning models while maintaining backward compatibility with all existing model providers.