[FEATURE] Better Thinking/Thinking Streaming support

[swombat]

Scope check

• This is core LLM communication (not application logic)
• This benefits most users (not just my use case)
• This can't be solved in application code with current RubyLLM
• I read the Contributing Guide

Due diligence

• I searched existing issues
• I checked the documentation

What problem does this solve?

When using Anthropic's extended thinking feature via RubyLLM:

1. Thinking content is not captured during streaming - The build_chunk method in providers/anthropic/streaming.rb only extracts data.dig('delta', 'text'), ignoring thinking_delta events.

2. Thinking blocks are lost in response parsing - The extract_text_content method in providers/anthropic/chat.rb only extracts blocks where type == 'text', discarding thinking blocks.

3. Conversation history breaks with thinking enabled - When thinking is enabled, Anthropic requires previous assistant messages to include their thinking blocks. Since we don't store/replay thinking content, multi-turn conversations fail with:

   messages.3.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`.
   When `thinking` is enabled, a final `assistant` message must start with a thinking block.
   

4. No unified API for thinking across providers - Each provider has different thinking implementations, but RubyLLM doesn't abstract this.

Proposed solution

1. Add thinking Attribute to Message Class

File: lib/ruby_llm/message.rb

class Message
  attr_reader :role, :model_id, :tool_calls, :tool_call_id, :input_tokens, :output_tokens,
              :cached_tokens, :cache_creation_tokens, :raw, :thinking

  def initialize(options = {})
    # ... existing code ...
    @thinking = options[:thinking]
  end

  def to_h
    {
      # ... existing fields ...
      thinking: thinking
    }.compact
  end
end

2. Parse Thinking in Anthropic Streaming

File: lib/ruby_llm/providers/anthropic/streaming.rb

def build_chunk(data)
  Chunk.new(
    role: :assistant,
    model_id: extract_model_id(data),
    content: extract_content_delta(data),
    thinking: extract_thinking_delta(data),
    # ... other fields ...
  )
end

def extract_content_delta(data)
  return data.dig('delta', 'text') if data.dig('delta', 'type') == 'text_delta'
  nil
end

def extract_thinking_delta(data)
  return data.dig('delta', 'thinking') if data.dig('delta', 'type') == 'thinking_delta'
  nil
end

3. Parse Thinking in Anthropic Response

File: lib/ruby_llm/providers/anthropic/chat.rb

def parse_completion_response(response)
  data = response.body
  content_blocks = data['content'] || []

  text_content = extract_text_content(content_blocks)
  thinking_content = extract_thinking_content(content_blocks)
  tool_use_blocks = Tools.find_tool_uses(content_blocks)

  build_message(data, text_content, thinking_content, tool_use_blocks, response)
end

def extract_thinking_content(blocks)
  thinking_blocks = blocks.select { |c| c['type'] == 'thinking' }
  thinking_blocks.map { |c| c['thinking'] }.join
end

def build_message(data, content, thinking, tool_use_blocks, response)
  Message.new(
    # ... existing fields ...
    thinking: thinking.presence
  )
end

4. Include Thinking in Message Formatting

File: lib/ruby_llm/providers/anthropic/chat.rb

When formatting assistant messages for the API, include thinking blocks:

def format_basic_message(msg)
  content_blocks = []

  # Include thinking block if present (or redacted_thinking placeholder)
  if msg.thinking.present?
    content_blocks << { type: 'thinking', thinking: msg.thinking }
  elsif msg.role == :assistant && thinking_enabled?
    # Placeholder for redacted thinking when original thinking is unavailable
    content_blocks << { type: 'redacted_thinking', data: '' }
  end

  # Add text content
  content_blocks.concat(Media.format_content(msg.content))

  {
    role: convert_role(msg.role),
    content: content_blocks
  }
end

5. Add Gemini Thinking Support

File: lib/ruby_llm/providers/gemini/streaming.rb

Parse thought parts in streaming:

def build_chunk(data)
  parts = data.dig('candidates', 0, 'content', 'parts') || []

  text_parts = parts.reject { |p| p['thought'] }.map { |p| p['text'] }.join
  thought_parts = parts.select { |p| p['thought'] }.map { |p| p['text'] }.join

  Chunk.new(
    content: text_parts.presence,
    thinking: thought_parts.presence,
    # ... other fields ...
  )
end

File: lib/ruby_llm/providers/gemini/chat.rb

Handle thought signatures for multi-turn conversations.

6. Add xAI Grok Reasoning Support

File: lib/ruby_llm/providers/openai/streaming.rb (xAI uses OpenAI-compatible API)

def build_chunk(data)
  Chunk.new(
    content: data.dig('choices', 0, 'delta', 'content'),
    thinking: data.dig('choices', 0, 'delta', 'reasoning_content'),
    # ... other fields ...
  )
end

7. Add Thinking Configuration Helpers

File: lib/ruby_llm/chat.rb (or new file lib/ruby_llm/thinking.rb)

module RubyLLM
  class Chat
    def with_thinking(budget: 10000, effort: nil)
      case detect_provider
      when :anthropic
        with_params(
          thinking: { type: "enabled", budget_tokens: budget },
          max_tokens: budget + 8000
        )
      when :gemini
        with_params(
          thinking_config: { include_thoughts: true, thinking_budget: budget }
        )
      when :openai
        with_params(
          reasoning: { effort: effort || "medium" },
          max_completion_tokens: budget
        )
      when :xai
        with_params(
          reasoning: { enabled: true },
          reasoning_effort: effort || "high"
        )
      end
    end
  end
end

Why this belongs in RubyLLM

This is core functionality (handling thinking from various providers). "Thinking" is a fairly core features of advanced models and many users of RubyLLM, particularly those setting up conversational frameworks, will want to make use of it. Currently it doesn't work reliably across providers, and is undocumented.

In this patch we will both improve the functionality of the thinking (and thinking streaming), and document it (currently there is no mention of the words "think" or "thought" in the documentation).

[swombat]

Working on a PR for this right now, as I need thinking mode for my application so will be switching to that fork whether or not this is accepted as core.

Thanks for the awesome framework!

[swombat]

Current implementation plan:

Executive Summary

Extended Thinking enables Claude, Gemini, and xAI Grok models to expose their internal reasoning process. This implementation adds thinking support with minimal API surface:

1. One method: with_thinking(budget:) - accepts Integer (token count) or Symbol (:low, :medium, :high)
2. Existing streaming: Chunks carry thinking attribute alongside content
3. Always persist: Thinking stored in database when using ActiveRecord
4. Use capabilities: Models declare reasoning capability in models.json
5. Fail fast: Unsupported providers raise UnsupportedFeatureError

The design feels discovered, not designed.

---

1. Public API Design

1.1 Enabling Thinking

# Default thinking (provider determines budget)
chat = RubyLLM.chat
  .with_model("claude-3-7-sonnet-20250219")
  .with_thinking

response = chat.ask("Solve this complex problem...")
puts response.thinking  # => "Let me work through this..."
puts response.content   # => "The answer is 42."

1.2 Configuring Thinking Depth

# Using effort levels (translated by gem)
chat.with_thinking(budget: :low)     # Minimal thinking
chat.with_thinking(budget: :medium)  # Balanced
chat.with_thinking(budget: :high)    # Maximum reasoning

# Using explicit token budget (power users)
chat.with_thinking(budget: 16_000)   # Exact token count

1.3 Streaming with Thinking

# Thinking comes through the same block as content
chat.with_thinking.ask("Analyze this...") do |chunk|
  if chunk.thinking
    render_thinking(chunk.thinking)
  end
  if chunk.content
    render_content(chunk.content)
  end
end

No separate callback. One block receives everything. The developer chooses what to display.

1.4 ActiveRecord Integration

chat = Chat.create!(model_id: "claude-3-7-sonnet-20250219")
chat.with_thinking
response = chat.ask("Complex question...")

# Thinking automatically persisted
message = chat.messages.last
message.thinking  # => "Let me analyze..."

1.5 Method Signature

# In Chat class
def with_thinking(budget: :medium)
  # budget: Integer or Symbol (:low, :medium, :high)
  # Returns: self (fluent interface)
  # Raises: UnsupportedFeatureError if model doesn't support thinking
end

---

2. Internal Architecture Changes

2.1 Message Class Changes

File: lib/ruby_llm/message.rb

class Message
  ROLES = %i[system user assistant tool].freeze

  attr_reader :role, :model_id, :tool_calls, :tool_call_id, :input_tokens, :output_tokens,
              :cached_tokens, :cache_creation_tokens, :raw, :thinking

  def initialize(options = {})
    # ... existing code ...
    @thinking = options[:thinking]
    @thinking_signature = options[:thinking_signature]
  end

  def to_h
    {
      role: role,
      content: content,
      model_id: model_id,
      tool_calls: tool_calls,
      tool_call_id: tool_call_id,
      input_tokens: input_tokens,
      output_tokens: output_tokens,
      cached_tokens: cached_tokens,
      cache_creation_tokens: cache_creation_tokens,
      thinking: thinking
    }.compact
  end

  protected

  attr_reader :thinking_signature
end

Note: thinking_signature is protected. It exists for multi-turn conversation support with Anthropic/Gemini. Providers access it via the signature_for helper method (see section 2.6).

2.2 Chunk Class

File: lib/ruby_llm/chunk.rb

No changes needed. Chunk inherits from Message, automatically gaining thinking attribute.

2.3 Chat Class Changes

File: lib/ruby_llm/chat.rb

class Chat
  include Enumerable

  attr_reader :model, :messages, :tools, :params, :headers, :schema

  def initialize(model: nil, provider: nil, assume_model_exists: false, context: nil)
    # ... existing code ...
    @thinking_budget = nil
  end

  # Enable thinking with unified interface
  def with_thinking(budget: :medium)
    validate_thinking_support!
    @thinking_budget = budget
    self
  end

  # Check if thinking is enabled
  def thinking_enabled?
    !@thinking_budget.nil?
  end

  private

  def validate_thinking_support!
    return if @model.supports?('reasoning')

    raise UnsupportedFeatureError,
      "Model '#{@model.id}' does not support extended thinking"
  end
end

Key simplifications:

• Single budget: parameter
• No without_thinking method
• No on_thinking callback
• Validation uses existing capabilities system
• No provider-specific logic

2.4 StreamAccumulator Changes

File: lib/ruby_llm/stream_accumulator.rb

class StreamAccumulator
  attr_reader :content, :model_id, :tool_calls, :thinking

  def initialize
    @content = +''
    @thinking = +''
    @thinking_signature = nil
    @tool_calls = {}
    # ... existing code ...
  end

  def add(chunk)
    RubyLLM.logger.debug chunk.inspect if RubyLLM.config.log_stream_debug
    @model_id ||= chunk.model_id

    if chunk.tool_call?
      accumulate_tool_calls chunk.tool_calls
    else
      @content << (chunk.content || '')
      @thinking << (chunk.thinking || '')
    end

    @thinking_signature = Messages.signature_for(chunk) || @thinking_signature

    count_tokens chunk
    RubyLLM.logger.debug inspect if RubyLLM.config.log_stream_debug
  end

  def to_message(response)
    Message.new(
      role: :assistant,
      content: content.empty? ? nil : content,
      thinking: thinking.empty? ? nil : thinking,
      thinking_signature: @thinking_signature,
      model_id: model_id,
      tool_calls: tool_calls_from_stream,
      input_tokens: @input_tokens,
      output_tokens: @output_tokens,
      cached_tokens: @cached_tokens,
      cache_creation_tokens: @cache_creation_tokens,
      raw: response
    )
  end
end

2.5 Provider Base Class Changes

File: lib/ruby_llm/provider.rb

class Provider
  def complete(messages, tools:, temperature:, model:, params: {}, headers: {},
               schema: nil, thinking: nil, &)
    normalized_temperature = maybe_normalize_temperature(temperature, model)

    payload = Utils.deep_merge(
      render_payload(
        messages,
        tools: tools,
        temperature: normalized_temperature,
        model: model,
        stream: block_given?,
        schema: schema,
        thinking: thinking
      ),
      params
    )

    if block_given?
      stream_response @connection, payload, headers, &
    else
      sync_response @connection, payload, headers
    end
  end
end

The thinking parameter is passed through. Providers that support it handle it; others ignore it. No validation at this layer.

2.6 Messages Helper Module

File: lib/ruby_llm/messages.rb

Add a helper for accessing thinking signature without the send dance:

module RubyLLM
  module Messages
    module_function

    def signature_for(message)
      message.thinking_signature if message.respond_to?(:thinking_signature, true)
    end
  end
end

This provides a clean internal API for providers to access the protected signature without verbose send(:thinking_signature) calls.

---

3. Provider-Specific Implementations

3.1 Anthropic Provider

File: lib/ruby_llm/providers/anthropic/chat.rb

module Chat
  module_function

  def render_payload(messages, tools:, temperature:, model:, stream: false,
                     schema: nil, thinking: nil)
    system_messages, chat_messages = separate_messages(messages)
    system_content = build_system_content(system_messages)

    build_base_payload(chat_messages, model, stream, thinking).tap do |payload|
      add_optional_fields(payload, system_content:, tools:, temperature:)
    end
  end

  def build_base_payload(chat_messages, model, stream, thinking)
    payload = {
      model: model.id,
      messages: chat_messages.map { |msg| format_message(msg, thinking_enabled: thinking) },
      stream: stream,
      max_tokens: calculate_max_tokens(model, thinking)
    }

    if thinking
      payload[:thinking] = {
        type: 'enabled',
        budget_tokens: resolve_budget(thinking, model)
      }
    end

    payload
  end

  def calculate_max_tokens(model, thinking)
    base = model.max_tokens || 4096
    return base unless thinking

    budget = resolve_budget(thinking, model)
    [base, budget + 8000].max
  end

  def resolve_budget(thinking, _model)
    case thinking
    when Integer then thinking
    when :low then 1024
    when :medium then 10_000
    when :high then 32_000
    else 10_000
    end
  end

  def parse_completion_response(response)
    data = response.body
    content_blocks = data['content'] || []

    text_content = extract_text_content(content_blocks)
    thinking_content = extract_thinking_content(content_blocks)
    thinking_sig = extract_thinking_signature(content_blocks)
    tool_use_blocks = Tools.find_tool_uses(content_blocks)

    build_message(data, text_content, thinking_content, thinking_sig,
                  tool_use_blocks, response)
  end

  def extract_thinking_content(blocks)
    thinking_blocks = blocks.select { |c| c['type'] == 'thinking' }
    thoughts = thinking_blocks.map { |c| c['thinking'] }.join
    thoughts.empty? ? nil : thoughts
  end

  def extract_thinking_signature(blocks)
    thinking_block = blocks.find { |c| c['type'] == 'thinking' }
    thinking_block&.dig('signature')
  end

  def build_message(data, content, thinking, thinking_sig, tool_use_blocks, response)
    usage = data['usage'] || {}
    cached_tokens = usage['cache_read_input_tokens']
    cache_creation_tokens = usage['cache_creation_input_tokens']
    if cache_creation_tokens.nil? && usage['cache_creation'].is_a?(Hash)
      cache_creation_tokens = usage['cache_creation'].values.compact.sum
    end

    Message.new(
      role: :assistant,
      content: content,
      thinking: thinking,
      thinking_signature: thinking_sig,
      tool_calls: Tools.parse_tool_calls(tool_use_blocks),
      input_tokens: usage['input_tokens'],
      output_tokens: usage['output_tokens'],
      cached_tokens: cached_tokens,
      cache_creation_tokens: cache_creation_tokens,
      model_id: data['model'],
      raw: response
    )
  end

  def format_message(msg, thinking_enabled: false)
    if msg.tool_call?
      format_tool_call_with_thinking(msg, thinking_enabled)
    elsif msg.tool_result?
      Tools.format_tool_result(msg)
    else
      format_basic_message_with_thinking(msg, thinking_enabled)
    end
  end

  def format_basic_message_with_thinking(msg, thinking_enabled)
    content_blocks = []

    if msg.role == :assistant && thinking_enabled
      sig = Messages.signature_for(msg)

      if msg.thinking.present?
        content_blocks << {
          type: 'thinking',
          thinking: msg.thinking,
          signature: sig
        }.compact
      elsif sig
        content_blocks << {
          type: 'redacted_thinking',
          data: sig
        }
      end
    end

    content_blocks.concat(Media.format_content(msg.content))

    {
      role: convert_role(msg.role),
      content: content_blocks
    }
  end

  def format_tool_call_with_thinking(msg, thinking_enabled)
    content_blocks = []

    if thinking_enabled && msg.thinking.present?
      sig = Messages.signature_for(msg)
      content_blocks << {
        type: 'thinking',
        thinking: msg.thinking,
        signature: sig
      }.compact
    end

    msg.tool_calls.each_value do |tool_call|
      content_blocks << {
        type: 'tool_use',
        id: tool_call.id,
        name: tool_call.name,
        input: tool_call.arguments
      }
    end

    {
      role: 'assistant',
      content: content_blocks
    }
  end
end

File: lib/ruby_llm/providers/anthropic/streaming.rb

module Streaming
  private

  def build_chunk(data)
    delta_type = data.dig('delta', 'type')

    Chunk.new(
      role: :assistant,
      model_id: extract_model_id(data),
      content: extract_content_delta(data, delta_type),
      thinking: extract_thinking_delta(data, delta_type),
      thinking_signature: extract_signature_delta(data, delta_type),
      input_tokens: extract_input_tokens(data),
      output_tokens: extract_output_tokens(data),
      cached_tokens: extract_cached_tokens(data),
      cache_creation_tokens: extract_cache_creation_tokens(data),
      tool_calls: extract_tool_calls(data)
    )
  end

  def extract_content_delta(data, delta_type)
    return data.dig('delta', 'text') if delta_type == 'text_delta'
    nil
  end

  def extract_thinking_delta(data, delta_type)
    return data.dig('delta', 'thinking') if delta_type == 'thinking_delta'
    nil
  end

  def extract_signature_delta(data, delta_type)
    return data.dig('delta', 'signature') if delta_type == 'signature_delta'
    nil
  end
end

3.2 Gemini Provider

File: lib/ruby_llm/providers/gemini/chat.rb

module Chat
  def render_payload(messages, tools:, temperature:, model:, stream: false,
                     schema: nil, thinking: nil)
    @model = model.id
    payload = {
      contents: format_messages(messages),
      generationConfig: {}
    }

    payload[:generationConfig][:temperature] = temperature unless temperature.nil?
    payload[:generationConfig].merge!(structured_output_config(schema, model)) if schema

    if thinking
      payload[:generationConfig][:thinkingConfig] = build_thinking_config(model, thinking)
    end

    payload[:tools] = format_tools(tools) if tools.any?
    payload
  end

  def build_thinking_config(model, thinking)
    config = { includeThoughts: true }

    if gemini_3_model?(model)
      config[:thinkingLevel] = resolve_effort_level(thinking)
    else
      config[:thinkingBudget] = resolve_budget(thinking)
    end

    config
  end

  # Gemini 3 uses thinkingLevel (string) while Gemini 2.5 uses thinkingBudget (integer).
  # String matching is unavoidable until Google provides a capability flag or stabilizes
  # their API versioning. This detection is confined to the Gemini provider where the
  # coupling is appropriate.
  def gemini_3_model?(model)
    model.id.to_s.include?('gemini-3')
  end

  def resolve_effort_level(thinking)
    case thinking
    when :low then 'low'
    when :medium then 'medium'
    when :high then 'high'
    when Integer then thinking > 16_000 ? 'high' : 'low'
    else 'high'
    end
  end

  def resolve_budget(thinking)
    case thinking
    when Integer then thinking
    when :low then 1024
    when :medium then 8192
    when :high then 24_576
    else 8192
    end
  end

  def parse_completion_response(response)
    data = response.body
    parts = data.dig('candidates', 0, 'content', 'parts') || []

    text_content = extract_text_parts(parts)
    thinking_content = extract_thought_parts(parts)
    tool_calls = extract_tool_calls(data)

    Message.new(
      role: :assistant,
      content: text_content,
      thinking: thinking_content,
      tool_calls: tool_calls,
      input_tokens: data.dig('usageMetadata', 'promptTokenCount'),
      output_tokens: calculate_output_tokens(data),
      model_id: data['modelVersion'] || response.env.url.path.split('/')[3].split(':')[0],
      raw: response
    )
  end

  def extract_text_parts(parts)
    text_parts = parts.reject { |p| p['thought'] }
    content = text_parts.map { |p| p['text'] }.compact.join
    content.empty? ? nil : content
  end

  def extract_thought_parts(parts)
    thought_parts = parts.select { |p| p['thought'] }
    thoughts = thought_parts.map { |p| p['text'] }.compact.join
    thoughts.empty? ? nil : thoughts
  end
end

File: lib/ruby_llm/providers/gemini/streaming.rb

module Streaming
  def build_chunk(data)
    parts = data.dig('candidates', 0, 'content', 'parts') || []

    Chunk.new(
      role: :assistant,
      model_id: extract_model_id(data),
      content: extract_text_content(parts),
      thinking: extract_thought_content(parts),
      input_tokens: extract_input_tokens(data),
      output_tokens: extract_output_tokens(data),
      tool_calls: extract_tool_calls(data)
    )
  end

  def extract_text_content(parts)
    text_parts = parts.reject { |p| p['thought'] }
    text = text_parts.map { |p| p['text'] }.compact.join
    text.empty? ? nil : text
  end

  def extract_thought_content(parts)
    thought_parts = parts.select { |p| p['thought'] }
    thoughts = thought_parts.map { |p| p['text'] }.compact.join
    thoughts.empty? ? nil : thoughts
  end
end

3.3 OpenAI Provider (for xAI Grok)

File: lib/ruby_llm/providers/openai/chat.rb

module Chat
  def render_payload(messages, tools:, temperature:, model:, stream: false,
                     schema: nil, thinking: nil)
    payload = {
      model: model.id,
      messages: format_messages(messages),
      stream: stream
    }

    payload[:temperature] = temperature unless temperature.nil?
    payload[:tools] = tools.map { |_, tool| tool_for(tool) } if tools.any?

    # xAI Grok reasoning
    if thinking && grok_model?(model)
      payload[:reasoning_effort] = resolve_effort(thinking)
    end

    payload[:stream_options] = { include_usage: true } if stream
    payload
  end

  def grok_model?(model)
    model.id.to_s.include?('grok')
  end

  def resolve_effort(thinking)
    case thinking
    when :low then 'low'
    when :high then 'high'
    when Integer then thinking > 10_000 ? 'high' : 'low'
    else 'high'
    end
  end

  def parse_completion_response(response)
    data = response.body
    return if data.empty?

    raise Error.new(response, data.dig('error', 'message')) if data.dig('error', 'message')

    message_data = data.dig('choices', 0, 'message')
    return unless message_data

    usage = data['usage'] || {}
    cached_tokens = usage.dig('prompt_tokens_details', 'cached_tokens')

    Message.new(
      role: :assistant,
      content: message_data['content'],
      thinking: message_data['reasoning_content'],
      tool_calls: parse_tool_calls(message_data['tool_calls']),
      input_tokens: usage['prompt_tokens'],
      output_tokens: usage['completion_tokens'],
      cached_tokens: cached_tokens,
      cache_creation_tokens: 0,
      model_id: data['model'],
      raw: response
    )
  end
end

File: lib/ruby_llm/providers/openai/streaming.rb

module Streaming
  def build_chunk(data)
    usage = data['usage'] || {}
    cached_tokens = usage.dig('prompt_tokens_details', 'cached_tokens')
    delta = data.dig('choices', 0, 'delta') || {}

    Chunk.new(
      role: :assistant,
      model_id: data['model'],
      content: delta['content'],
      thinking: delta['reasoning_content'],
      tool_calls: parse_tool_calls(delta['tool_calls'], parse_arguments: false),
      input_tokens: usage['prompt_tokens'],
      output_tokens: usage['completion_tokens'],
      cached_tokens: cached_tokens,
      cache_creation_tokens: 0
    )
  end
end

---

4. ActiveRecord Integration

4.1 Migration

Users add two columns to their messages table. Document in CHANGELOG:

# db/migrate/XXXXXXX_add_thinking_to_messages.rb
class AddThinkingToMessages < ActiveRecord::Migration[7.0]
  def change
    add_column :messages, :thinking, :text
    add_column :messages, :thinking_signature, :text
  end
end

No generator needed. Rails developers know how to create migrations.

4.2 Updated Install Generator Template

File: lib/generators/ruby_llm/install/templates/create_messages_migration.rb.tt

class Create<%= message_model_name.gsub('::', '').pluralize %> < ActiveRecord::Migration<%= migration_version %>
  def change
    create_table :<%= message_table_name %> do |t|
      t.string :role, null: false
      t.text :content
      t.json :content_raw
      t.text :thinking
      t.text :thinking_signature
      t.integer :input_tokens
      t.integer :output_tokens
      t.integer :cached_tokens
      t.integer :cache_creation_tokens
      t.timestamps
    end

    add_index :<%= message_table_name %>, :role
  end
end

4.3 MessageMethods Changes

File: lib/ruby_llm/active_record/message_methods.rb

module MessageMethods
  def to_llm
    RubyLLM::Message.new(
      role: role.to_sym,
      content: extract_content,
      thinking: thinking_value,
      thinking_signature: thinking_signature_value,
      tool_calls: extract_tool_calls,
      tool_call_id: extract_tool_call_id,
      input_tokens: input_tokens,
      output_tokens: output_tokens,
      cached_tokens: cached_value,
      cache_creation_tokens: cache_creation_value,
      model_id: model_association&.model_id
    )
  end

  private

  def thinking_value
    has_attribute?(:thinking) ? self[:thinking] : nil
  end

  def thinking_signature_value
    has_attribute?(:thinking_signature) ? self[:thinking_signature] : nil
  end

  def cached_value
    has_attribute?(:cached_tokens) ? self[:cached_tokens] : nil
  end

  def cache_creation_value
    has_attribute?(:cache_creation_tokens) ? self[:cache_creation_tokens] : nil
  end
end

4.4 ChatMethods Changes

File: lib/ruby_llm/active_record/chat_methods.rb

module ChatMethods
  def with_thinking(...)
    to_llm.with_thinking(...)
    self
  end

  private

  def persist_message_completion(message)
    return unless message

    transaction do
      attrs = base_message_attributes(message)
      add_thinking_attributes(attrs, message)
      add_token_attributes(attrs, message)

      # ... existing persistence logic ...
    end
  end

  def add_thinking_attributes(attrs, message)
    if message_class.column_names.include?('thinking')
      attrs[:thinking] = message.thinking
    end
    if message_class.column_names.include?('thinking_signature')
      attrs[:thinking_signature] = Messages.signature_for(message)
    end
  end
end

---

5. Model Capabilities

5.1 Add reasoning Capability to models.json

Models that support thinking get the reasoning capability. This is the single source of truth.

{
  "id": "claude-3-7-sonnet-20250219",
  "capabilities": ["streaming", "function_calling", "vision", "reasoning"],
  ...
}

5.2 Model Info Helper

Already exists: model.reasoning? checks for the capability.

# In Model::Info
def reasoning?
  supports?('reasoning')
end

---

6. Error Handling

6.1 New Error Class

File: lib/ruby_llm/errors.rb

module RubyLLM
  class UnsupportedFeatureError < Error
    def initialize(message)
      super(nil, message)
    end
  end
end

6.2 Validation in Chat

def validate_thinking_support!
  return if @model.supports?('reasoning')

  raise UnsupportedFeatureError,
    "Model '#{@model.id}' does not support extended thinking"
end

No provider-specific validation. If Anthropic requires budget_tokens >= 1024, let Anthropic return that error. The gem should not know or care about provider constraints.

---

[NielsKSchjoedt]

What about support for vertexai provider...?

[short-ventures]

This is great work, the gemini thought signatures can be particularly tricky in my experience.

[crmne]

Fixed in ff4037a