How to use Gemini Search Grounding with Structured Output?

[swistaczek]

I'm trying to use Gemini's built-in Google Search grounding feature with ruby_llm. I found that passing tools: [{ google_search: {} }] via with_params works:

chat = RubyLLM.chat(model: 'gemini-2.0-flash', provider: :gemini)
  .with_params(tools: [{ google_search: {} }])
  .ask('What is the current stock price of AAPL?')
# => "As of today, December 30, 2025, the current stock price of Apple (AAPL) is $273.76..."

This works great for free-form responses. However, when I try to combine it with with_schema for structured output, the search grounding doesn't trigger:

schema = {
  type: 'object',
  properties: {
    price: { type: 'string' },
    currency: { type: 'string' }
  },
  required: ['price', 'currency'],
  additionalProperties: false
}

chat = RubyLLM.chat(model: 'gemini-2.0-flash', provider: :gemini)
  .with_params(tools: [{ google_search: {} }])
  .with_schema(schema)
  .ask('What is the current Apple stock price?')
# => {"price" => "N/A", "currency" => "USD"}  # Search not triggered

Questions:

1. Is this a known limitation of Gemini's API (google_search incompatible with responseMimeType: 'application/json')?

2. Is there a recommended pattern in ruby_llm for combining search grounding with structured responses? For example, should I parse JSON from the text response manually?

3. Would it make sense to have a built-in with_search_grounding helper that handles this case?

Current workaround: Use search grounding without schema and parse JSON from text response:

response = chat
  .with_params(tools: [{ google_search: {} }])
  .ask('Return JSON with price and currency for AAPL stock')

json = JSON.parse(response.content.match(/\{.*\}/m)[0])

Thanks for this great gem!