How to ensure LLMAgent(..., output_schema=...) always returns valid output?

[lenixbyte]

I'm Using:

LLMAgent(
name="MyAgent",
instructions="...",
output_schema=MySchema
)

If the LLM produces invalid structure or missing fields, Pydantic validation fails and the frontend receives an unpredictable response.

What is the best practice to guarantee that final outputs always match output_schema?

Specifically:

Where should schema enforcement + fallback happen?

How to emit a safe final event when validation fails?

Should intermediate events also follow a minimal schema?

Looking for official ADK guidance or production patterns to keep the contract stable when the model deviates.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: The most robust method to guarantee schema validation is to use a multi-agent "Generator-Critic" pattern, where one agent generates the output and a second agent validates it. For simpler cases, you can use an after_model_callback to inspect the output and handle failures.

---

That's an excellent question about ensuring output reliability. Here are some recommended patterns and practices based on ADK's capabilities:

1. The Generator-Critic Pattern (Most Robust)

The cleanest and most powerful way to enforce output structure is by using a multi-agent pattern where different agents have different responsibilities. [1] You can orchestrate this using a SequentialAgent:

• Agent 1 (Generator): An LlmAgent responsible for generating the response based on your instructions and attempting to fit the output_schema.
• Agent 2 (Critic/Validator): A second agent (either a custom BaseAgent or another LlmAgent) that takes the output from the Generator. Its sole job is to validate the data against your Pydantic schema.
  • If validation passes, it finalizes the turn and returns the valid output.
  • If validation fails, it can either return a safe, default error response or send feedback to the Generator to retry. You could wrap this interaction in a LoopAgent for a set number of retries.

2. Using Callbacks for Validation (Simpler)

For less complex scenarios, you can use callbacks as a lightweight way to inspect and intercept the agent's process. [2]

• after_model_callback: This callback is triggered after the LLM call is complete but before the final event is yielded. You can inspect the llm_response here, try to parse it with your Pydantic schema, and if it fails, you can return a new LlmResponse object containing a safe fallback message. This prevents the invalid response from propagating.

  def validate_output(resource, llm_response, **kwargs):
      try:
          MySchema.parse_obj(llm_response.final_output)
          return llm_response  # It's valid, do nothing
      except ValidationError as e:
          # Validation failed, return a safe response
          return LlmResponse(
              final_output={"error": "Invalid output from model", "details": str(e)}
          )
  
  agent = LlmAgent(
      ...,
      after_model_callback=validate_output
  )

3. How to Emit a Safe Final Event

When validation fails in a callback or a Critic agent, you can emit a custom, safe event. This ensures your frontend always receives a predictable structure.

# Inside an agent or callback
if validation_failed:
    yield Event(
        author=self.name,
        content={"status": "error", "message": "Output validation failed."}
    )

4. Schema for Intermediate Events

You asked if intermediate events should follow a schema. While not strictly enforced by the framework for all events, it is a strong best practice. [3] ADK's own components for logging and analytics rely on a consistent event structure. Defining a schema for your intermediate events will make your application much easier to debug, extend, and maintain.

Finally, it's worth noting that ADK now supports using output_schema and tools together. When you do this, ADK automatically creates a special tool for the model to call with the final structured output, which can also help improve reliability. [4]

[1] https://google.github.io/adk-docs/agents/multi-agents/
[2] https://google.github.io/adk-docs/callbacks/design-patterns-and-best-practices/
[3] https://google.github.io/adk-docs/events/
[4] https://github.com/google/adk-python/blob/main/contributing/samples/output_schema_with_tools/README.md

[cabmeron]

@adk-bot

thanks adk-bot, your service is greatly appreciated

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: ADK strictly enforces the output_schema on the final response of an LLMAgent. If the model's output does not validate against your Pydantic schema, ADK will raise a pydantic.ValidationError. The recommended practice is to wrap your agent execution call in a try...except block to catch this exception and implement your own fallback logic, such as sending a default response or logging the error. Intermediate events, like tool calls, have their own structure and are not subject to the final output_schema.

---

Hello! That's an excellent question about ensuring API contract stability when working with LLMs. Here is some guidance based on ADK's design and best practices.

Schema Enforcement and Fallback

When you define an output_schema for an LLMAgent, ADK requires the agent's final response to be a JSON string that strictly conforms to that schema. [1] This validation is handled automatically by ADK, which uses pydantic.BaseModel.model_validate_json internally. [2]

If the LLM produces an output that is not a valid JSON or does not match the structure of your Pydantic model (e.g., missing fields, incorrect types), the ADK will raise a pydantic.ValidationError. [3]

ADK does not have a built-in automatic fallback mechanism. The intended pattern is for you to handle these validation failures explicitly in your application code. The best practice is to wrap the agent invocation (e.g., in a call to run_async) within a try...except pydantic.ValidationError block.

from pydantic import ValidationError

try:
    # This is where you run your agent
    final_event = await agent.run_async(prompt="...")
    # Process the successful, validated response

except ValidationError as e:
    # Fallback logic here
    print(f"Schema validation failed: {e}")
    # You can emit a safe, default final event, or a custom error message.
    # For example:
    # final_event = {'error': 'Model output failed validation', 'details': str(e)}

Emitting a Safe Final Event

As shown in the example above, your except block is the correct place to define what a "safe" final event looks like. When a ValidationError is caught, you can:

1. Log the error for debugging.
2. Create a default response object that signals a failure to the frontend in a predictable way.
3. Attempt a retry strategy, perhaps by re-prompting the model with additional instructions about the desired format.

This explicit error handling ensures that your application maintains a stable contract with the frontend, even when the model deviates.

Schema for Intermediate Events

The output_schema applies specifically to the final output of the agent. [1] Intermediate events, such as model-generated text or the invocation of tools, follow their own structure as defined by the ADK framework. [4]

For agents that use both tools and an output_schema, ADK injects a special tool (often named set_model_response) that the model must call to provide its final, structured response. [5] The arguments to this special tool are what get validated against your output_schema.

There is no requirement or mechanism to enforce a custom schema on these intermediate events. Their structure is integral to how the agent reasons and executes tasks.

I hope this guidance helps you build more robust and reliable applications with ADK!

---

[1] Publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/agents/llm-agents/
[2] Publicly accessible HTTPS URL of the document: https://github.com/google/adk-python/blob/main/src/google/adk/tools/agent_tool.py
[3] Publicly accessible HTTPS URL of the document: https://github.com/google/adk-python/blob/main/CHANGELOG.md
[4] Publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/streaming/dev-guide/part3/
[5] Publicly accessible HTTPS URL of the document: https://github.com/google/adk-python/blob/main/contributing/samples/output_schema_with_tools/README.md