Define TemporalAgents at runtime

[jhammarstedt]

Description

Our agentic solution relies on agents being defined at runtime based on configuration that varies depending on the user.

Using temporal the agents need to be set up during startup which creates a problems since we don't know the exact configuration we want the model to have at that time. We have solved this by defining a few global agents with a superset of tools that we then overwrite at runtime with an agent that has a subset of these. This solve the problem partly but we still cannot set model configuration at runtime.

What is the best approach here? How are we supposed to configure our agents at runtime?

References

No response

[DouweM]

@jhammarstedt Thanks for filing the issue; I agree we should come up with a plan for this.

Let me start by quoting from a related conversation I had with someone else recently:

The way Pydantic AI is built currently, Python function tools and model class instances cannot be serialized, so they can’t be sent from the workflow to the activity — they need to be known ahead of time so that they'll be available in the activity process. Pretty much everything else (including which tools or MCP servers are available) can be modified at agent.run-time via arguments or by combining deps with dynamic instruction functions and other functions that take RunContext + deps.

So you could also think of the Agent instance as the "agent spec", and the combination of agent + run args + deps as the "agent", each of which could be doing something very different while being built on the same spec.

There's no fundamental reason we can't also serialize (some) tool functions (e.g. those that only use input args and RunContext.deps) or models, and allow entire Agent instances to be built on the fly in the workflow, and that's something we could work on in the future (or take contributions for 🙂). But you can get very far with the existing runtime args if you think of what we (currently) call "agent" and "agent run" more as "agent spec" vs "agent".

---

We have solved this by defining a few global agents with a superset of tools that we then overwrite at runtime with an agent that has a subset of these.

Great, that's pretty much the "agent spec" approach I described above.

This solve the problem partly but we still cannot set model configuration at runtime.

What model configuration are you not currently able to override at runtime?

See #3407 for being able to swap models at runtime.

[Goldokpa]

GitHub Issue Comment for #3506

Suggested Comment

Thanks for the detailed explanation, @DouweM! I've been working through this issue and have some insights to share.

Current Limitations

Regarding your question about what model configuration can't be overridden at runtime:

Cannot be set at runtime:

• The model instance itself (e.g., switching from gpt-4 to claude-3) - must be set at agent creation time
• Toolsets - must be set at agent creation time for use in workflows
• Tools - must be set at agent creation time for use in workflows

Can be set at runtime:

• Model settings (temperature, max_tokens, etc.) - ✅ via model_settings parameter
• Instructions - ✅ via instructions parameter
• Dependencies - ✅ via deps parameter

So the main limitation is that we can't swap the actual model instance at runtime (e.g., switching from OpenAI to Anthropic based on user configuration).

Recommended Solution

Based on my analysis of the codebase, the best approach is to use a configuration-based agent registry pattern:

1. Pre-register all possible agent configurations at module import time (before workflows are defined)
2. Map user settings to configuration IDs that select pre-registered agents
3. Use runtime parameters (model_settings, instructions, deps) for fine-grained customization

This works within Temporal's constraints because:

• All activities are registered upfront (required by Temporal)
• Agent names and toolset IDs remain stable (required for activity naming)
• Workflows remain deterministic (required by Temporal)

Example Implementation

I've created a complete example demonstrating this pattern: runtime_agent_configuration.py

The key pattern is:

# Register all configurations at startup
AGENT_REGISTRY: dict[str, TemporalAgent] = {}

def register_agent_config(config_id: str, model: Model, toolsets: list):
    agent = Agent(model=model, name=f'agent_{config_id}', toolsets=toolsets)
    temporal_agent = TemporalAgent(agent)
    AGENT_REGISTRY[config_id] = temporal_agent
    return temporal_agent

# In workflow, select by configuration ID
@workflow.defn
class UserAgentWorkflow:
    @workflow.run
    async def run(self, user_config_id: str, prompt: str):
        agent = AGENT_REGISTRY[user_config_id]
        return await agent.run(prompt, model_settings={...})

Trade-offs

The main trade-off is that all configurations must be known at startup. However, this is a reasonable constraint given Temporal's architecture, and you can:

• Use model_settings to customize model behavior at runtime
• Use instructions to customize agent behavior at runtime
• Version configurations (e.g., premium_user_v2) to support changes without breaking active workflows

Future Enhancement

For true dynamic model selection, we'd need to serialize model instances (as mentioned in #3407). Until then, the registry pattern provides a practical solution that works with the current architecture.

Would you like me to create a PR with documentation and examples for this pattern?