Version 0.4

What's Changed

Observability

• Added token and round-trip observability at the provider and agent level
• Added unit tests

Usage

Observable agents and providers should mirror the standard API (and might be merged eventually). If you're already using the standard API, you can just swap the import statement:

import agents.observability as agents

Adding OpenAI Batch API support

I added a new AzureOpenAIBatchProvider provider class which works in conjunction with a new _BatchAPIHelper class to gather all OpenAI web requests into a single batch request, process that request, and return the results back to the requesting agents

This should involve some breaking changes to the API, notably:

• Providers now call endpoint_fn, which is monkey-patched depending on whether chat or batch endpoints are required.
• Providers get async with entry/exit points for async task cleanup in the batch case (superfluous in the chat endpoint case)
• Substantial re-writing of the Processor class (see below)

Example Usage

https://github.com/cdcai/agents/blob/ba5f99dc472dd45f9d0375414ccba638f64d9f04/examples/batch_api.py#L108-L116

async def main():
    async with agents.AzureOpenAIBatchProvider(
        "gpt-4o-batch", batch_size=5, n_workers=2
    ) as provider:
        # Kind of a hacky way to use this, but just for demonstration purposes
        proc = agents.BatchProcessorIterable(
            [i for i in range(10)], KnockKnockAgent, batch_size=1, provider=provider
        )

        jokes = await proc.process()

Processor Changes

Overhauled the standard Processor logic to handle batch API usage.

Rationale

Previous method fired only as many agents as there were requested workers at any given time. This is inefficient for the Batch API case where we'd like to fire all the agents at once and let the batcher handle processing them all.

API Changes

• ProcessorIterable / ProcessorDF are the chat endpoint variants
• BatchProcessorIterable / BatchProcessorDF are the batch endpoint variants
• Provider is now a required field upon init to handle case batch case

Misc

• tests/ -> test/
• Asyncio version of tqdm used instead of standard variant (probably not needed)
• Fixed CI/CD for unit tests by adding test deps requirement file

Full Changelog: v0.3.3...v0.4