1.10.0

Release Notes

Very excited to announce initial support the Anthropic messages API for providers! With our initial implementation we worked to build a compatibility layer between you and the provider for all providers that don't yet natively support the messages API. Over the coming months we'll be adding even more robust testing and verification to make sure that it works seamlessly for all providers. We have one integration test that runs to check all of the providers, but there are a few providers that we don't actively run integration tests against, so please feel empowered to file github issues if you encounter any issues and we'll work to help you patch it fast.

What's Changed

• feat: add Anthropic Messages API as provider-level feature by @njbrake in #895

Full Changelog: 1.9.0...1.10.0

1.9.0

Breaking Change

If you use anthropic provider, please note that per #894 we now support adaptive thinking and older models will need to manually specify the "thinking budget" parameter, we don't set that for you by default anymore. We try to keep breaking changes at a minimum but this felt necessary in order to pull anthropic back in line with all of the other providers. Please let us know if you experience any issues. 🙏

What's Changed

• ci: allow maintainers to trigger integration tests on fork PRs via label by @njbrake in #863
• fix(openai): map max_tokens to max_completion_tokens by @njbrake in #865
• feat(mzai): add mzai provider with platform JWT token authentication by @dpoulopoulos in #853
• fix: move max_tokens → max_completion_tokens remapping to BaseOpenAIProvider by @njbrake in #868
• fix(mzai): remove redundant SUPPORTS_* attributes by @dpoulopoulos in #869
• fix: correct malformed pip install command in missing packages error message by @Anandesh-Sharma in #871
• fix: correct typo "provideers" in comment by @Anandesh-Sharma in #872
• fix(tests): remove duplicate MISTRAL entry in provider_reasoning_model_map by @Anandesh-Sharma in #873
• fix: update responses/aresponses/aembedding docstrings to recommend provider:model format by @Anandesh-Sharma in #874
• fix(tests): remove duplicate OPENAI entry in provider_client_config by @Anandesh-Sharma in #875
• chore: remove dead PACKAGES_INSTALLED attribute from providers by @Anandesh-Sharma in #876
• tests: add unit tests for moonshot provider by @Anandesh-Sharma in #877
• tests: add unit tests for lmstudio provider by @Anandesh-Sharma in #878
• tests: add unit tests for minimax provider by @Anandesh-Sharma in #879
• tests: add unit tests for databricks provider by @Anandesh-Sharma in #880
• feat: Return ParsedChatCompletion[T] for structured output across all providers by @bilelomrani1 in #812
• fix(ci): mock completion call for docs test so that forks can pass the docs test by @njbrake in #890
• chore(deps): bump actions/github-script from 7 to 8 by @dependabot[bot] in #887
• chore(deps-dev): bump pre-commit-uv from 4.2.0 to 4.2.1 by @dependabot[bot] in #888
• pre-commit autoupdate by @github-actions[bot] in #886
• feat(anthropic): add PDF support for Anthropic provider by @njbrake in #891
• feat: accept dataclass types in response_format for structured output by @njbrake in #892
• feat(anthropic): support adaptive thinking by @njbrake in #894

New Contributors

• @Anandesh-Sharma made their first contribution in #871

Full Changelog: 1.8.6...1.9.0

1.8.6

What's Changed

• fix: openapi spec by @njbrake in #821
• chore: drop the demos by @njbrake in #820
• chore: update models for integration tests of cerebras and fireworks by @njbrake in #822
• chore: bot to flag prs that have the template deleted by @njbrake in #825
• fix(tests): anthropic dropped haiku 3.5 by @njbrake in #826
• fix(gemini): Extract cached tokens into prompt_tokens_details by @bilelomrani1 in #815
• fix(groq): Extract cached tokens into prompt_tokens_details by @bilelomrani1 in #813
• fix(bedrock): Extract cached tokens into prompt_tokens_details by @bilelomrani1 in #814
• fix(deepseek): Extract cached tokens into prompt_tokens_details by @bilelomrani1 in #816
• Add max_completion_tokens to gateway ChatCompletionRequest by @sarmientoF in #817
• fix: openapi spec by @njbrake in #828
• fix(gateway): pricing lookup uses wrong separator, cost always NULL by @sarmientoF in #827
• fix: make integration tests work for contrib approved workflow runs by @njbrake in #829
• fix: llamafile supports tools by @njbrake in #830
• feat: anthropic support for response format by @njbrake in #831
• fix(anthropic): strip OpenAI-specific fields from messages by @khaledosman in #832
• fix(gateway): use atomic SQL update for user spend increment by @njbrake in #835
• fix(gateway): prevent budget race condition with row locking by @njbrake in #836
• fix(gateway): sanitize error messages to prevent information leakage by @njbrake in #839
• fix(gateway): make CORS configurable, remove unsafe wildcard+credentials by @njbrake in #837
• fix(gateway): raise error for unresolved env var references in config by @njbrake in #838
• fix(gateway): isolate usage logging within a savepoint by @njbrake in #841
• fix(gateway): use last non-zero value for streaming token aggregation by @njbrake in #848
• fix(gateway): add reset functions for global DB and config state by @njbrake in #840
• fix(gateway): add upper bounds to pagination parameters by @njbrake in #843
• fix(gateway): cache and clean up Vertex AI temp credential files by @njbrake in #851
• fix(gateway): add structural validation for chat completion messages by @njbrake in #842
• fix(gateway): validate pricing and budget values are non-negative by @njbrake in #844
• fix(gateway): prevent provider config from overwriting user request fields by @njbrake in #845
• feat(gateway): add configurable per-user rate limiting by @njbrake in #846
• fix(gemini): pass api_base to genai.Client via http_options by @njbrake in #852
• fix(gateway): emit SSE error event on mid-stream failures by @njbrake in #847
• fix(gateway): use timezone-aware datetimes consistently by @njbrake in #849
• fix(pytest): remove unused marker by @njbrake in #854
• fix(huggingface): switch from Inference Endpoints to Inference Provid… by @njbrake in #855
• fix(gateway): preserve usage and reset logs when deleting users by @njbrake in #850
• fix(gateway): allow free models by @njbrake in #857
• chore: fix gateway test by @njbrake in #859
• feat(lmstudio): now supports responses by @njbrake in #858
• fix: platform should not be a standalone provider by @njbrake in #861

New Contributors

• @sarmientoF made their first contribution in #817
• @khaledosman made their first contribution in #832

Full Changelog: 1.8.5...1.8.6

1.8.4

What's Changed

• chore: Bump any-llm-platform-client>=0.2.2 by @daavoo in #818

Full Changelog: 1.8.3...1.8.4

1.8.3

What's Changed

• refactor(platform): Reject client_name at _acompletion. by @daavoo in #811

Full Changelog: 1.8.2...1.8.3

1.8.2

What's Changed

• fix(platform): Also handle client_name inside _acompletion by @daavoo in #808

Full Changelog: 1.8.1...1.8.2

1.8.1

What's Changed

• fix: handle empty provider response in _convert_chat_completion by @lanesket in #803
• fix(platform): Avoid sending client_name to downstream provider call. by @daavoo in #807

New Contributors

• @lanesket made their first contribution in #803

Full Changelog: 1.8.0...1.8.1

1.8.0

What's Changed

• feat(platform): include SDK version in usage event requests by @tbille in #742
• feat: bedrock support for images by @njbrake in #751
• fix: add anthropic prompt caching by @njbrake in #750
• refactor: update Docker workflow for testing and liveness checks by @njbrake in #740
• feat: support VertexAIAnthropic Provider by @njbrake in #776
• Feat: Support OpenResponses Spec by @njbrake in #775
• fix: moonshot model id by @njbrake in #779
• feat: several providers don't yet support python 3.14 by @njbrake in #781
• chore(deps): bump lodash from 4.17.21 to 4.17.23 in /demos/finder/frontend by @dependabot[bot] in #777
• chore(deps): bump lodash from 4.17.21 to 4.17.23 in /demos/chat/frontend by @dependabot[bot] in #772
• chore: retry integration tests and generate report by @njbrake in #782
• chore: add override decorator by @njbrake in #784
• feat: API_BASE env var for each provider by @njbrake in #783
• feat: consolidate xml parse logic by @njbrake in #785
• fix: let pydantic ValidationError bubble up unchanged by @bilelomrani1 in #800
• fix(together): support OpenAI-style dict response_format by @ThanhNguyxn in #798

New Contributors

• @ThanhNguyxn made their first contribution in #798

Full Changelog: 1.7.0...1.8.0

1.7.0

What's Changed

• chore: add CLAUDE.md as a symlink to AGENTS.md by @njbrake in #732
• fix(together): don't use prerelease version by @njbrake in #733
• feat: platform docs by @njbrake in #737
• chore: AGENTS.md specify that tests must be standalone functions by @njbrake in #736
• feat: mistral batch support by @njbrake in #734
• feat: add xhigh reasoning effort level by @peteski22 in #738

New Contributors

• @peteski22 made their first contribution in #738

Full Changelog: 1.6.2...1.7.0

1.6.2

What's Changed

• refactor: streamline version handling in pyproject.toml by @njbrake in #723
• refactor(platform): migrate usage events to JWT Bearer token auth by @tbille in #727

Full Changelog: 1.6.1...1.6.2