0.8.0

What's new

• bump minimum vLLM requirement to v0.10.0, drop prompt adapters by @dtrifiro, @heyselbi in #269

Note

In our testing, vllm==0.10.0 fails to start up on CPU

What's Changed

• build(deps): bump ruff from 0.9.6 to 0.11.11 by @dependabot[bot] in #255
• build(deps): bump types-requests from 2.32.0.20241016 to 2.32.0.20250515 by @dependabot[bot] in #253

Full Changelog: 0.7.1...0.8.0

0.7.1

What's Changed

• ♻️ Consolidate vllm utils and allow vllm version to be overwritten by @evaline-ju in #247
• build(deps): bump accelerate from 1.3.0 to 1.6.0 by @dependabot in #246
• build(deps): bump mypy from 1.12.1 to 1.15.0 by @dependabot in #220
• build(deps): bump accelerate from 1.6.0 to 1.7.0 by @dependabot in #251
• pre-commit autoupdate by @dependabot in #227
• build(deps): bump pytest from 8.3.3 to 8.3.5 by @dependabot in #236
• 🔥 remove version pins for opentelemetry by @joerunde in #250
• gha improvements by @dtrifiro in #252

New Contributors

• @evaline-ju made their first contribution in #247

Full Changelog: 0.7.0...0.7.1

0.7.0

What's new

• Compatibility with vLLM v1 engine (vllm>=0.8.0) by @tjohnson31415 in #245

What's Changed

• gha: use inline annotations for ruff linting errors by @dtrifiro in #237
• redact guided decoding params by @rafvasq in #238
• pre-commit autoupdate by @github-actions in #222
• gha: fix ci caching by @dtrifiro in #241
• fix after change to MistralTokenizer upstream by @tjohnson31415 in #240
• gha: separate uv caches by @dtrifiro in #243
• delete legacy tgis metrics by @joerunde in #244

Full Changelog: 0.6.3...0.7.0

0.6.3

What's Changed

Bugfixes

• create socket before starting the engine and pass socket into the vllm server on startup by @jberkhahn in #233
• fix package level logger name by @tjohnson31415 in #223

Changes

• Stop caching LoRA requests and use vllm server's cache @jberkhahn in #232
• deps: bump minimum vllm version to 0.7.2 by @dtrifiro in #234

Misc

• build(deps): bump hf-transfer from 0.1.8 to 0.1.9 by @dependabot in #217
• build(deps): bump ruff from 0.8.3 to 0.9.6 by @dependabot in #218
• build(deps): bump prometheus-client from 0.21.0 to 0.21.1 by @dependabot in #219
• build(deps): bump mypy-protobuf from 3.5.0 to 3.6.0 by @dependabot in #221

New Contributors

• @jberkhahn made their first contribution in #233

Full Changelog: 0.6.2...0.6.3

0.6.2

What's Changed

• linting fixes by @dtrifiro in #209
• 🐛 fixup correlation-id log for grpc calls by @joerunde in #206

deps

• build(deps): bump grpcio from 1.67.0 to 1.70.0 by @dependabot in #212
• build(deps): bump types-requests from 2.32.0.20240914 to 2.32.0.20241016 by @dependabot in #175
• build(deps): bump codecov/codecov-action from 4 to 5 by @dependabot in #186
• build(deps): bump accelerate from 0.34.2 to 1.3.0 by @dependabot in #214
• deps: bump grpcio-, opentelemetry- by @dtrifiro in #215 #216
• pre-commit autoupdate by @github-actions in #205 #210, #211

Full Changelog: 0.6.1...0.6.2

0.6.1

What's Changed

• 🐛 await init_app_state by @joerunde in #207
• deps: relax vllm to >= 0.6.4.post1 by @vaibhavjainwiz in #208

Full Changelog: 0.6.0...0.6.1

0.6.0

What's Changed

• build(deps): bump mypy from 1.11.2 to 1.12.0 by @dependabot in #164
• fix: duplicate stacktrace on engine error by @wallashss in #171
• build(deps): bump mypy from 1.12.0 to 1.12.1 by @dependabot in #168
• build(deps): bump grpcio from 1.66.2 to 1.67.0 by @dependabot in #169
• Updates for vllm v0.6.4 by @tjohnson31415 in #183
• Allow greedy decoding with 0.0 temperature by @joerunde in #182
• build(deps): bump ruff from 0.6.9 to 0.8.1 by @dependabot in #187
• build(deps): bump pytest-cov from 5.0.0 to 6.0.0 by @dependabot in #189
• 🔊 overhaul logging on engine by @joerunde in #191
• build(deps): bump ruff from 0.8.1 to 0.8.3 by @dependabot in #193
• 🐛 fixup request cancellation for v0.6.5 by @joerunde in #196
• Bump vllm to 0.6.6.post1 by @fialhocoelho in #195
• tests: fix for 0.6.7.dev by @dtrifiro in #202
• nox: use uv as default venv backend by @dtrifiro in #203
• gha: use uv in release workflow by @dtrifiro in #204
• pre-commit autoupdate by @github-actions in #176 #190 #197 #166 #170 #173

New Contributors

• @fialhocoelho made their first contribution in #195

Full Changelog: 0.5.3...0.6.0

0.5.3

What's Changed

• 🐛 handle MistralTokenizer special case by @prashantgupta24 in #162

Full Changelog: 0.5.2...0.5.3

0.5.2

What's Changed

• gha: fix caching strategy by @dtrifiro in #154
• build(deps): bump prometheus-client from 0.20.0 to 0.21.0 by @dependabot in #151
• build(deps): bump types-protobuf from 5.27.0.20240907 to 5.28.0.20240924 by @dependabot in #150
• ✨ add pt_to_prompt cli by @prashantgupta24 in #155
• Exploit vLLM options to return deltas/final-output only by @njhill in #137
• build(deps): bump ruff from 0.6.7 to 0.6.9 by @dependabot in #157
• Fix bug in example. by @tdoublep in #159
• pre-commit autoupdate by @github-actions in #142
• fix linter complaints, cleanup gha caching by @dtrifiro in #160
• gha: add ccache by @dtrifiro in #147
• build(deps): bump grpcio from 1.62.2 to 1.66.2 by @dependabot in #153
• Fail startup with root-cause exception by @njhill in #156
• ✨ invoke caikit -> peft conversion at load time by @joerunde in #161

New Contributors

• @tdoublep made their first contribution in #159

Full Changelog: 0.5.1...0.5.2

0.5.1

What's Changed

• gha: implement caching by @dtrifiro in #145
• 🐛 don't use vllm.commit by @prashantgupta24 in #148
• fix: add_special_tokens in tokenize by @rafvasq in #144

Full Changelog: 0.5.0...0.5.1