Skip to content

Conversation

@dependabot
Copy link

@dependabot dependabot bot commented on behalf of github Sep 22, 2025

Bumps tokenizers from 0.19.1 to 0.22.1.

Release notes

Sourced from tokenizers's releases.

v0.22.1

Release v0.22.1

Main change:

  • Bump huggingface_hub upper version (#1866) from @​Wauplin
  • chore(trainer): add and improve trainer signature (#1838) from @​shenxiangzhuang
  • Some doc updates: c91d76ae558ca2dc1aa725959e65dc21bf1fed7e, 7b0217894c1e2baed7354ab41503841b47af7cf9, 57eb8d7d9564621221784f7949b9efdeb7a49ac1

v0.22.0

What's Changed

New Contributors

Full Changelog: huggingface/tokenizers@v0.21.3...v0.22.0rc0

v0.21.4

Full Changelog: huggingface/tokenizers@v0.21.3...v0.21.4

No change, the 0.21.3 release failed, this is just a re-release.

https://github.com/huggingface/tokenizers/releases/tag/v0.21.3

v0.21.3

What's Changed

Full Changelog: huggingface/tokenizers@v0.21.2...v0.21.3

... (truncated)

Commits
  • afaae08 push the minor
  • 6cbd461 update version to 0.22.1.rc0
  • 972e7fc Bump huggingface_hub upper version (#1866)
  • c0d3697 chore(trainer): add and improve trainer signature (#1838)
  • c91d76a Add DecodeStream section to decoders documentation
  • 7b02178 Nit rendering the doc
  • 57eb8d7 Add ByteLevel normalizer to documentation
  • da1cc3b update dev version to 0.22.1-dev.0
  • 9bafd82 Revert "RUSTSEC-2024-0436 - replace paste with pastey (#1834)"
  • bd1149c Tokenizer: Add native async bindings, via py03-async-runtimes. (#1843)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [tokenizers](https://github.com/huggingface/tokenizers) from 0.19.1 to 0.22.1.
- [Release notes](https://github.com/huggingface/tokenizers/releases)
- [Changelog](https://github.com/huggingface/tokenizers/blob/main/RELEASE.md)
- [Commits](huggingface/tokenizers@v0.19.1...v0.22.1)

---
updated-dependencies:
- dependency-name: tokenizers
  dependency-version: 0.22.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update python code labels Sep 22, 2025
@dependabot @github
Copy link
Author

dependabot bot commented on behalf of github Jan 5, 2026

Superseded by #19.

@dependabot dependabot bot closed this Jan 5, 2026
@dependabot dependabot bot deleted the dependabot/pip/tokenizers-0.22.1 branch January 5, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant