Skip to content

Pull requests: zurawiki/tiktoken-rs

New pull request New
8 Open 79 Closed
8 Open 79 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Update with GPT-5 Support
#116 opened Aug 11, 2025 by LoganBresnahan Loading…
Add tiktoken CLI 🚀
#115 opened Aug 7, 2025 by kyle-rader Loading…
api: Use bpe singleton for num_tokens_from_messages
#114 opened Aug 7, 2025 by smitbarmase Loading…
Add: o200k_harmony tokenizer for gpt-oss and update model mapping.
#113 opened Aug 5, 2025 by CaffeeLake Loading…
fix(deps): update rust crate async-openai to 0.29.0
#112 opened Jul 1, 2025 by renovate bot Loading…
1 task
chore(deps): update swatinem/rust-cache digest to 98c8021
#111 opened Jul 1, 2025 by renovate bot Loading…
1 task
feat: llama3 tokenizer
#64 opened May 21, 2024 by omarkilani Loading…
4
feat: add multi-language bindings for tiktoken using polywrap wasm wrapper
#36 opened Sep 8, 2023 by Niraj-Kamdar • Draft
ProTip! Mix and match filters to narrow down what you’re looking for.