Skip to content

Commit 9ca9d34

Browse files
Bump tiktoken from 0.5.2 to 0.6.0 (#110)
[//]: # (dependabot-start) ⚠️ **Dependabot is rebasing this PR** ⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. --- [//]: # (dependabot-end) Bumps [tiktoken](https://github.com/openai/tiktoken) from 0.5.2 to 0.6.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/openai/tiktoken/blob/main/CHANGELOG.md">tiktoken's changelog</a>.</em></p> <blockquote> <h2>[v0.6.0]</h2> <ul> <li>Optimise regular expressions for a 20% performance improvement, thanks to <a href="https://github.com/paplorinc"><code>@​paplorinc</code></a>!</li> <li>Add <code>text-embedding-3-*</code> models to <code>encoding_for_model</code></li> <li>Check content hash for downloaded files</li> <li>Allow pickling <code>Encoding</code> objects. Registered <code>Encoding</code> will be pickled by reference</li> <li>Workaround PyO3 bug for frozenset conversion</li> </ul> <p>Thank you to <a href="https://github.com/paplorinc"><code>@​paplorinc</code></a>, <a href="https://github.com/mdwelsh"><code>@​mdwelsh</code></a>, <a href="https://github.com/Praneet460"><code>@​Praneet460</code></a>!</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/openai/tiktoken/commit/84d88dca52d591481b986ae386108e632b32cd61"><code>84d88dc</code></a> Allow use of gpt-2 and gpt-3.5 in encoding_for_model (<a href="https://redirect.github.com/openai/tiktoken/issues/185">#185</a>)</li> <li><a href="https://github.com/openai/tiktoken/commit/01df4360a1fa4581550b6f144e1541c952824569"><code>01df436</code></a> Update cibuildwheel</li> <li><a href="https://github.com/openai/tiktoken/commit/89153d70db062e35c585c06568f6ace6f43079bd"><code>89153d7</code></a> Sync codebase</li> <li><a href="https://github.com/openai/tiktoken/commit/55c8d83da33a2986b5cb259997d6b2c20f078d79"><code>55c8d83</code></a> added two new embedding model's encoding (<a href="https://redirect.github.com/openai/tiktoken/issues/247">#247</a>)</li> <li><a href="https://github.com/openai/tiktoken/commit/6cc3a46c8dd43a913612b9c839a51936a2c6bd41"><code>6cc3a46</code></a> Optimize regular expressions used for splitting by ~20% (<a href="https://redirect.github.com/openai/tiktoken/issues/234">#234</a>)</li> <li><a href="https://github.com/openai/tiktoken/commit/db5bda9fc93b3171db6c4afea329394e6b6d31ca"><code>db5bda9</code></a> Clarify language models in README (<a href="https://redirect.github.com/openai/tiktoken/issues/203">#203</a>)</li> <li><a href="https://github.com/openai/tiktoken/commit/3ee6c3517d3465775dc22f06a034cfcf8d06eba7"><code>3ee6c35</code></a> Add support for checking hash of downloaded files before use. (<a href="https://redirect.github.com/openai/tiktoken/issues/230">#230</a>)</li> <li>See full diff in <a href="https://github.com/openai/tiktoken/compare/0.5.2...0.6.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=tiktoken&package-manager=pip&previous-version=0.5.2&new-version=0.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 parent ec99222 commit 9ca9d34

File tree

2 files changed

+39
-39
lines changed

2 files changed

+39
-39
lines changed

poetry.lock

Lines changed: 38 additions & 38 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ langchain = ">=0.0.352,<0.1.6"
2626
pytest-docker = ">=2.0.1,<4.0.0"
2727
pytest-xdist = "^3.5.0"
2828
openai = "^1.6.0"
29-
tiktoken = "^0.5.2"
29+
tiktoken = ">=0.5.2,<0.7.0"
3030
pytest-cov = "^4.1.0"
3131

3232
[tool.poetry.group.codespell]

0 commit comments

Comments
 (0)