Skip to content

Commit 752eaf1

Browse files
author
Philip Sampaio
committed
Release v0.3.1
1 parent 5c924cc commit 752eaf1

File tree

4 files changed

+18
-5
lines changed

4 files changed

+18
-5
lines changed

CHANGELOG.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [v0.3.1] - 2023-04-06
11+
12+
### Added
13+
14+
- Add binary variants for accessing encoding data. This way we can convert encoding
15+
data to tensors without additional allocations. The following functions were added:
16+
17+
- `get_u32_ids/1`
18+
- `get_u32_attention_mask/1`
19+
- `get_u32_type_ids/1`
20+
- `get_u32_special_tokens_mask/1`
21+
1022
## [v0.3.0] - 2023-03-04
1123

1224
### Added
@@ -42,7 +54,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
4254

4355
First release.
4456

45-
[Unreleased]: https://github.com/elixir-nx/tokenizers/compare/v0.3.0...HEAD
57+
[Unreleased]: https://github.com/elixir-nx/tokenizers/compare/v0.3.1...HEAD
58+
[v0.3.1]: https://github.com/elixir-nx/tokenizers/compare/v0.3.0...v0.3.1
4659
[v0.3.0]: https://github.com/elixir-nx/tokenizers/compare/v0.2.0...v0.3.0
4760
[v0.2.0]: https://github.com/elixir-nx/tokenizers/compare/v0.1.0...v0.2.0
4861
[v0.1.0]: https://github.com/elixir-nx/tokenizers/releases/tag/v0.1.0

mix.exs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ defmodule Tokenizers.MixProject do
22
use Mix.Project
33

44
@source_url "https://github.com/elixir-nx/tokenizers"
5-
@version "0.4.0-dev"
5+
@version "0.3.1"
66

77
def project do
88
[

native/ex_tokenizers/Cargo.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

native/ex_tokenizers/Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
[package]
22
name = "ex_tokenizers"
3-
version = "0.3.0"
3+
version = "0.3.1"
44
authors = []
5-
edition = "2018"
5+
edition = "2021"
66

77
[lib]
88
name = "ex_tokenizers"

0 commit comments

Comments
 (0)