Skip to content

Commit 5a6c087

Browse files
committed
Bump version to 1.37.0
1 parent d5c1908 commit 5a6c087

File tree

2 files changed

+13
-1
lines changed

2 files changed

+13
-1
lines changed

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,18 @@ The project follows [semantic versioning 2.0.0](https://semver.org/). The API co
2626

2727
### Fixes and improvements
2828

29+
## [v1.37.0](https://github.com/OpenNMT/Tokenizer/releases/tag/v1.37.0) (2023-02-28)
30+
31+
### New features
32+
33+
* Add tokenization option `allow_isolated_marks` to allow combining marks to appear isolated in the tokenization output in specific conditions
34+
35+
### Fixes and improvements
36+
37+
* Fix infinite loop when the text contains an invalid Unicode character
38+
* Fix segmentation fault when the `BPELearner` does not not find any pairs of characters in the tokenized data
39+
* [Python] Update ICU to 72.1
40+
2941
## [v1.36.0](https://github.com/OpenNMT/Tokenizer/releases/tag/v1.36.0) (2023-01-11)
3042

3143
### New features
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
"""Version information."""
22

3-
__version__ = "1.36.0"
3+
__version__ = "1.37.0"

0 commit comments

Comments
 (0)