We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 0190d0a commit 41b0f77Copy full SHA for 41b0f77
README.md
@@ -37,9 +37,9 @@ Mix.install([
37
{:ok, tokenizer} = Tokenizers.Tokenizer.from_pretrained("bert-base-cased")
38
{:ok, encoding} = Tokenizers.Tokenizer.encode(tokenizer, "Hello there!")
39
Tokenizers.Encoding.get_tokens(encoding)
40
-# {:ok, ["Hello", "there", "!"]}
+# ["Hello", "there", "!"]
41
Tokenizers.Encoding.get_ids(encoding)
42
-# {:ok, [8667, 1175, 106]}
+# [8667, 1175, 106]
43
```
44
45
The [notebooks](./notebooks) directory has [an introductory Livebook](./notebooks/pretrained.livemd) to give you a feel for the API.
0 commit comments