You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[](https://colab.research.google.com/github/Xerv-AI/crayon/blob/main/colab_benchmark.ipynb)
279
+
### ✅ Quick Verify Snippet
274
280
275
-
1. Open the notebook.
276
-
2. Change Runtime type to **T4 GPU**.
277
-
3. Run the cells to verify `crayon_cuda` compiles and smashes tokens at >100M/sec.
281
+
```python
282
+
from crayon import CrayonVocab
283
+
284
+
# Initialize with Auto-Backend (AVX2/CUDA/ROCm)
285
+
tokenizer = CrayonVocab(device="auto")
278
286
287
+
# 1. Test Standard subword-heavy profile
288
+
tokenizer.load_profile("standard")
289
+
print(tokenizer.tokenize("that is a test for the standard profile"))
0 commit comments