llama : add comment about thread-safety [no ci] (#9449)

ggerganov · ggerganov · commit 739842703e32 · 2024-09-28T15:13:42.000+03:00
diff --git a/include/llama.h b/include/llama.h
@@ -911,6 +911,8 @@ extern "C" {
     //
     // Tokenization
     //
+    // The API is thread-safe.
+    //
 
     /// @details Convert the provided text into tokens.
     /// @param tokens The tokens pointer must be large enough to hold the resulting tokens.
