llama : add comment about thread-safety [no ci] (ggml-org#9449)

ggerganov · dsx1986 · commit fcefb5b23714 · 2024-10-29T10:30:38.000-07:00
diff --git a/include/llama.h b/include/llama.h
@@ -911,6 +911,8 @@ extern "C" {
     //
     // Tokenization
     //
+    // The API is thread-safe.
+    //
 
     /// @details Convert the provided text into tokens.
     /// @param tokens The tokens pointer must be large enough to hold the resulting tokens.
