Skip to content

Conversation

V2arK
Copy link

@V2arK V2arK commented Apr 28, 2025

✨ Add CentML server-less support to huggingface.js/inference

What’s in this PR

Area Change
README Adds CentML to the supported-provider tables and links to the partner-model list.
Provider wiring * src/lib/getProviderHelper.ts – registers CentML for conversational .
* src/providers/centml.ts – new task classes
Docs Links to https://huggingface.co/api/partners/centml/models.
Misc Uses the current CentML base URL https://api.centml.com.

Why

* CentML just opened public server-less endpoints; supporting it keeps huggingface.js a one-stop client for all HF partners.
* CentML routes are OpenAI-shaped, so we extend the existing base classes.

How I tested

enable tests in packages/inference/test/InferenceClient.spec.ts, and did

pnpm test -- \
  --reporter verbose \
  test/InferenceClient.spec.ts -t "CentML"

> @huggingface/[email protected] test /Users/honglin/centML/huggingface.js/packages/inference
> vitest run --config vitest.config.mts "--reporter" "verbose" "test/InferenceClient.spec.ts" "-t" "CentML"


 RUN  v0.34.6 /Users/honglin/centML/huggingface.js/packages/inference

stderr | unknown test
Set HF_TOKEN in the env to run the tests for better rate limits

 ✓ test/InferenceClient.spec.ts (100) 4226ms
   ✓ InferenceClient (100) 4225ms
     ↓ backward compatibility (1) [skipped]
       ↓ works with old HfInference name [skipped]
     ↓ HF Inference (48) [skipped]
       ↓ throws error if model does not exist [skipped]
       ↓ fillMask [skipped]
       ↓ works without model [skipped]
       ↓ summarization [skipped]
       ↓ questionAnswering [skipped]
       ↓ tableQuestionAnswering [skipped]
       ↓ documentQuestionAnswering [skipped]
       ↓ documentQuestionAnswering with non-array output [skipped]
       ↓ visualQuestionAnswering [skipped]
       ↓ textClassification [skipped]
       ↓ textGeneration - gpt2 [skipped]
       ↓ textGeneration - openai-community/gpt2 [skipped]
       ↓ textGenerationStream - meta-llama/Llama-3.2-3B [skipped]
       ↓ textGenerationStream - catch error [skipped]
       ↓ textGenerationStream - Abort [skipped]
       ↓ tokenClassification [skipped]
       ↓ translation [skipped]
       ↓ zeroShotClassification [skipped]
       ↓ sentenceSimilarity [skipped]
       ↓ FeatureExtraction [skipped]
       ↓ FeatureExtraction - auto-compatibility sentence similarity [skipped]
       ↓ FeatureExtraction - facebook/bart-base [skipped]
       ↓ FeatureExtraction - facebook/bart-base, list input [skipped]
       ↓ automaticSpeechRecognition [skipped]
       ↓ audioClassification [skipped]
       ↓ audioToAudio [skipped]
       ↓ textToSpeech [skipped]
       ↓ imageClassification [skipped]
       ↓ zeroShotImageClassification [skipped]
       ↓ objectDetection [skipped]
       ↓ imageSegmentation [skipped]
       ↓ imageToImage [skipped]
       ↓ imageToImage blob data [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage with parameters [skipped]
       ↓ imageToText [skipped]
       ↓ request - openai-community/gpt2 [skipped]
       ↓ tabularRegression [skipped]
       ↓ tabularClassification [skipped]
       ↓ endpoint - makes request to specified endpoint [skipped]
       ↓ chatCompletion modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId - OpenAI Specs [skipped]
       ↓ chatCompletionStream modelId Fail - OpenAI Specs [skipped]
       ↓ chatCompletion - OpenAI Specs [skipped]
       ↓ chatCompletionStream - OpenAI Specs [skipped]
       ↓ custom mistral - OpenAI Specs [skipped]
       ↓ custom openai - OpenAI Specs [skipped]
       ↓ OpenAI client side routing - model should have provider as prefix [skipped]
     ↓ Fal AI (4) [skipped]
       ↓ textToImage - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage - SD LoRAs [skipped]
       ↓ textToImage - Flux LoRAs [skipped]
       ↓ automaticSpeechRecognition - openai/whisper-large-v3 [skipped]
     ↓ Featherless (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textGeneration [skipped]
     ↓ Replicate (10) [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-schnell [skipped]
       ↓ textToImage canonical - black-forest-labs/FLUX.1-dev [skipped]
       ↓ textToImage canonical - stabilityai/stable-diffusion-3.5-large-turbo [skipped]
       ↓ textToImage versioned - ByteDance/SDXL-Lightning [skipped]
       ↓ textToImage versioned - ByteDance/Hyper-SD [skipped]
       ↓ textToImage versioned - playgroundai/playground-v2.5-1024px-aesthetic [skipped]
       ↓ textToImage versioned - stabilityai/stable-diffusion-xl-base-1.0 [skipped]
       ↓ textToSpeech versioned [skipped]
       ↓ textToSpeech OuteTTS -  usually Cold [skipped]
       ↓ textToSpeech Kokoro [skipped]
     ↓ SambaNova (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ featureExtraction [skipped]
     ↓ Together (4) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Nebius (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ 3rd party providers (1) [skipped]
       ↓ chatCompletion - fails with unsupported model [skipped]
     ↓ Fireworks (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Hyperbolic (4) [skipped]
       ↓ chatCompletion - hyperbolic [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
       ↓ textGeneration [skipped]
     ↓ Novita (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Black Forest Labs (2) [skipped]
       ↓ textToImage [skipped]
       ↓ textToImage URL [skipped]
     ↓ Cohere (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Cerebras (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ↓ Nscale (3) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
       ↓ textToImage [skipped]
     ↓ Groq (2) [skipped]
       ↓ chatCompletion [skipped]
       ↓ chatCompletion stream [skipped]
     ✓ CentML (4) 4225ms
       ✓ chat completions (4) 4225ms
         ✓ basic chat completion 1047ms
         ✓ chat completion with multiple messages 970ms
         ✓ chat completion with parameters 1555ms
         ✓ chat completion stream 653ms

 Test Files  1 passed (1)
      Tests  4 passed | 96 skipped (100)
   Start at  01:20:35
   Duration  4.72s (transform 254ms, setup 9ms, collect 297ms, tests 4.23s, environment 0ms, prepare 54ms)

Thanks for reviewing! 🙏

@V2arK V2arK marked this pull request as draft April 28, 2025 17:09
@julien-c julien-c changed the title Add CentML as an inference provide Add CentML as an inference provider Apr 28, 2025
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@V2arK V2arK marked this pull request as ready for review April 29, 2025 07:24
Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you @V2arK for the contribution!
I tested the PR (using the free credits you offer for new users) and it works as expected ✅

@V2arK
Copy link
Author

V2arK commented May 4, 2025

Is there anything else I need to do to get it merged?

@V2arK V2arK requested a review from julien-c May 5, 2025 20:03
@SBrandeis SBrandeis added the inference-providers integration of a new or existing Inference Provider label May 23, 2025
@SBrandeis SBrandeis self-assigned this May 23, 2025
@V2arK V2arK closed this Jul 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inference-providers integration of a new or existing Inference Provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants