Skip to content

Conversation

@mfuntowicz
Copy link
Member

This PR attempts to track usage stats for endpoints deployed with hfendpoints library

@Wauplin
Copy link
Contributor

Wauplin commented May 15, 2025

@mfuntowicz Hmm, it feels weird to me to define hfendpoints as a library on the Hub. Am I correct to say that this is not really a library but more an internal tool we use to maintain images to deploy?

Am I right to say that when someone deploys an Inference Endpoint from e.g. https://huggingface.co/hfendpoints-images/embeddings-sentence-transformers-cpu, it downloads a snapshot of the repo and then execute it on the server (using the Dockerfile/scripts)? And that you want to track the downloads for handler.py to get an idea of the usage of each image?

If that's the case, I would recommend to add an empty config.json file in the repo and make sure it is downloaded when launching the Inference Endpoint. It's a bit artificial but at least if would not require to add hfendpoints as an ML library. Does this solution work for you ?

(for hfendpoints repos with already a config.json, it should be fine)

docsUrl: "https://hezarai.github.io/hezar",
countDownloads: `path:"model_config.yaml" OR path:"embedding/embedding_config.yaml"`,
},
hfendpoints: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the images should show up here: https://huggingface.co/models?other=hfendpoints (whisper + sglang one too)

prettyLabel: "Hugging Face Endpoints",
repoName: "hfendpoints",
repoUrl: "https://github.com/huggingface/hfendpoints",
countDownloads: `path:"handler.py"`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tho I wonder if we can't just add a config.json in the root of the repo and get usage stats automatically? IIRC IE clones the entire repo so config.json would be too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jinx 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or just use another log for telemetry (we have other systems at the infra level)

@mfuntowicz
Copy link
Member Author

No big deal! I'm fine using an empty config.json - it was somewhat cleaner at first sight to me 😊.

@mfuntowicz mfuntowicz closed this May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants