Skip to content

Conversation

@Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Jun 4, 2025

Fix after #1514.

Now that we use a placeholder for access token to load from env, there is no direct way to explictly generatea snippet for either a "direct request" or a "routed request" (determined here using accessToken.startsWith("hf_")). This PR adds a directRequest?: boolean; option to the parameters which solves this problem.

Will require a follow-up PR in moon-landing.

cc @SBrandeis who found out the root cause

expected behavior

display routed request by default in https://huggingface.co/deepseek-ai/DeepSeek-R1-0528?inference_api=true&inference_provider=fireworks-ai&language=sh

image

@Wauplin
Copy link
Contributor Author

Wauplin commented Jun 4, 2025

Thanks!

@Wauplin Wauplin merged commit c3bf626 into main Jun 4, 2025
5 checks passed
@Wauplin Wauplin deleted the fix-inference-snippets-routed-requests branch June 4, 2025 16:35
@Wauplin Wauplin changed the title [Inference Snippet] Add adirectRequest option (false by default) [Inference Snippet] Add a directRequest option (false by default) Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants