Skip to content

handle moonshot#75

Open
regan-huff wants to merge 3 commits intomainfrom
reganh/handle-moonshot
Open

handle moonshot#75
regan-huff wants to merge 3 commits intomainfrom
reganh/handle-moonshot

Conversation

@regan-huff
Copy link
Copy Markdown
Contributor

I am attempting to score an external submission for AstaBench:
https://huggingface.co/datasets/allenai/asta-bench-submissions/tree/main/1.0.0/test/EvoScientist_EvoScientist_Coder_2026-03-19_16-22-34

The solver args show this provider openrouter/moonshotai/kimi-k2.5, which comes through in the inspect model usage objects like this:

{
model: "moonshotai/kimi-k2.5-0127",
usage: {
input_tokens: 7467,
output_tokens: 829,
total_tokens: 8296,
input_tokens_cache_write: null,
input_tokens_cache_read: 6144,
reasoning_tokens: 284
}

Moonshot is supported as an inference provider in litellm and some cost objects have been added to https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json

However, these are not available in the version of litellm which we have chosen to pin, 1.75.8 (and in fact may not yet be in a released version). This PR adds pricing in local_cost to handle this provider/model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant