Skip to content

Conversation

@guicho271828
Copy link
Contributor

@guicho271828 guicho271828 commented Sep 10, 2025

Added a missing huggingface tools test that works only on some models.
The test file is separated because I had to use mistral 7B to make it work.
It is relatively small, but is not as small as 2B models which is used in other tests.
If there are other smaller models that work, let me know.

It also adds a add_generation_prompt argument to apply_chat_template which is definitely wrong without it.

@mergify
Copy link

mergify bot commented Sep 10, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?:

@guicho271828
Copy link
Contributor Author

Comment on the tool calling feature:

Tool calling feature is limited to only some models whose tokenizer supports a specialized chat template.
Does it still make sense now to keep using such a chat template? With enough prompts and structured decoding a good model would understand the task and would generate a similar response. We just need to do it manually for the user inside generate_from_context using structured decoding.

@guicho271828 guicho271828 force-pushed the masa/tools-huggingface-test branch from 6066a01 to 6e5d100 Compare September 19, 2025 02:40
@guicho271828 guicho271828 force-pushed the masa/tools-huggingface-test branch from 6e5d100 to 77715c8 Compare September 30, 2025 17:12
Copy link
Contributor

@jakelorocco jakelorocco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jakelorocco jakelorocco merged commit a4b6d27 into generative-computing:main Oct 1, 2025
4 checks passed
@guicho271828 guicho271828 deleted the masa/tools-huggingface-test branch October 1, 2025 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants