Skip to content

Conversation

@psriramsnc
Copy link
Collaborator

@psriramsnc psriramsnc commented Oct 8, 2025

🚀 Add support for Chat Template params

✨ Summary

This PR adds support for chat_template_params in model configurations and updates the documentation accordingly.
The purpose is to enable dynamic parameter injection into chat templates, allowing more flexible control over model behavior (e.g., reasoning effort).


🛠️ Explain the features implemented:

  • ➕ Added chat_template_params field in model_configuration.md under Configuration Properties.
  • 🔄 Updated all model generation methods in custom_models.py to pass chat_template_params dynamically during request payload creation.
  • 📝 Extended get_chat_formatted_text() to accept and apply **chat_template_params.
  • 📚 Enhanced documentation to indicate that both parameters and chat_template_params can be overridden in graph YAML files.

⚡ Performance impact (if any):

  • Negligible.
  • No measurable performance degradation; parameters are applied only when specified.

🧪 How to Test the feature

Steps for reviewers to verify functionality:

  1. 📂 Navigate to your model configuration YAML file.
  2. ✍️ Add custom valid model chat template parameters under chat_template_params, for example:
    gpt-oss-120b:
      completions_api: true
      chat_template_params:
        reasoning_effort: high
  3. ✍️ Add chat_template_params in graph_config.yaml to override model configuration YAML file. for example:
    model:
      name: gpt-oss-120b
      chat_template_params:
        reasoning_effort: low

4.▶️ Run a graph YAML that uses this model.
5. 👀 Observe that the model request payloads include the provided chat template parameters and modify output behavior accordingly.


📸 Screenshots (if applicable)

N/A — documentation-only visual updates and backend enhancements.


✅ Checklist

  • Lint fixes and unit testing done
  • End to end task testing
  • Documentation updated

📝 Notes

This change improves extensibility for models using complex prompt templates.

Backward-compatible — existing configurations remain valid without modification.

@psriramsnc psriramsnc self-assigned this Oct 8, 2025
@psriramsnc psriramsnc changed the title Add support for Chat Template params [Enhancement] Add support for Chat Template params Oct 8, 2025
@psriramsnc psriramsnc added the enhancement New feature or request label Oct 8, 2025
@psriramsnc psriramsnc marked this pull request as ready for review October 15, 2025 10:14
@psriramsnc psriramsnc requested a review from a team as a code owner October 15, 2025 10:14
Copy link
Member

@zephyrzilla zephyrzilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Copy link
Collaborator

@amitsnow amitsnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vipul-mittal vipul-mittal enabled auto-merge (squash) October 15, 2025 13:04
@vipul-mittal vipul-mittal merged commit 76eae4b into main Oct 15, 2025
12 checks passed
@vipul-mittal vipul-mittal deleted the scratch/add_chat_template_params_support branch October 15, 2025 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants