Feature Description
Add in support for the sampling_params variables truncation_side and truncate_prompt_tokens to the vllm, huggingface, litellm, and olmo_core providers to match the vllm_server implementation (#208).
Use Case
Currently, any setting of truncation_side or truncate_prompt_tokens for a non-vllm_server provider generates a warning. This limits providers that can be used for the safety judge, and any other length-limited model that is run
Feature Description
Add in support for the sampling_params variables truncation_side and truncate_prompt_tokens to the vllm, huggingface, litellm, and olmo_core providers to match the vllm_server implementation (#208).
Use Case
Currently, any setting of truncation_side or truncate_prompt_tokens for a non-vllm_server provider generates a warning. This limits providers that can be used for the safety judge, and any other length-limited model that is run