[llm] TOKENIZER_ONLY download misses chat_template.jinja for S3-backed models

[pengw0048]

What happened + What you expected to happen

When using download_model_files(model_id="s3://...", download_model=NodeModelDownloadable.TOKENIZER_ONLY) with S3-backed models, chat_template.jinja is not downloaded. This causes ChatTemplateStage to fail with: ValueError: Cannot use apply_chat_template because this processor does not have a chat template.. This affects models that use the newer HuggingFace convention of external chat_template.jinja files (e.g. Qwen3.5) instead of the legacy chat_template.json format.

Root cause

In ray/llm/_internal/common/utils/cloud_utils.py, the S3 file filter for tokenizer_only=True is:

tokenizer_file_substrings = (
    ["tokenizer", "config.json"] if tokenizer_only else []
)

chat_template.jinja doesn't match either "tokenizer" or "config.json", so it's excluded. The legacy chat_template.json is included (matches "config.json" substring), which is why older models work.

Suggested fix

Add ".jinja" or "chat_template" to the substring list:

tokenizer_file_substrings = (
    ["tokenizer", "config.json", ".jinja"] if tokenizer_only else []
)

Versions / Dependencies

• Ray: 3.0.0.dev0 (nightly, commit f996fa0)
• transformers: 4.57.3
• Model: Qwen/Qwen3.5-35B-A3B (S3-backed)

Reproduction script

from ray.llm._internal.common.utils.download_utils import download_model_files, NodeModelDownloadable
import os, shutil

# Clear any existing cache first
cache_dir = os.path.expanduser('~/.cache/huggingface/hub')
os.makedirs(cache_dir, exist_ok=True)
for d in os.listdir(cache_dir):
    if 'Qwen3.5' in d:
        shutil.rmtree(os.path.join(cache_dir, d))
        print(f'Cleared {d}')

path = download_model_files(
    model_id='s3://.../Qwen3.5-35B-A3B',
    mirror_config=None,
    download_model=NodeModelDownloadable.TOKENIZER_ONLY,
    download_extra_files=False,
)
print('Downloaded to:', path)
print('Files:', sorted(os.listdir(path)))
print('chat_template.jinja present:', 'chat_template.jinja' in os.listdir(path))

Issue Severity

Medium: It is a significant difficulty but I can work around it.