[Serve LLM] Fix OpenAiIngress scale-to-zero when all models set min_replicas=0

[thjung123]

Description

Summary

• Fix an issue where OpenAiIngress does not scale to zero when all LLM models
  are configured with min_replicas=0
• Update get_deployment_options() to inspect llm_configs and propagate
  min_replicas=0 to the ingress when appropriate
• Applies to all ingress types inheriting from OpenAiIngress

Problem

OpenAiIngress.get_deployment_options() previously returned
DEFAULT_INGRESS_OPTIONS as-is, ignoring the provided llm_configs.
Because DEFAULT_INGRESS_OPTIONS does not include a min_replicas field,
Ray Serve’s AutoscalingConfig defaults to min_replicas=1, which prevents
the Serve application from fully scaling to zero even when all model
deployments have scaled down.

Solution

• Determine whether all LLM deployments explicitly configure
  autoscaling_config.min_replicas=0
• If so, propagate min_replicas=0 to the ingress deployment options
• Otherwise, preserve the existing default behavior
• Continue to give precedence to user-provided ingress_deployment_config

Fixes #60664

[gemini-code-assist]

Code Review

This pull request addresses an issue where OpenAiIngress would not scale to zero even if all underlying LLM models were configured with min_replicas=0. The fix correctly inspects the llm_configs and propagates min_replicas=0 to the ingress deployment options when appropriate. The changes are well-implemented and include a comprehensive set of tests covering various scenarios. My review includes a couple of suggestions to improve code conciseness and maintainability.

[gemini-code-assist]

For creating a mutable copy of DEFAULT_INGRESS_OPTIONS, using copy.deepcopy() is more idiomatic and robust against future changes to the structure of DEFAULT_INGRESS_OPTIONS. This avoids potential bugs if the dictionary becomes more deeply nested.

You'll need to add import copy at the top of the file.

        options = copy.deepcopy(DEFAULT_INGRESS_OPTIONS)

[thjung123]

I intentionally used a shallow dict comprehension here since
DEFAULT_INGRESS_OPTIONS is only two levels deep and all nested values are plain dicts.

Using copy.deepcopy would also work, but feels a bit overkill for the current structure.
Happy to switch if maintainers prefer it.

[kouroshHakha]

yes let's do deepcopy to avoid future bugs that might get introduced if overlooked. The cost is minimal since these dicts don't get that big and are not on the data path.

[thjung123]

Agreed, thanks! I've updated this to use copy.deepcopy() as suggested.

[kouroshHakha]

Just the nit. LGTM otherwise. Thanks.

[kouroshHakha]

yes let's do deepcopy to avoid future bugs that might get introduced if overlooked. The cost is minimal since these dicts don't get that big and are not on the data path.