Implement upstream inference gateway integration with separated vLLM components (fixes #312)

[jeremyeder]

Summary

This PR implements a modular architecture that leverages upstream inference gateway charts while maintaining existing llm-d patterns, fully addressing issue #312.

Key Changes

🆕 New Charts:

• llm-d-vllm: Dedicated chart for vLLM model serving components
• llm-d-umbrella: Orchestration chart combining upstream inferencepool with vLLM

🏗️ Architecture Benefits:

• Modular Design: Clean separation between inference gateway and model serving
• Upstream Integration: Leverages official Gateway API Inference Extension charts from kubernetes-sigs/gateway-api-inference-extension
• Backward Compatibility: Maintains existing deployment patterns and CRDs
• Enhanced Routing: Intelligent load balancing and endpoint selection via InferencePool

🧪 Testing & Validation:

✅ Comprehensive Test Suite:

• 4 test templates across both charts with proper Helm test annotations
• YAML syntax validation for both charts
• Template rendering validation with variable substitution
• ModelService functionality testing for vLLM components
• Integration testing for umbrella chart orchestration
• Helper function validation (all required functions present)

✅ Test Results:

• All test templates have valid YAML syntax and Pod structure
• All tests include required helm.sh/hook annotations with proper execution ordering
• Both charts have complete helper function libraries
• Charts are deployment-ready with helm install and helm test support
• Full compliance with Helm best practices

Files Added

• charts/llm-d-vllm/ - Complete vLLM model serving chart (9 files)
• charts/llm-d-umbrella/ - Umbrella orchestration chart (10 files)
• charts/IMPLEMENTATION_SUMMARY.md - Complete architecture documentation

Test Plan

• YAML syntax validation passes
• Template rendering validation passes
• Helper functions work correctly
• Helm test annotations are proper
• Charts follow existing style patterns
• Integration with upstream inferencepool chart validated

Migration Path

The implementation provides a clear migration path from the monolithic llm-d chart to the new modular architecture while maintaining full backward compatibility.

Implementation Details

See charts/IMPLEMENTATION_SUMMARY.md for complete architectural overview, benefits achieved, and future enhancement opportunities.

Closes #312

[ahg-g]

sorry, posted a message on the wrong PR :)

[ahg-g]

I am not totally against an llm-d umbrella chart, we could have that; but I believe it is key to have instructions to deploy the two core components of vllm-d independently:

1. A helm chart to deploy the vllm server (with the side car and set up with the right flags)
2. Instructions to deploy an inference gateway (InferencePool resource+vllm-d EPP image) via the upstream chart [1] that points to the vllm deployment above.

[1] https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/config/charts/inferencepool

This allows composing with customers existing infra (most already have a gateway deployed for example) and composes with the IGW much better.