Skip to content

Update AEP-8026: Allow per-VPA component configuration parameters #8505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Update AEP-8026: Allow per-VPA component configuration parameters #8505

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
35c8fbe
update AEP
omerap12 Sep 6, 2025
a7bf2c8
Revert to old configurations
omerap12 Sep 6, 2025
6a99eb4
fixed typo
omerap12 Sep 6, 2025
bad2138
Add more info
omerap12 Sep 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 102 additions & 24 deletions ...ical-pod-autoscaler/enhancements/8026-per-vpa-component-configuration/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@
- [Container Policy Parameters](#container-policy-parameters)
- [Update Policy Parameters](#update-policy-parameters)
- [Design Details](#design-details)
- [Configuration Level Considerations](#configuration-level-considerations)
- [Parameter Level Analysis](#parameter-level-analysis)
- [API Changes](#api-changes)
- [OOM Parameter Consolidation](#oom-parameter-consolidation)
- [Parameter Coexistence and Global Configuration](#parameter-coexistence-and-global-configuration)
- [Phase 1 (Current Proposal)](#phase-1-current-proposal)
- [Future Extensions](#future-extensions)
- [Feature Enablement and Rollback](#feature-enablement-and-rollback)
Expand Down Expand Up @@ -44,7 +48,6 @@ Currently, supporting these different needs requires running multiple VPA compon
- Support different optimization strategies for different workloads in the same cluster
- Maintain backward compatibility with existing global configuration
- Initially support the following parameters:
- oomBumpUpRatio
- oomMinBumpUp
- memoryAggregationInterval
- memoryAggregationIntervalCount
Expand All @@ -64,41 +67,34 @@ The configuration will be split into two sections: container-specific recommenda
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: oom-test-vpa
name: simple-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: oom-test
name: my-app
updatePolicy:
updateMode: Auto
evictAfterOOMThreshold: "5m"
resourcePolicy:
containerPolicies:
- containerName: "*"
oomBumpUpRatio: "1.5"
oomMinBumpUp: 104857600
oomBumpUp: "1.5"
memoryAggregationInterval: "12h"
memoryAggregationIntervalCount: 5
```

### Parameter Descriptions

#### Container Policy Parameters
#### Container Policy Parameters
* `oomBumpUpRatio` (Quantity):
- Multiplier applied to memory recommendations after OOM events
- Represented as a Quantity (e.g., "1.5")
- Must be greater than or equal to 1
- Setting to 1 effectively disables the OOM ratio-based increase
- Controls how aggressively memory is increased after container crashes

* `oomMinBumpUp` (bytes):
- Minimum absolute memory increase after OOM events
- Setting to 0 effectively disables the OOM minimum increase
- When both `oomBumpUpRatio` = 1 and `oomMinBumpUp` = 0, OOM-based memory increases are completely disabled
- Ensures meaningful increases even for small containers

* `oomBumpUp` (Quantity):
- Controls memory increase after OOM events
- Can be specified as either:
- A ratio (e.g., "1.5" for 50% increase)
- An absolute value (e.g., "100Mi" for fixed increase)
- When specified, overrides global OOM configuration
- Allows fine-tuning of OOM behavior per container

* `memoryAggregationInterval` (duration):
- Time window for aggregating memory usage data
- Affects how quickly VPA responds to memory usage changes
Expand All @@ -117,13 +113,94 @@ Each parameter can be configured independently, falling back to global defaults

## Design Details

### Configuration Level Considerations

When designing the configuration parameters, we analyzed each parameter to determine the most appropriate configuration level:

#### Parameter Level Analysis

1. **OOM-Related Parameters (`oomBumpUp`)**
- **Recommended Level**: Container-level
- **Rationale**:
- Different containers in the same pod may have different memory requirements and OOM patterns
- Memory recommendations in VPA are already handled at container level
- Consistent with how VPA handles other resource-related configurations
- While `oomBumpUp` is container-level, VPA-wide configuration can still be achieved using the wildcard container name "*"

2. **Memory Aggregation Parameters (`memoryAggregationInterval`, `memoryAggregationIntervalCount`)**
- **Recommended Level**: Container-level
- **Rationale**:
- Different containers may have different memory usage patterns
- Some containers might need faster reaction times to memory changes
- Consistent with existing VPA container-level resource policies
- While `memoryAggregationInterval` and `memoryAggregationIntervalCount` are container-level, VPA-wide configuration can still be achieved using the wildcard container name "*"

3. **Eviction Parameters (`evictAfterOOMThreshold`)**
- **Recommended Level**: VPA-level
- **Rationale**:
- Eviction decisions affect the entire pod
- Pod-level operation that shouldn't vary by container
- Simpler operational model for pod lifecycle management
- Consistent with how Kubernetes handles pod evictions


### API Changes

### OOM Parameter Consolidation

In this proposal, we are consolidating `oomBumpUpRatio` and `oomMinBumpUp` into a single `oomBumpUp` parameter for several reasons:
1. **Simplified Configuration**:
- Instead of managing two separate parameters that affect the same behavior, users can specify their intent with a single parameter
- Makes configuration more intuitive and less error-prone

2. **Flexible Expression**:
- `oomBumpUp` can express both ratio-based and absolute increases:
- Ratio format: "1.5" (50% increase)
- Absolute format: "100Mi" (100 MiB increase)
- This flexibility eliminates the need for separate parameters while maintaining all functionality

3. **Consistent with Kubernetes Patterns**:
- Uses the standard Kubernetes Quantity type
- Similar to how other Kubernetes components handle resource specifications

4. **Backward Compatibility**:
- The new parameter can express all use cases covered by the previous parameters
- Migration path is straightforward:
- For ratio-based increases: Use the same value in `oomBumpUp`
- For minimum increases: Convert to absolute quantity in `oomBumpUp`

### Parameter Coexistence and Global Configuration

The VPA recommender maintains global configuration values for OOM-related parameters. The new `oomBumpUp` parameter interacts with these global configurations as follows:

1. **Priority Order**:
- If `oomBumpUp` is specified in the VPA object, it takes precedence over any global configuration
- If `oomBumpUp` is not specified, the global configuration values from the recommender will be used
- This allows for a smooth transition and maintains backward compatibility

2. **Validation**:
- The admission controller can only validate the syntax and range of `oomBumpUp` when specified
- Cannot validate against global configuration as these values live in the recommender
- Values must be either:
- A ratio >= 1 (e.g., "1.5")
- An absolute memory value > 0 (e.g., "100Mi")

3. **Example Scenarios**:
```yaml
# Scenario 1: oomBumpUp specified - overrides global config
containerPolicies:
- containerName: "*"
oomBumpUp: "1.5"

# Scenario 2: No oomBumpUp - uses global config from recommender
containerPolicies:
- containerName: "*"


#### Phase 1 (Current Proposal)

Extend `ContainerResourcePolicy` with:
* `oomBumpUpRatio`
* `oomMinBumpUp`
* `oomBumpUp`
* `memoryAggregationInterval`
* `memoryAggregationIntervalCount`

Expand Down Expand Up @@ -175,15 +252,15 @@ The `PerVPAConfig` feature requires VPA version 1.5.0 or higher. The feature is
### Validation via CEL and Testing

Initial validation rules (CEL):
* `oomMinBumpUp` >= 0
* `oomBumpUP` >= 1
* `memoryAggregationInterval` > 0
* `evictAfterOOMThreshold` > 0
* `memoryAggregationIntervalCount` > 0
* `evictAfterOOMThreshold` > 0

Validation via Admission Controller:
Some components cann't be validated using Common Expression Language (CEL). This validation is performed within the admission controller.

* `oomBumpUpRatio` – Using Kubernetes Quantity type for validation. The value must be greater than or equal to 1.
* `oomBumpUP` – Using Kubernetes Quantity type for validation. The value must be greater than or equal to 1.

Additional validation rules will be added as new parameters are introduced.
E2E tests will be included to verify:
Expand All @@ -201,6 +278,7 @@ E2E tests will be included to verify:
## Implementation History

- 2025-04-12: Initial proposal
- 2025-09-06: Consolidated `oomBumpUpRatio` and `oomMinBumpUp` into single `oomBumpUp` parameter
- Future: Additional parameters will be added based on user feedback and requirements

## Future Work
Expand Down
Loading