Skip to content

generalize pd-profile-handler behavior for better usability and unified token-based configuration #519

@mayabar

Description

@mayabar

Description:
Extend the behavior of pd-profile-handler to make it more general and user-friendly, while integrating the data preparation feature.
Some related changes are in GIE (listed here for reference).

To avoid duplicated definitions of block size across plugins, use tokens as the standard unit instead of the number of blocks. All length-related parameters should be defined in tokens.

Goals

  • Ease of configuration
    • Define all relevant length parameters in tokens. For example, the blockSize parameter of prefix-cache-scorer should be defined in tokens.
  • Consistency across components
    • In PrefixCacheMatchInfo, use values calculated in tokens and rename totalBlocks to totalLength.
    • Data stored in cycleState and PrepareData should contain prefix length in tokens.
  • Data-driven prefix length
    • Use the prefix length per pod from data stored in PrepareRequestData.
  • Testing
    • Update all related unit and integration tests to reflect these changes.

Metadata

Metadata

Assignees

Labels

needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions