Skip to content

Update PD disaggregation example #130

@elieserr

Description

@elieserr

On deploying the PD Disaggregation example I found multiple issues:

  • deprecated objects being created like described in Deprecate InferenceModel #121
  • current Inference scheduler for PD disaggregation is using ghcr.io/llm-d/llm-d-inference-scheduler:v0.2.1 and the latest version available is ghcr.io/llm-d/llm-d-inference-scheduler:v0.3.1
  • the config file for the inferencescheduler in the example uses the default config and not the default-pd-config.yaml
  • i found an issue where the vLLM baked in the "ghcr.io/llm-d/llm-d:v0.2.0" image does not have available the proper NixlConnector

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions