Skip to content

[Feature]: Load/Evaluate W4A16 Qwen/Qwen-Image on vllm-omni #1494

@yiliu30

Description

@yiliu30

Feature Description

As the tile

Motivation and Use Case

Model: https://huggingface.co/Qwen/Qwen-Image
Target dtypes: w4a16

Alternatives Considered

No response

Definition of Done

  • Load the quantized model
  • Align the accuray w/ the qdq mode.

Additional Context

No response

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions