Skip to content

[Feature]: Quantize/save/evaluate the AIDC-AI/Ovis-Image-7B in w4a16 format #1609

@lvliang-intel

Description

@lvliang-intel

Feature Description

Quantize/save/evaluate the AIDC-AI/Ovis-Image-7B in w4a16 format

Motivation and Use Case

Model: https://huggingface.co/AIDC-AI/Ovis-Image-7B
Target dtypes: w4a16

Save the quantized model for vllm-omni.

Alternatives Considered

No response

Definition of Done

No response

Additional Context

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions