Support diffusion model AIDC-AI/Ovis-Image-7B quantization#1616
Support diffusion model AIDC-AI/Ovis-Image-7B quantization#1616lvliang-intel wants to merge 2 commits intomainfrom
Conversation
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
…upport_ovis_image
There was a problem hiding this comment.
Pull request overview
Adds explicit support for quantizing the diffusion model AIDC-AI/Ovis-Image-7B by extending the diffusion compressor’s known block-output mapping and updating the diffusion documentation to reflect validation status.
Changes:
- Register Ovis-Image transformer block class names in
output_configsso block outputs can be interpreted correctly during calibration/quantization. - Update the diffusion compressor warning to indicate validation on Flux and Ovis-Image models.
- Update the diffusion support matrix docs to include AIDC-AI/Ovis-Image-7B.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| auto_round/compressors/diffusion/compressor.py | Adds Ovis-Image block output mappings and updates validation warning text. |
| auto_round/compressors/diffusion/README.md | Updates diffusion support matrix to include Ovis-Image-7B. |
| output_configs = { | ||
| "FluxTransformerBlock": ["encoder_hidden_states", "hidden_states"], | ||
| "FluxSingleTransformerBlock": ["encoder_hidden_states", "hidden_states"], | ||
| "OvisImageTransformerBlock": ["encoder_hidden_states", "hidden_states"], | ||
| "OvisImageSingleTransformerBlock": ["encoder_hidden_states", "hidden_states"], | ||
| } |
There was a problem hiding this comment.
New Ovis-Image support is introduced by adding OvisImage* block class names to output_configs, but the diffusion test suite currently only exercises FLUX. Consider adding a regression/smoke test that runs AutoRoundDiffusion on an Ovis-Image pipeline (or a lightweight stub that uses these block class names/outputs) to ensure the new mapping stays correct and prevents runtime assertion failures in _get_block_outputs.
|
Quantize Script Run inference with vLLM-Omni(with patch for ovis image) CUDA_VISIBLE_DEVICES=0 run_ovis_image.py --model /mnt/disk4/lvl/Ovis-Image-7B/ --prompt "A watercolor fox reading a book" --guidance-scale 5.0 --num-inference-steps 50 CUDA_VISIBLE_DEVICES=0 python run_ovis_image.py --model ./tmp_ovis_image_w4a16/ --prompt "A watercolor fox reading a book" --guidance-scale 5.0 --num-inference-steps 50 |
Description
Support diffusion model AIDC-AI/Ovis-Image-7B quantization
Type of Change
Related Issues
#1609
Checklist Before Submitting