Skip to content

Conversation

@Ankit-06679
Copy link

🎯 Problem

Users trying to export fine-tuned Gemma3 models to ONNX format encounter this error:

This prevents users from converting their Gemma3 models to ONNX format for deployment and optimization.

🔧 Solution

This PR adds native Gemma3 support to optimum by implementing the Gemma3OnnxConfig class and properly registering it in the tasks manager.

📝 Changes Made

✨ Core Implementation

Added Gemma3OnnxConfig class** in optimum/exporters/onnx/model_configs.py

  • Inherits from TextDecoderOnnxConfig (same pattern as Gemma2)
  • Uses GemmaDummyPastKeyValuesGenerator for compatibility
  • Supports text generation and classification tasks
  • Requires transformers >= 4.50.0 (when Gemma3 was introduced)

🔧 Supporting Changes

  • Updated optimum/exporters/onnx/utils.py** - Added "gemma3" to MODEL_TYPES_REQUIRING_POSITION_IDS
  • Updated test files** - Added Gemma3OnnxConfig imports and test model mappings
  • Updated supported architectures** - Added proper version checks for Gemma3 support

🧪 Testing & Verification

  • ✅ All syntax checks pass
  • ✅ Configuration properly registered in tasks manager
  • ✅ Follows existing Gemma/Gemma2 implementation patterns
  • ✅ No breaking changes to existing functionality
  • ✅ Comprehensive verification script confirms all changes

📋 Usage Example

After this change, users can export Gemma3 models seamlessly:

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM

# This will now work without errors! 🎉
model_id = "path/to/gemma3/model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForCausalLM.from_pretrained(model_id, export=True)
model.save_pretrained('output_onnx/')

- Add Gemma3OnnxConfig class with proper configuration
- Register gemma3 model type for text generation and classification tasks
- Add Gemma3 to supported architectures and test mappings
- Set minimum transformers version requirement to 4.50.0
- Follow same pattern as existing Gemma/Gemma2 implementations

Fixes: ValueError when exporting Gemma3 models to ONNX format
Resolves: 'gemma3 model, that is a custom or unsupported architecture' error
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@IlyasMoutawwakil
Copy link
Member

Thanks ! This was already done in #70
What would be interesting is adding support for gemma3 VLM variant.

@Ankit-06679
Copy link
Author

Thanks ! This was already done in #70 What would be interesting is adding support for gemma3 VLM variant.

Oh,I didn't notice that. I will try the gemma3 VLM variant if possible.

@IlyasMoutawwakil
Copy link
Member

yes please do, you can use #50 for reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants