Skip to content

Commit 21a33fd

Browse files
Add default int4 config for inceptionai/jais-13b (#1519)
1 parent bdedce9 commit 21a33fd

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

optimum/intel/openvino/configuration.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,15 @@ class OVQuantizationMethod(str, Enum):
350350
"sym": False,
351351
"group_size": -1,
352352
},
353+
"inceptionai/jais-13b": {
354+
"bits": 4,
355+
"sym": False,
356+
"group_size": 128,
357+
"ratio": 1.0,
358+
"advanced_parameters": nncf.AdvancedCompressionParameters(
359+
group_size_fallback_mode=nncf.GroupSizeFallbackMode.ADJUST,
360+
),
361+
},
353362
}
354363

355364
# Add configs for model id aliases

0 commit comments

Comments
 (0)