Skip to content

Fix AutoScheme low memory flag propagation from CLI #1596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chensuyue merged 4 commits into from
Mar 27, 2026
Merged

Fix AutoScheme low memory flag propagation from CLI #1596

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6d70a38
Fix AutoScheme low memory flag propagation from CLI
lvliang-intel Mar 23, 2026
e52f970
Merge branch 'main' of https://github.com/intel/auto-round into lvl/f…
lvliang-intel Mar 23, 2026
555d2a6
Update auto_round/__main__.py
lvliang-intel Mar 23, 2026
efa033f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions auto_round/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,10 +140,18 @@ def __init__(self, *args, **kwargs):
"Useful when working with large models that don't fit in GPU memory.",
)
basic.add_argument(
"--low_cpu_mem_usage", action="store_true", help="Deprecated, Lower CPU memory mode. Defaults to False."
"--low_cpu_mem_usage",
action="store_true",
help=(
"Deprecated: low CPU memory mode is enabled by default. "
"This flag is kept only for backward compatibility and has no effect "
"beyond explicitly re-enabling the default behavior."
),
)
basic.add_argument(
"--disable_low_cpu_mem_usage", action="store_true", help="disable lower CPU memory mode. Defaults to False."
"--disable_low_cpu_mem_usage",
action="store_true",
help=("Disable low CPU memory mode. " "Use this flag to turn off the default low CPU memory behavior."),
)
basic.add_argument(
"--format",
Expand Down Expand Up @@ -676,6 +684,10 @@ def tune(args):
layer_config = parse_layer_config_arg(args.layer_config)
args.layer_config = layer_config

low_cpu_mem_usage = True
if args.disable_low_cpu_mem_usage:
low_cpu_mem_usage = False

if args.avg_bits is not None:
if args.options is None:
raise ValueError("please set --options for auto scheme")
Expand All @@ -684,10 +696,9 @@ def tune(args):
avg_bits=args.avg_bits,
shared_layers=args.shared_layers,
ignore_scale_zp_bits=args.ignore_scale_zp_bits,
low_gpu_mem_usage=args.low_gpu_mem_usage,
low_cpu_mem_usage=low_cpu_mem_usage,
)
low_cpu_mem_usage = True
if args.disable_low_cpu_mem_usage:
low_cpu_mem_usage = False

autoround: BaseCompressor = AutoRound(
model=model_name,
Expand Down
Loading