Update pull.yml to test snapshot saving and loading

[mikekgfb]

test snapshot saving and loading

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1486

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5d098bd with merge base 083fdaf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[mikekgfb]

@jerryzh168 @Jack-Khuu can you please have a look what happens with reloading of the Int4 quantized Linear class from torchao?

https://hud.pytorch.org/pr/pytorch/torchchat/1486#36825796920 shows this:

2025-02-07T02:27:59.4429304Z Traceback (most recent call last):
2025-02-07T02:27:59.4429938Z   File "/pytorch/torchchat/torchchat/cli/builder.py", line 653, in _initialize_model
2025-02-07T02:27:59.4430764Z     model = torch.load(builder_args.snapshot_path, weights_only=False)
2025-02-07T02:28:01.1584442Z Traceback (most recent call last):
2025-02-07T02:28:01.1585203Z   File "/home/ec2-user/actions-runner/_work/torchchat/torchchat/test-infra/.github/scripts/run_with_env_secrets.py", line 102, in <module>
2025-02-07T02:28:01.1585907Z             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-02-07T02:28:01.1586450Z   File "/opt/conda/lib/python3.11/site-packages/torch/serialization.py", line 1478, in load
2025-02-07T02:28:01.1586943Z     return _load(
2025-02-07T02:28:01.1587161Z            ^^^^^^
2025-02-07T02:28:01.1587709Z   File "/opt/conda/lib/python3.11/site-packages/torch/serialization.py", line 1971, in _load
2025-02-07T02:28:01.1588217Z     result = unpickler.load()
2025-02-07T02:28:01.1588484Z              ^^^^^^^^^^^^^^^^
2025-02-07T02:28:01.1588987Z   File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1938, in __getattr__
2025-02-07T02:28:01.1589534Z     raise AttributeError(
2025-02-07T02:28:01.1589913Z AttributeError: 'Linear' object has no attribute '_linear_extra_repr'

pull / test-gpu-aoti-bfloat16 (cuda, stories15M) / linux-job (gh)
AttributeError: 'Linear' object has no attribute '_linear_extra_repr'

[Jack-Khuu]

Thanks for the find, it's using cuda so it should be using the new subclass APIs too hmmm

torchchat/torchchat/utils/quantize.py

Lines 114 to 117 in 53a1004

	if (device == "cuda" or device == "xpu") and quantizer == "linear:int4":
	quantize_(model, int4_weight_only(q_kwargs["groupsize"]))
	if not support_tensor_subclass:
	unwrap_tensor_subclass(model)

[mikekgfb]

@Jack-Khuu updated to run at a minimum the CPU test correctly. Recommend we land that, then put a new PR to enable CUDA and you can assign to the right stakeholder working with torchao?

[Jack-Khuu]

I was just going to recommend that

Sounds like a plan, thanks for following up

[jerryzh168]

thanks, will follow up later, created an issue to track here: pytorch/ao#1727