Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Update pull.yml to test snapshot saving and loading #1486

Merged
merged 6 commits into from
Feb 18, 2025
Merged

Update pull.yml to test snapshot saving and loading #1486

merged 6 commits into from
Feb 18, 2025

Conversation

@mikekgfb
Copy link
Contributor

test snapshot saving and loading

@pytorch-bot
Copy link

pytorch-bot bot commented Jan 31, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1486

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5d098bd with merge base 083fdaf (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jan 31, 2025
@mikekgfb
Update pull.yml
30a0d31 
Fixed typos.
@mikekgfb
Update pull.yml
2afca1e 
cuda-32.json because somebody would rather fail a job than accept a partil group
@mikekgfb
Merge branch 'pytorch:main' into patch-48
cb24aec
@mikekgfb
Copy link
Contributor Author

mikekgfb commented Feb 8, 2025

@jerryzh168 @Jack-Khuu can you please have a look what happens with reloading of the Int4 quantized Linear class from torchao?

https://hud.pytorch.org/pr/pytorch/torchchat/1486#36825796920 shows this:

2025-02-07T02:27:59.4429304Z Traceback (most recent call last):
2025-02-07T02:27:59.4429938Z   File "/pytorch/torchchat/torchchat/cli/builder.py", line 653, in _initialize_model
2025-02-07T02:27:59.4430764Z     model = torch.load(builder_args.snapshot_path, weights_only=False)
2025-02-07T02:28:01.1584442Z Traceback (most recent call last):
2025-02-07T02:28:01.1585203Z   File "/home/ec2-user/actions-runner/_work/torchchat/torchchat/test-infra/.github/scripts/run_with_env_secrets.py", line 102, in <module>
2025-02-07T02:28:01.1585907Z             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-02-07T02:28:01.1586450Z   File "/opt/conda/lib/python3.11/site-packages/torch/serialization.py", line 1478, in load
2025-02-07T02:28:01.1586943Z     return _load(
2025-02-07T02:28:01.1587161Z            ^^^^^^
2025-02-07T02:28:01.1587709Z   File "/opt/conda/lib/python3.11/site-packages/torch/serialization.py", line 1971, in _load
2025-02-07T02:28:01.1588217Z     result = unpickler.load()
2025-02-07T02:28:01.1588484Z              ^^^^^^^^^^^^^^^^
2025-02-07T02:28:01.1588987Z   File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1938, in __getattr__
2025-02-07T02:28:01.1589534Z     raise AttributeError(
2025-02-07T02:28:01.1589913Z AttributeError: 'Linear' object has no attribute '_linear_extra_repr'

pull / test-gpu-aoti-bfloat16 (cuda, stories15M) / linux-job (gh)
AttributeError: 'Linear' object has no attribute '_linear_extra_repr'

@mikekgfb
Update pull.yml
e733606 
Remove fp16 and fp32 int4 quantized models for now.  
@jerryzh168 Not sure why these dtypes are not compatible with int4 quantization?
@Jack-Khuu
Copy link
Contributor

Thanks for the find, it's using cuda so it should be using the new subclass APIs too hmmm

torchchat/torchchat/utils/quantize.py

Lines 114 to 117 in 53a1004

if (device == "cuda" or device == "xpu") and quantizer == "linear:int4":
quantize_(model, int4_weight_only(q_kwargs["groupsize"]))
if not support_tensor_subclass:
unwrap_tensor_subclass(model)

@mikekgfb
Update pull.yml
5d098bd 
add DEVICE specification for snapshot and use device cpu
@mikekgfb
Copy link
Contributor Author

@Jack-Khuu updated to run at a minimum the CPU test correctly. Recommend we land that, then put a new PR to enable CUDA and you can assign to the right stakeholder working with torchao?

@Jack-Khuu
Copy link
Contributor

I was just going to recommend that

Sounds like a plan, thanks for following up

@Jack-Khuu Jack-Khuu merged commit 384a728 into pytorch:main Feb 18, 2025
70 checks passed
@jerryzh168 jerryzh168 mentioned this pull request Feb 18, 2025
Open
@jerryzh168
Copy link
Contributor

thanks, will follow up later, created an issue to track here: pytorch/ao#1727

@Jack-Khuu Jack-Khuu mentioned this pull request Feb 21, 2025
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@Jack-Khuu Jack-Khuu Jack-Khuu approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mikekgfb @Jack-Khuu @jerryzh168 @facebook-github-bot