Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Conversation

@Jack-Khuu
Copy link
Contributor

2 mini fixes on #896

  • Enum logic typos (was not found when testing originally due to Python magic)
  • max_seq_len discrepancy (found when testing)
python3 torchchat.py export llama3.1 --quantize '{"precision": {"dtype":"bfloat16"}, "executor":{"accelerator":"cuda"}}' --output-aoti-package-path /tmp/model3.pt2

python3 torchchat.py generate llama3.1 --aoti-package-path /tmp/model3.pt2 --prompt "Once upon a time,"  --num-samples 3

Compared to

python3 torchchat.py export llama3.1 --quantize '{"precision": {"dtype":"bfloat16"}, "executor":{"accelerator":"cuda"}}' --output-dso-path /tmp/model3.so

python3 torchchat.py generate llama3.1 --dso-path /tmp/model3.so --prompt "Once upon a time,"  --num-samples 3

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 5, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1343

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit f35d5fc with merge base 4510ba0 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 5, 2024
@Jack-Khuu Jack-Khuu merged commit 54455a3 into main Nov 5, 2024
52 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants