Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Conversation

@Jack-Khuu
Copy link
Contributor

As titled, torchchat expects logits as (1, 1, vocab size), but depending on the exporting logic the batch dim is not provided (for example with pte generated from the ET repo

This just adds a simple check and layer if needed.

Tested by

python3 torchchat.py generate llama3.1 --pte-path <pte from ET repo export>
python3 torchchat.py generate llama3.1 --pte-path <pte from TC repo export>

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 23, 2024
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 23, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1177

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 0d700de with merge base 971ed93 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@Jack-Khuu Jack-Khuu changed the title Fix support for ET repo generated pte by adding batch dim Fix support for ET repo generated pte by optionally adding batch dim Sep 23, 2024
@Jack-Khuu Jack-Khuu merged commit 72d2d20 into main Sep 23, 2024
51 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants