Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Conversation

@Jack-Khuu
Copy link
Contributor

@Jack-Khuu Jack-Khuu commented Sep 23, 2024

Currently, when loading a .pte file is provided, the base model is loaded in order to obtain the model.config: ModelArgs. This is overkill (and is not resolved in this PR) since the rest of the base model is discarded.

This PR avoids this wasteful loading in one condition: a manual params_path is provided, which can be used to construct ModelArgs


Tested by passing in a pte w/ and without a manual param

python torchchat.py generate llama3.1 --pte-path model.pte
- Load

python torchchat.py generate llama3.1 --pte-path model.pte --params-path config.json
- No Load

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 23, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1178

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit fe90ac7 with merge base 72d2d20 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 23, 2024
@Jack-Khuu Jack-Khuu merged commit b037b71 into main Sep 23, 2024
51 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants