Download fix #1
Open
Download fix #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…h#1337) * [AOTI] Remove the original model weights in Python deployment Summary: Fixes pytorch#1302. Because AOTI-compiled model contains a copy of model weights, we need to release the corresponding eager model weights in the Python deployment path. * Revert "[AOTI] Remove the original model weights in Python deployment" This reverts commit 962ec0d. * Refactor the code * Add setup_cache for aoti_package_path --------- Co-authored-by: Jack-Khuu <[email protected]>
…t don't need a model (pytorch#1349) These changes add a little complexity with the lazy and local imports, but they also greatly improve the CLI's response for --help, list, and where. Changes: * Move `import torch` into function bodies that need them * Use `importlib.metadata.version` to check the torch version rather than torch.__version__ * Switch from using torch.inference_mode as a decorator to using it as a context manager. * I also removed it from convert_hf_checkpoint_to_tune since that does not use torch at all * In build_utils, wrap the dtype values in lambdas so they're lazily fetched. pytorch#1347 Branch: FasterCli-1347 Signed-off-by: Gabe Goodhart <[email protected]>
rename name of. slack channel for our valuable contributors from torchchat-contribution to torchchat-contributors
gabe-l-hart
force-pushed
the
download-fix
branch
2 times, most recently
from
a74ddc9 to
0c14af5
Compare
The previous Python version check was incorrect, allowing installations on unsupported interpreter versions, which caused installation failures. Additionally, we now respect the specified interpreter version if provided, consistently using it throughout the installation process by enforcing it with pip. Signed-off-by: Sébastien Han <[email protected]>
Co-authored-by: Jack-Khuu <[email protected]>
* toeknizer was missing an include * fix a nit --------- Co-authored-by: Jesse <[email protected]> Co-authored-by: Jack-Khuu <[email protected]>
…resses ctrl+c (pytorch#1352) Setup a SIGINT handler to gracefully exit the program once the user presses ctrl+c. Signed-off-by: Sébastien Han <[email protected]> Co-authored-by: Jack-Khuu <[email protected]>
…ed values (pytorch#1359) * Update cli.py to make --device/--dtype pre-empt quantize dict-specified values Users may expect that cli parameters override the JSON, as per pytorch#1278. Invert logic - case split: 1 - if none (no value) is specified, use value specified in quantize dict, if present; else 2 - if value is specified, override the respective handler if present. * Fix typo in cli.py fix typo --------- Co-authored-by: Jack-Khuu <[email protected]>
…ytorch#1369) * Only set up during the first sample * Cleaner
…pytorch#1368) * Update install_requirements.sh to support python 3.10 >= , <3.13 * Update install_requirements.sh * Update install_requirements.sh
`gguf` was listed twice on the dependency list. Signed-off-by: Sébastien Han <[email protected]>
If the chat is exited or interrupted it will still print the stats with NaN values which is unnecessary. Signed-off-by: Sébastien Han <[email protected]>
…torch#1372) Let's gracefully fail if no model is given to the `download` command. Signed-off-by: Sébastien Han <[email protected]>
Downloading a Mistral model fails because it includes multiple weight mapping files. The regression was introduced in commit `766bee9f4a1fcb187fae543a525495d3ff482097`. I'm unclear on the original intent, but perhaps the exception was meant to apply only to Granite models. This isn’t an ideal fix, but it does enable Mistral to be downloaded and used for chat. Signed-off-by: Sébastien Han <[email protected]>
The previous logic didn't handle .bin files, so if a model (like mistral) has both .bin and .safetensors, it would download both. Branch: download-fix Signed-off-by: Gabe Goodhart <[email protected]>
This will not actually be needed for mistral with the fix in download to handle .bin files, but it may be needed for other models, so it's worth having. Branch: download-fix Signed-off-by: Gabe Goodhart <[email protected]>
gabe-l-hart
force-pushed
the
download-fix
branch
from
9a0b5fa to
5747a71
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds a couple of more robust fixes to fix the mistral download problems:
.binfiles correctly in the bin vs safetensors logic