fix: name check to support TP #174
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the change
This updates a check in
fp8_linear_typeto ensure the module name is not none before checking it against a string.See this todo here: https://github.com/foundation-model-stack/foundation-model-stack/blob/f4d8d1f062cf57d8258b52b18db7239722731553/fms/modules/attention.py#L779-L780
NB: I am not an expert here and don't know if this is correct
Related issues or PRs
How to verify the PR
Load the model
ibm-ai-platform/micro-g3.3-8b-instruct-1b-FP8with TP>1 using fmsWas the PR tested
Coverage is difficult because I don't know what the correct behavior should be, but this change at least gets around the error about checking None with the
inoperatorChecklist for passing CI/CD:
git commit -signoffor equivalenttox -e fixtox -e linttox -e spellchecktox -e unitNote: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.