update inferentia2 image #106
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
oOraph
commented
Mar 13, 2025
oOraph
commented
- Update base image to ubuntu 22.04 (20.04 reaches end of support this year anyway)
- Remove some non required dependencies for the webservice to start
- Remove unused build arg from docker file
Contributor
Author
|
side note: I may be wrong in removing all the dependencies (openjdk and stuff). I did so because I do not see where they are used, so no need to embed them, but I may miss the whole picture about the image usage. I just built the inf2 image as if I were to deploy it on HF endpoints, for the encoder models it now supports on inf2 |
Contributor
Author
|
for the inf2_env.py helper script, strongly inspired from the script we made on tgi neuronx side |
oOraph
force-pushed
the
dev/inf2_image
branch
3 times, most recently
from
038e9eb to
78a245c
Compare
20.04 is reaching end of support this year anyway remove some unused dependencies for the inference web service to run Signed-off-by: Raphael Glon <[email protected]>
Signed-off-by: Raphael Glon <[email protected]>
- + fix broken dep install (kenlm) Signed-off-by: Raphael Glon <[email protected]>
use the same HF_MODEL_ID env var as the main server Signed-off-by: Raphael Glon <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.