Build VLLM CUDA from AIPCC wheels #86
Conversation
…d-hat-data-services#85) * Update Dockerfile.ubi to install vllm-cuda using wheel from RHEL AI team the install script is located in payload/run.sh. An args file was also added with the custom parameters, and is referenced in the tekton pipeline. * update payload/run.sh to use bot token * add trap to guarantee run.sh deletion
|
Let's hold on merging this until we figure out if we can get provide a vLLM v0.7.3 tag in the midstream and a build from the AIPCC team. |
|
@dtrifiro should I leave the 2.19 branch the way it is? |
|
Yeah, let's wait until we come up with a definitive strategy |
|
sounds good, I will draft this for now and will not make any changes to 2.19 (which already has this change merged in there) |
| WHEEL_RELEASE_ARTIFACTS="https://gitlab.com/api/v4/projects/68045055/packages/generic/rhelai-wheels/${WHEEL_RELEASE}/wheels-${WHEEL_RELEASE}.tar.gz" | ||
|
|
||
|
|
||
| # NOTE - ensure that flashinfer is included in wheel thing |
There was a problem hiding this comment.
Looks like it's included? https://issues.redhat.com/browse/AIPCC-49
|
@tarukumar fyi |
|
|
||
| ## CUDA Base ################################################################### | ||
| FROM python-install as cuda-base | ||
| FROM python-install AS cuda-base |
There was a problem hiding this comment.
Does this cuda-base layer include all of the rpm dependencies required by the wheel set? Should be enough to check with ldd (?)
There was a problem hiding this comment.
If I was able to get a green build, does that mean that all the required dependencies are there? Or do I need to do additional checks?
ckhordiasma
changed the title
ckhordiasma
force-pushed
the
nm-wheel-patch
branch
from
9059191 to
77c398d
Compare
|
/build |
|
/build |
|
/build |
|
/build |
|
/build-from-rhelai-wheels |
|
/build-from-rhelai-wheels |
|
/build-from-rhelai-wheels |
|
/build-from-rhelai-wheels |
|
|
||
| ## Base Layer ################################################################## | ||
| FROM registry.access.redhat.com/ubi9/ubi-minimal:${BASE_UBI_IMAGE_TAG} as base | ||
| FROM registry.access.redhat.com/ubi9/ubi-minimal:${BASE_UBI_IMAGE_TAG} AS base |
There was a problem hiding this comment.
could you please try with a base image built by AIPCC: http://quay.io/aipcc/base-images/cuda
if your builder doesn't have access to that image, you can ping me on Slack and I'll send you the procedure to get access.
There was a problem hiding this comment.
also for reference, this is the dockerfile for RHAIIS: https://gitlab.com/redhat/rhel-ai/rhaiis/containers/-/blob/main/Containerfile.cuda-ubi9?ref_type=heads
|
outdated |
the install script is located in payload/run.sh. An args file was also added with the custom parameters, and is referenced in the tekton pipeline.
update payload/run.sh to use bot token
add trap to guarantee run.sh deletion