Is it possible to specify resources.limits
for TaskDefinition.add_container
?
#30529
Unanswered
KMouratidis
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm trying to deploy vllm on an inf2 instance. With Nvidia GPUs, setting up the
gpu_count
is enough, but for inf2 it seems these devices are not visible (No neuron device available [...] Cannot find Neuron devices.
). I've stumbled across this issue / comment where they mention addingaws.amazon.com/neuron: 1
toresources.limits
of the docker compose. For theTaskDefinition.add_container
method, CPU/GPU/memory limits are specified with dedicated parameters, but I see no way to specify other resource limits.Using Python,
aws-cdk-lib==2.137.0
, and here is the link to relevant CDK class in the docsBeta Was this translation helpful? Give feedback.
All reactions