This repository was archived by the owner on May 20, 2025. It is now read-only.
File tree Expand file tree Collapse file tree 2 files changed +19
-2
lines changed Expand file tree Collapse file tree 2 files changed +19
-2
lines changed Original file line number Diff line number Diff line change @@ -382,6 +382,9 @@ async def do_download_audio_model(ctx: MessageContext):
382382@main_api.post (" /download-model" )
383383async def download_audio (ctx : HttpContext):
384384 model_id = ctx.req.query.get(" model" , audio_model_id)
385+
386+ if isinstance (model_id, list ):
387+ model_id = model_id[0 ]
385388 # asynchronously download the model
386389 await download_audio_model.publish({ " model_id" : model_id })
387390
@@ -662,7 +665,7 @@ nitric stack new test aws
662665This will generate a nitric stack file called ` test ` which defines how we want to deploy a stack to AWS. We can update this stack file with settings to configure our batch service and the AWS Compute environment it will run in.
663666
664667``` yaml title: nitric.test.yaml
665- provider : nitric/aws@1.14.2
668+ provider : nitric/aws@1.15.4
666669# The target aws region to deploy to
667670# See available regions:
668671# https://docs.aws.amazon.com/general/latest/gr/lambda-service.html
Original file line number Diff line number Diff line change @@ -269,6 +269,9 @@ async def do_download_audio_model(ctx: MessageContext):
269269@main_api.post (" /download-model" )
270270async def download_audio (ctx : HttpContext):
271271 model_id = ctx.req.query.get(" model" , audio_model_id)
272+
273+ if isinstance (model_id, list ):
274+ model_id = model_id[0 ]
272275 # asynchronously download the model
273276 await download_audio_model.publish({ " model_id" : model_id })
274277
@@ -331,7 +334,8 @@ FROM ghcr.io/astral-sh/uv:python3.11-bookworm AS builder
331334
332335ARG HANDLER
333336ENV HANDLER=${HANDLER}
334-
337+ # Set flags for common execution environment
338+ ENV CMAKE_ARGS="-DLLAMA_NATIVE=OFF -DGGML_NATIVE=OFF -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DGGML_AVX512=OFF -DGGML_AVX512_VNNI=OFF -DGGML_AVX512_VBMI=OFF -DGGML_AVX512_BF16=OFF"
335339ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy PYTHONPATH=.
336340WORKDIR /app
337341RUN --mount=type=cache,target=/root/.cache/uv \
@@ -378,6 +382,16 @@ README.md
378382model.zip
379383```
380384
385+ ## Update our .env file
386+
387+ We'll change some of the environment variables in our ` .env ` file to factor in the extra init time it will take to start our LLM job:
388+
389+ ``` sh title:.env
390+ PYTHONPATH=.
391+ # diff +
392+ WORKER_TIMEOUT=300
393+ ```
394+
381395Finally, we need to tell Nitric to use these files to create the script service. We can do this by updating the ` nitric.yaml ` file:
382396
383397``` yaml title:nitric.yaml
You can’t perform that action at this time.
0 commit comments