Skip to content
This repository was archived by the owner on May 20, 2025. It is now read-only.

Commit 841889b

Browse files
committed
Guide updates.
1 parent 9860226 commit 841889b

File tree

2 files changed

+19
-2
lines changed

2 files changed

+19
-2
lines changed

docs/guides/python/ai-podcast-part-1.mdx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -378,6 +378,9 @@ async def do_download_audio_model(ctx: MessageContext):
378378
@main_api.post("/download-model")
379379
async def download_audio(ctx: HttpContext):
380380
model_id = ctx.req.query.get("model", audio_model_id)
381+
382+
if isinstance(model_id, list):
383+
model_id = model_id[0]
381384
# asynchronously download the model
382385
await download_audio_model.publish({ "model_id": model_id })
383386

@@ -658,7 +661,7 @@ nitric stack new test aws
658661
This will generate a nitric stack file called `test` which defines how we want to deploy a stack to AWS. We can update this stack file with settings to configure our batch service and the AWS Compute environment it will run in.
659662

660663
```yaml title: nitric.test.yaml
661-
provider: nitric/aws@1.14.2
664+
provider: nitric/aws@1.15.4
662665
# The target aws region to deploy to
663666
# See available regions:
664667
# https://docs.aws.amazon.com/general/latest/gr/lambda-service.html

docs/guides/python/ai-podcast-part-2.mdx

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,9 @@ async def do_download_audio_model(ctx: MessageContext):
269269
@main_api.post("/download-model")
270270
async def download_audio(ctx: HttpContext):
271271
model_id = ctx.req.query.get("model", audio_model_id)
272+
273+
if isinstance(model_id, list):
274+
model_id = model_id[0]
272275
# asynchronously download the model
273276
await download_audio_model.publish({ "model_id": model_id })
274277

@@ -331,7 +334,8 @@ FROM ghcr.io/astral-sh/uv:python3.11-bookworm AS builder
331334

332335
ARG HANDLER
333336
ENV HANDLER=${HANDLER}
334-
337+
# Set flags for common execution environment
338+
ENV CMAKE_ARGS="-DLLAMA_NATIVE=OFF -DGGML_NATIVE=OFF -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DGGML_AVX512=OFF -DGGML_AVX512_VNNI=OFF -DGGML_AVX512_VBMI=OFF -DGGML_AVX512_BF16=OFF"
335339
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy PYTHONPATH=.
336340
WORKDIR /app
337341
RUN --mount=type=cache,target=/root/.cache/uv \
@@ -378,6 +382,16 @@ README.md
378382
model.zip
379383
```
380384

385+
## Update our .env file
386+
387+
We'll change some of the environment variables in our `.env` file to factor in the extra init time it will take to start our LLM job:
388+
389+
```sh title:.env
390+
PYTHONPATH=.
391+
# diff +
392+
WORKER_TIMEOUT=300
393+
```
394+
381395
Finally, we need to tell Nitric to use these files to create the script service. We can do this by updating the `nitric.yaml` file:
382396

383397
```yaml title:nitric.yaml

0 commit comments

Comments
 (0)