Deploy whisper with Lambda Container Image (AWS)

[crestrepoq]

There is a way to use it with Lambda? I have been trying whit Lambda Container Image Triggered by S3 when is uploaded a file, but I have had some troubles.

This is the docker file I'm building:

FROM public.ecr.aws/lambda/python:3.9

FROM katalonstudio/katalon
RUN apt-get -y update &&  apt-get install -y ffmpeg 
RUN apt-get update && apt-get install -y python3-pip

RUN mkdir /tmp/data

ENV TRANSFORMERS_CACHE "/tmp/data"

COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY app.py .

CMD [ "python3", "app.py" ]

This is the app.py for test:

import os
import json
import urllib.parse
import whisper
import boto3

s3 = boto3.client("s3")

def handler(event, context):
    try:
        bucket = event["Records"][0]["s3"]["bucket"]["name"]
        key = event['Records'][0]['s3']['object']['key']
        print("Bucket:", bucket, "key:", key)
        
        # Definir la nueva carpeta /tmp/data
        os.makedirs("/tmp/data", exist_ok=True)

        audio_file = '/tmp/data/{}'.format(key)
        print(audio_file)
        print(os.listdir()) # Chekcing files before download audio

        # Downloading file to transcribe
        s3.download_file(bucket, key, audio_file)
        print(os.listdir()) # Chekcing if file was downloaded

        # Configuring a variable needed by Huggingface I do not know why
        os.environ["TRANSFORMERS_CACHE"] = "/tmp/data"

        model = whisper.load_model("base")
        print("Model is up")
        result = model.transcribe(audio_file, fp16=False)
        print(os.listdir())
        return {
            "statusCode": 200,
            "body": json.dumps(result["text"])
        }
    except Exception as e:
        print(e)
        return {
            "statusCode": 500,
            "body": json.dumps("Error processing the file")
        }

This is the requirements.txt:

boto3
openai-whisper
setuptools-rust

This is the current error:

+ echo Entrypoint
Entrypoint
+ '[' -z '' ']'
+ exec python3 app.py
+ echo Entrypoint
Entrypoint
+ '[' -z '' ']'
+ exec python3 app.py
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
RequestId: e06adc5f-f05c-4fb3-9f56-4bd83240ed81 Error: Runtime exited without providing a reason
Runtime.ExitError

Maybe some of you have worked with Lambda connected to ECS (Lambda Containe Image) and know what I am making bad?

[hannahkamundson]

I am also getting this error

[shengdade]

GPT-4:

It looks like you are trying to run a Lambda function using the Katalon Studio image and the AWS Lambda Python image. The issue might be related to the multi-stage build you are using in your Dockerfile. I would recommend the following changes:

1. Use a single base image for your Dockerfile. Since you need Python and some additional packages, you can start with the katalonstudio/katalon image and install Python on top of it.

Here's an updated version of your Dockerfile:

FROM katalonstudio/katalon

# Install Python and pip
RUN apt-get -y update && \
    apt-get install -y python3 python3-pip ffmpeg

# Create a directory for data
RUN mkdir /tmp/data

# Set the TRANSFORMERS_CACHE environment variable
ENV TRANSFORMERS_CACHE "/tmp/data"

# Copy requirements.txt and install the required packages
COPY requirements.txt .
RUN pip3 install -r requirements.txt

# Copy app.py
COPY app.py .

# Set the CMD for the Lambda function
CMD [ "app.handler" ]

2. Make sure to install the AWS Lambda Python SDK in your requirements.txt file:

boto3
openai-whisper
setuptools-rust
awslambdaric

3. Update the app.py file to use the AWS Lambda Python SDK:

import os
import json
import urllib.parse
import whisper
import boto3
from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
from aws_lambda_powertools.utilities.typing import LambdaContext

s3 = boto3.client("s3")

logger = Logger(service="whisper-lambda", level="INFO", correlation_paths=correlation_paths.ALL)
app = ApiGatewayResolver()

@app.post("/transcribe")
def transcribe(event: dict, context: LambdaContext) -> dict:
    try:
        bucket = event["Records"][0]["s3"]["bucket"]["name"]
        key = event['Records'][0]['s3']['object']['key']
        logger.info(f"Bucket: {bucket}, key: {key}")

        # Define the new folder /tmp/data
        os.makedirs("/tmp/data", exist_ok=True)

        audio_file = '/tmp/data/{}'.format(key)
        logger.info(f"Audio file: {audio_file}")

        # Downloading file to transcribe
        s3.download_file(bucket, key, audio_file)

        # Configuring a variable needed by Huggingface
        os.environ["TRANSFORMERS_CACHE"] = "/tmp/data"

        model = whisper.load_model("base")
        logger.info("Model is up")
        result = model.transcribe(audio_file, fp16=False)

        return {
            "statusCode": 200,
            "body": json.dumps(result["text"])
        }
    except Exception as e:
        logger.exception(e)
        return {
            "statusCode": 500,
            "body": json.dumps("Error processing the file")
        }

def handler(event, context):
    return app.resolve(event, context)

4. Deploy the Lambda function using the updated Dockerfile and app.py file.

With these changes, your Lambda function should work as expected. If you still encounter issues, please check the Lambda function logs for more information on the error.

[randomhash]

Just a quick note, if you need to process data in large amounts, lambda is not a way to go here.
Costs will bite your budget a bit :)

[suthi5iale]

What you are suggesting instead of Lambda?

[rvryan67]

Did you get this to work, I want to do same.

[gadi-playstream-gg]

Did u guys used SAM CLI to build and deploy this one?
I'm really stuggling to invoke the lambda locally.
anyone can point to some kind of tutorial how to get this working in a lambda project?

[hannahkamundson]

My issue was memory. I increased the memory for AWS lambda and that fixed it

[gadi-playstream-gg]

trying to anaylze a 45 seconds wav file on the base modal at it seems like I hit max memory

[nayan924]

your error is entry, ur dockerfile needs to enter app.handler, not python app.py, all lambda images need to enter via app.handler, and write ur logic in handler fundtion.