Haystack in Docker container on Linux - problem with tiktoken

[mgardner-chegg]

I am really struggling to get Haystack to run in a Docker container - the application leverages 'gpt-3.5-turbo' and 'gpt-4' models from OpenAI for generation and 'text-embedding-ada-002' for embeddings.

The crux of the problem is Haystack seems to assume tiktoken not available when it is. The result it Haystack falls back to GPT2TokenizerFast. This breaks my application since I am using large prompts.

Here is my dockerfile:

# app/Dockerfile

FROM python:3.10-slim

RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    software-properties-common \
    git \
    && rm -rf /var/lib/apt/lists/*

COPY . /app/.

RUN pip install --upgrade pip
RUN pip install contentful rich-text-renderer beautifulsoup4 
RUN pip install streamlit streamlit-chat 
RUN pip install farm-haystack 
RUN pip install 'farm-haystack[faiss]'
RUN pip install tiktoken

WORKDIR /app

EXPOSE 8501
HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health
ENTRYPOINT ["streamlit", "run", "chat-app.py", "--server.port=8501", "--server.address=0.0.0.0"]

The docker container builds but when I run the app I get this warning:

OpenAI tiktoken module is not available for Python < 3.8,Linux ARM64 and AARCH64. Falling back to GPT2TokenizerFast.

If I spin up python on the container I can see tiktoken is installed just fine:

# python 
Python 3.10.11 (main, May  3 2023, 08:40:14) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tiktoken
>>> 

The platform is:

>>> platform.platform()
'Linux-5.15.49-linuxkit-aarch64-with-glibc2.31'
>>> platform.machine()
'aarch64'

I wonder if this line of code is required?

Any pointers here - I need to leverage OpenAI fully and this is stopping me distribute my application.

[mgardner-chegg]

OK - my workaround was to use amd64 base image. Updated dockerfile to:

# app/Dockerfile

FROM amd64/python:3.10-slim

RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    software-properties-common \
    git \
    && rm -rf /var/lib/apt/lists/*

COPY . /app/.

RUN pip install --upgrade pip
RUN pip install contentful rich-text-renderer beautifulsoup4 
RUN pip install streamlit streamlit-chat 
RUN pip install farm-haystack 
RUN pip install 'farm-haystack[faiss]'
RUN pip install tiktoken

WORKDIR /app

EXPOSE 8501
HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health

# To avoid file not found issue described here https://github.com/streamlit/streamlit/issues/4842
ENTRYPOINT [ "streamlit", "run" ]
CMD [ "app.py", "--server.headless", "true", "--server.fileWatcherType", "none", "--browser.gatherUsageStats", "false"]