ERROR Running in Docker on all OS's

[dancesWithCycles]

Hi folks,
Thank you so much for providing and maintaining this repository!

Operating system

uname -a
Linux ltpt490 5.10.0-12-amd64 #1 SMP Debian 5.10.103-1 (2022-03-07) x86_64 GNU/Linux

Python version

Python is not required on the development system when you are going the Docker way.

Problem description

Console / terminal output if any

I build a Docker image following the documentation in the Wiki. I published the image and pulled and run it on the following host.

uname -a
Linux ltpt490 5.10.0-12-amd64 #1 SMP Debian 5.10.103-1 (2022-03-07) x86_64 GNU/Linux

Docker comes straight back with the following error.

docker run -it -p 5555:5000 -v "`pwd`":/app danceswithcycles/static-gtfs-manager:march-18-2022
python3: can't open file 'GTFSManager.py': [Errno 2] No such file or directory

Expected Action

I expected Docker to start and run the image.

Details of data you are working with, if relevant

Q: Is there anything peculiar about the data you are working with, which may be causing the issue? Any extra files in the feed? Extra columns? Any mandatory columns/fields omitted?
A: <!--- enter answer here --->

Q: What is the size of the dataset : how many lines in stops.txt, routes.txt and stop_times.txt?
A: <!--- enter answer here --->

Q: Are you starting a new feed from scratch and haven't filled in any of the other files like calendar.txt?
A: <!--- enter answer here --->

Anything else

I appreciate any hint in the right direction.

Cheers!

[answerquest]

Hi @dancesWithCycles thanks for posting this. I'm not able to determine right now what may be causing this issue; haven't checked this project in a long time. Will try to reproduce in this weekend.

[answerquest]

@dancesWithCycles I got around to testing this.. built and ran a dockerized version of this application successfully without incident.

Sharing the commands from my linux system:

docker build -t static-gtfs-manager .
docker run --rm -it -p 5000:5000 -v "$(pwd)":/app static-gtfs-manager

Can you try it once?

[dancesWithCycles]

Hi @answerquest ,
Thanks a lot for coming back to me. I'll check out your suggestion and keep you in the loop.

Cheers!

[dancesWithCycles]

@answerquest: Let me provide a short reply now and more details later: The two mentioned instructions of you above are working on my laptop. Now I am eager to learn, what is the difference between this observation and the one on my cloud server that led to this issue in the first place. I'll keep you in the loop!

[dancesWithCycles]

Hi @answerquest,
I am back to deliver on my promise providing more details.

1. Using Docker I can build and run this repository locally on my Laptop (Debian 11.2).
2. Using Docker I can build and run this repository locally on my Server (Debian 11.3).
3. Using Docker I can NOT build this repository on my Laptop (Debian 11.2), push it to a Docker repository and finally run it on my Server (Debian 11.3).

Doing use case 3. results in this issue.

Any ideas about what goes wrong?

Cheers!

[answerquest]

@dancesWithCycles wow, this looks unique, since the docker image ought to be a self-contained operating system in itself independent of the host OS that can be run anywhere; at least that's what they say on the packaging ;)

I'd only tested this out on a local ubuntu-based laptop; the core docker image python:3.6-slim-stretch seems proper to me. You can try upgrading to some later version at your end to see if it works? Though that can cause other issues regd dependencies (i think I read about some clash between tornado and python 3.9) so if more things go wrong then better to abandon.

Are you pulling it from your docker repository properly? try docker images on the server and check?

[dancesWithCycles]

@dancesWithCycles wow, this looks unique, since the docker image ought to be a self-contained operating system in itself independent of the host OS that can be run anywhere; at least that's what they say on the packaging ;)

I'd only tested this out on a local ubuntu-based laptop; the core docker image python:3.6-slim-stretch seems proper to me. You can try upgrading to some later version at your end to see if it works? Though that can cause other issues regd dependencies (i think I read about some clash between tornado and python 3.9) so if more things go wrong then better to abandon.

Are you pulling it from your docker repository properly? try docker images on the server and check?

@answerquest : How can I pull it not properly? I am pulling it from hub.docker.com. What do you mean with docker images? That lists lokally available images. The image in question will be there, if I pull it from the hub by running docker run.... Cheers!

[dancesWithCycles]

@answerquest: How do you deploy on the production server? Do you call docker build... locally on that production server or just docker run... the docker repo based image?

[danshilm]

@dancesWithCycles I ran into this myself when trying to set this project up and this error is happening because while the container has the /app folder containing all the code for this project, it's that same folder that's being mounted in a volume. And if you're mounting an empty folder to the /app folder, then there's nothing to run inside that folder.

That's not the correct way to use Docker volumes to persist data; instead, only mount the folder(s) where you need the data to be persisted. Usually that's the folder for the database, settings/configuration files, etc... From a quick look at the files/folders that have been created when I create a GTFS feed from scratch and taking into consideration which folders are already in the .gitignore, I believe that mounting the db, export, logs and uploads folders would be good; that's been working out totally fine for me.

@answerquest Is there anything outside of those 4 folders that need to be persisted across (Docker) image updates?

Here's a quick docker-compose.yml file I wrote to make deploying this a lot easier:

version: '3.7'

services:
  static-gtfs-manager:
    build: .
    ports:
      - 5000:5000
    volumes:
      - ./db:/app/db
      - ./export:/app/export
      - ./logs:/app/logs
      - ./uploads:/app/uploads

With this, irrespective of the host OS, I can just run clone the git repo, run docker-compose up -d and Docker will build the image and mount the folders I mentioned earlier so that the data is persisted when the Docker image is recreated. You can go a step further and also upload the built image to Docker Hub so that you don't have to build the image on another system; you can just pull the image you already built the first time and pushed to Docker Hub.

[dancesWithCycles]

Cheers @danshilm !

I'll check your suggestion ASAP and I'll come back here with my experiences.

I call you soon, dear raccoon!

[nukeador]

This is now failing because the docker image is based on an old debian which repo URL is no longer online.

[b-a0]

A fix to keep using the old debian release is to change the sources list:

FROM python:3.6-slim-stretch
RUN sed -i s/deb.debian.org/archive.debian.org/g /etc/apt/sources.list && \
    sed -i s/security.debian.org/archive.debian.org/g /etc/apt/sources.list && \
    sed -i '/stretch-updates/d' /etc/apt/sources.list
RUN apt-get update && apt-get -y upgrade && \
    apt-get install -y python3-pip \
    && rm -rf /var/lib/apt/lists/*
RUN mkdir -p /app
WORKDIR /app
COPY . /app/
RUN pip3 install -r requirements.txt
EXPOSE 5000
CMD ["python3", "GTFSManager.py"]