Problem with Netbox 3.4.2 after Ubuntu upgrade 18.04 to 20.04

[F3d0s]

I used to run Netbox v 2.11.11 on Ubuntu 18.04 (Azure VM), then I upgraded Netbox to v3.4.2 which need python 3.8 to run properly. Ubuntu 18.04 runs python 3.6, so I installed python 3.10 and selected through update-alternatives --config python3 and it works. The problem is that the Azure VM agent walinuxagent, doesn't work with python 3.6, to resolve that issue, I updated to Ubuntu 20.04 which use python 3.8 as original, but when I start the services netbox and netbox-rq I got the following errors.

$ sudo systemctl status netbox netbox-rq
● netbox.service - NetBox WSGI Service
     Loaded: loaded (/etc/systemd/system/netbox.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2023-02-03 16:32:36 UTC; 6s ago
       Docs: https://netbox.readthedocs.io/en/stable/
    Process: 13896 ExecStart=/opt/netbox/venv/bin/gunicorn --pid /var/tmp/netbox.pid --pythonpath /opt/netbox/netbox --config /opt/netbox/gunicorn.py netbox.wsgi (code=exited, status=1/FAILURE)
   Main PID: 13896 (code=exited, status=1/FAILURE)

● netbox-rq.service - NetBox Request Queue Worker
     Loaded: loaded (/etc/systemd/system/netbox-rq.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Fri 2023-02-03 16:32:36 UTC; 6s ago
       Docs: https://netbox.readthedocs.io/en/stable/
    Process: 13897 ExecStart=/opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py rqworker (code=exited, status=1/FAILURE)
   Main PID: 13897 (code=exited, status=1/FAILURE)

Feb 03 16:32:36 AXIGLB-S181-DEV systemd[1]: netbox-rq.service: Main process exited, code=exited, status=1/FAILURE
Feb 03 16:32:36 AXIGLB-S181-DEV systemd[1]: netbox-rq.service: Failed with result 'exit-code'.

Python configuration:

$ ls -sl /usr/bin/python*
   0 lrwxrwxrwx 1 root root       7 Apr 15  2020 /usr/bin/python -> python2
   0 lrwxrwxrwx 1 root root       9 Mar 13  2020 /usr/bin/python2 -> python2.7
3580 -rwxr-xr-x 1 root root 3662032 Jul  1  2022 /usr/bin/python2.7
   0 lrwxrwxrwx 1 root root      25 Feb  3 18:52 /usr/bin/python3 -> /etc/alternatives/python3
   0 lrwxrwxrwx 1 root root      16 Mar 13  2020 /usr/bin/python3-config -> python3.8-config
5352 -rwxr-xr-x 1 root root 5477688 Jan 28 19:09 /usr/bin/python3.10
   0 lrwxrwxrwx 1 root root      34 Jan 28 19:09 /usr/bin/python3.10-config -> x86_64-linux-gnu-python3.10-config
5368 -rwxr-xr-x 1 root root 5494584 Nov 14 12:59 /usr/bin/python3.8
   0 lrwxrwxrwx 1 root root      33 Nov 14 12:59 /usr/bin/python3.8-config -> x86_64-linux-gnu-python3.8-config

$ which python3
/usr/bin/python3

$ which python 3.8
/usr/bin/python

$ which python 3.10
/usr/bin/python

$ which python
/usr/bin/python

$ update-alternatives --list python
/usr/bin/python2.7

$ update-alternatives --display python3
python3 - manual mode
  link best version is /usr/bin/python3.8
  link currently points to /usr/bin/python3.8
  link python3 is /usr/bin/python3
/usr/bin/python3.10 - priority 100
/usr/bin/python3.8 - priority 200

[candlerb]

Try journalctl -eu netbox to see the errors from when Netbox tries to start up.

My guess is: after upgrading to ubuntu 20.04, you will need to re-run Netbox's ./upgrade.sh to rebuild the virtualenv with the correct version of Python (e.g. the libraries were built with python 3.10, but now the selected python is 3.8)

If you want to continue running Netbox with python 3.10, then you'll need to do

PYTHON=/usr/bin/python3.10 ./upgrade.sh

[F3d0s]

Hi, this is the result of journalctl -eu netbox, these messages are repeated.

Feb 06 07:27:26 NETBOX-DEV systemd[1]: Started NetBox WSGI Service.
Feb 06 07:27:26 NETBOX-DEV gunicorn[389086]: Traceback (most recent call last):
Feb 06 07:27:26 NETBOX-DEV gunicorn[389086]:   File "/opt/netbox/venv/bin/gunicorn", line 5, in <module>
Feb 06 07:27:26 NETBOX-DEV gunicorn[389086]:     from gunicorn.app.wsgiapp import run
Feb 06 07:27:26 NETBOX-DEV gunicorn[389086]: ModuleNotFoundError: No module named 'gunicorn'
Feb 06 07:27:26 NETBOX-DEV systemd[1]: netbox.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 07:27:26 NETBOX-DEV systemd[1]: netbox.service: Failed with result 'exit-code'.
Feb 06 07:27:57 NETBOX-DEV systemd[1]: netbox.service: Scheduled restart job, restart counter is at 7207.
Feb 06 07:27:57 NETBOX-DEV systemd[1]: Stopped NetBox WSGI Service.

I will try what you suggested, re-run ./upgrade.sh to rebuild the virtualenv with Python 3.8 as I don't want to run two version of python again, like I used to do on the old Ubuntu version and see if that will fix the issue.

Edit: I tried that and it works! I also upgraded from Netbox 3.4.2 to 3.4.4 🙂