Failed at step EXEC spawning /opt/netbox/venv/bin

[Lloydr149]

Hello,

I try to install Netbox v3.0.7 on RHEL 8.4 on a Secured Custom Enterprise Template.

I followed the isntall process here : https://netbox.readthedocs.io/en/stable/ and I am blocked when I try to start the netbox and netbox-rq services.
Can you please help me?

Nov 15 16:34:42 host-netbox sudo[3104]: pam_unix(sudo:session): session closed for user root
Nov 15 16:34:42 host-netbox systemd[3110]: netbox.service: Failed to execute command: Permission non accordée
Nov 15 16:34:42 host-netbox systemd[3110]: netbox.service: Failed at step EXEC spawning /opt/netbox/venv/bin/gunicorn: Permission non accordée
-- Subject: Process /opt/netbox/venv/bin/gunicorn could not be executed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The process /opt/netbox/venv/bin/gunicorn could not be executed and failed.
--
-- The error number returned by this process is 13.
Nov 15 16:34:42 host-netbox systemd[1]: netbox.service: Main process exited, code=exited, status=203/EXEC
Nov 15 16:34:42 host-netbox systemd[1]: netbox.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit netbox.service has entered the 'failed' state with result 'exit-code'.
Nov 15 16:34:42 host-netbox systemd[3111]: netbox-rq.service: Failed to execute command: Permission non accordée
Nov 15 16:34:42 host-netbox systemd[3111]: netbox-rq.service: Failed at step EXEC spawning /opt/netbox/venv/bin/python3: Permission non accordée
-- Subject: Process /opt/netbox/venv/bin/python3 could not be executed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support

I fisrt had a lot of issue (upgrade.sh did not work ... proxy errors... pip upgrade... ) Now, this. And I am blocked.
When a launch the following commands as root, it is working good, but as netbox user, it is not:

ExecStart=/opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py rqworker
ExecStart=/opt/netbox/venv/bin/gunicorn --pid /var/tmp/netbox.pid --pythonpath /opt/netbox/netbox --config /opt/netbox/gunicorn.py netbox.wsgi

Thank you for your help.

[Lloydr149]

If it can help:

Package                       Version
----------------------------- ---------
aniso8601                     7.0.0
asgiref                       3.4.1
certifi                       2021.10.8
charset-normalizer            2.0.7
click                         8.0.3
coreapi                       2.3.3
coreschema                    0.0.4
Django                        3.2.8
django-cors-headers           3.10.0
django-debug-toolbar          3.2.2
django-filter                 21.1
django-graphiql-debug-toolbar 0.2.0
django-js-asset               1.2.2
django-mptt                   0.13.4
django-pglocks                1.0.4
django-prometheus             2.1.0
django-redis                  5.0.0
django-rq                     2.4.1
django-tables2                2.4.1
django-taggit                 1.5.1
django-timezone-field         4.2.1
djangorestframework           3.12.4
drf-yasg                      1.20.0
ghp-import                    2.0.2
graphene                      2.1.9
graphene-django               2.15.0
graphql-core                  2.3.2
graphql-relay                 2.0.1
gunicorn                      20.1.0
idna                          3.3
importlib-metadata            4.8.1
inflection                    0.5.1
itypes                        1.2.0
Jinja2                        3.0.2
Markdown                      3.3.4
markdown-include              0.6.0
MarkupSafe                    2.0.1
mergedeep                     1.3.4
mkdocs                        1.2.3
mkdocs-material               7.3.5
mkdocs-material-extensions    1.0.3
netaddr                       0.8.0
packaging                     21.0
Pillow                        8.4.0
pip                           21.3.1
prometheus-client             0.11.0
promise                       2.3
psycopg2-binary               2.9.1
Pygments                      2.10.0
pymdown-extensions            9.0
pyparsing                     3.0.3
python-dateutil               2.8.2
pytz                          2021.3
PyYAML                        6.0
pyyaml_env_tag                0.1
redis                         3.5.3
requests                      2.26.0
rq                            1.10.0
ruamel.yaml                   0.17.16
ruamel.yaml.clib              0.2.6
Rx                            1.6.1
setuptools                    50.3.2
singledispatch                3.7.0
six                           1.16.0
sqlparse                      0.4.2
svgwrite                      1.4.1
tablib                        3.1.0
text-unidecode                1.3
uritemplate                   4.1.1
urllib3                       1.26.7
watchdog                      2.1.6
zipp                          3.6.0

[root@host-netbox]# python3 -V Python 3.9.2

[candlerb]

That list of packages doesn't help, unless you ran it while the virtualenv was activated. You shouldn't install any packages into the system environment (i.e. don't be tempted to do pip3 install -r requirements.txt)

When a launch the following commands as root, it is working good, but as netbox user, it is not:

Try it from the command line, you might get a clearer error.

sudo su - netbox
/opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py rqworker

Sounds to me like either the permissions on this file aren't correct:

ls -l /opt/netbox/venv/bin/python3

Or maybe there's an issue with SELinux (you can try setting it to permissive mode to see if that's it)

[Lloydr149]

I first sent list of packages because first issue was the upgrade.sh script told me it upgraded the packages but it did not. So, when I launch it a second time, it told me exactly the same things that it upgrades the list from the same frist versions... but ok, anyway ....

[root@host-netbox]# sudo su - netbox
[netbox@host-netbox ~]$ /opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py rqworker
-bash: /opt/netbox/venv/bin/python3: Permission denied
[netbox@host-netbox ~]$ ls -l /opt/netbox/venv/bin/python3
ls: cannot access '/opt/netbox/venv/bin/python3': Permission denied

[netbox@host-netbox ~]$ exit
logout

[root@vocblab-netbox ]# ls -l /opt/netbox/venv/bin/python3
lrwxrwxrwx. 1 root root 16 Nov 15 15:12 /opt/netbox/venv/bin/python3 -> /usr/bin/python3

but

[root@host-netbox ]# ls -l /opt/netbox/venv/
total 16
drwx------. 3 root root 4096 Nov 15 15:12 bin

[candlerb]

I first sent list of packages because first issue was the upgrade.sh script told me it upgraded the packages but it did not.

It did. The packages are installed inside a virtual environment. If you want to see the packages there, then you have to activate it first:

cd /opt/netbox
. venv/bin/activate
pip list

When you run ./upgrade.sh again, it destroys the virtualenv and recreates it from scratch.

[root@host-netbox ]# ls -l /opt/netbox/venv/
total 16
drwx------. 3 root root 4096 Nov 15 15:12 bin

Ah, there's your problem then. I don't know how it was created like that; maybe something about the default umask setting in RedHat? Mine (Ubuntu 20.04) is:

root@netbox3:~# ls -l /opt/netbox/venv/
total 11
drwxr-xr-x 3 root root 39 Nov 12 14:00 bin
drwxr-xr-x 2 root root  2 Nov 12 13:58 include
drwxr-xr-x 3 root root  3 Nov 12 13:58 lib
lrwxrwxrwx 1 root root  3 Nov 12 13:58 lib64 -> lib
-rw-r--r-- 1 root root 70 Nov 12 13:58 pyvenv.cfg
drwxr-xr-x 3 root root  3 Nov 12 13:58 share

The fix should be straightforward:

chmod go+rx /opt/netbox/venv/bin

... but there may be other directories which need fixing too.

[Lloydr149]

When you run ./upgrade.sh again, it destroys the virtualenv and recreates it from scratch.
OK, I did not got that point. Thanks

chmod go+rx /opt/netbox/venv/bin
Done

chmod go+rx /opt/netbox/venv/
Done too, the folder was on the same state as "bin"

Now, I have got
Permission denied: '/opt/netbox-3.0.7/venv/pyvenv.cfg'

Should I change all /opt/netbox folder/sub-folders/files permission to netbox user? If yes, how can I do it ?
Why is it 'root' and not 'netbox' user the owner?

[candlerb]

Why is it 'root' and not 'netbox' user the owner?

It's more secure that way. If someone breaks into the netbox application as the "netbox" user, they should not be able to modify the application itself. The only directory which it the netbox user should be able to write to is the "media" directory, as documented here.

However, obviously the netbox application needs to be readable for it to be able to run.

Fixing this is just a normal sysadmin thing. I'd do something like this:

find /opt/netbox -type f | xargs chmod +r
find /opt/netbox -type d | xargs chmod +rx

[berkeleycowboy]

I was able to get Gunicorn operational by changing the owner and group owner of the

[root@scdl-ifipam-001 opt]# chown -R netbox:netbox netbox-3.5.8/

so the rights \ permissions are as follows:

[dennis@scdl-ifipam-001 opt]$ ls -l
total 4
lrwxrwxrwx. 1 netbox netbox 18 Aug 21 20:38 netbox -> /opt/netbox-3.5.8/
drwxrwxr-x. 9 netbox netbox 4096 Sep 5 09:20 netbox-3.5.8
[dennis@scdl-ifipam-001 opt]$

I'm now troubleshooting nginx 502 Bad Gateway error message.

I believe I'm making progress.