Upgrading 3.4.10 -> 3.7.4 whoes

[ziggekatten]

All, I'm trying to upgrade my local environment from 3.4.10 to 3.7.4

I've done these types of ugrades before, but this time I'm having an issue.

OS: Ubuntu 22.04

Verified that apt package requirements are in place

Fetched the tarball

New version untared into /opt/netbox-3.7.4

Config files copied as of documentation

The link /opt/netbox points to the new version

So far so good, but running upgrade.sh yields an errror on xmlsec:

Using cached pyyaml_env_tag-0.1-py3-none-any.whl (3.9 kB)
Using cached referencing-0.33.0-py3-none-any.whl (26 kB)
Using cached rpds_py-0.18.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Using cached watchdog-4.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for xmlsec (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      WARNING setuptools_scm._integration.setuptools pyproject.toml does not contain a tool.setuptools_scm section
      WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/xmlsec
      copying src/xmlsec/constants.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
      copying src/xmlsec/__init__.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
      copying src/xmlsec/tree.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
      copying src/xmlsec/template.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
      copying src/xmlsec/py.typed -> build/lib.linux-x86_64-cpython-310/xmlsec
      running build_ext
      error: Unable to invoke pkg-config.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for xmlsec
Failed to build xmlsec
ERROR: Could not build wheels for xmlsec, which is required to install pyproject.toml-based projects

Surely I'm missing something here

[markkuleinio]

I don't do Ubuntu but I run Debians. In my Debian 12 setup (using Python 3.11):

...
Using cached pyyaml_env_tag-0.1-py3-none-any.whl (3.9 kB)
Using cached referencing-0.33.0-py3-none-any.whl (26 kB)
Using cached rpds_py-0.18.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Using cached watchdog-4.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: [...]

Why does it try to install/build xmlsec? I don't have it installed in my NetBoxes (except when I have the SAML plugin installed, then I install xmlsec1 with apt)

Since the error is Unable to invoke pkg-config you may try installing it with apt. But, I don't have it installed in my Debian (but again, it doesn't try to build xmlsec)

[ziggekatten]

You nailed it. Seems that I had the saml package in my requirements file for some reason (used it in earlier setups but have moved to OIDC).

However, it seems Ubuntu needs libXML2 to be built from source looking at the docs. For those using SAML there will be an challenge....

[kennedymeadows]

Did you find a precise solution? I'm doing a new build of 3.7.4 on ubuntu and having the same issue.
Never had an issue with 3.6.3. This isn't an upgrade, it's a new build via chef automation.

[kennedymeadows]

Yes I did try that - no luck doing it that way. Removing python3-saml has allowed me to build netbox via chef automation, I am going to keep trying to troubleshoot this package.

[kennedymeadows]

SAML-Toolkits/python3-saml#356 The issue is definitely not a netbox issue, and apparently not even a python3-saml issue.

[markkuleinio]

Ok

I'm using https://github.com/jeremyschulman/netbox-plugin-auth-saml2 for SAML authentication in NetBox, and for it installing xmlsec1 with apt has been enough.

[kennedymeadows]

So it looks like you have the two projects at odds over who will be fixing this issue, but in the meantime, this is the list of apt packages I have included in my install which ended up working and allowing me to install python3-saml:

default['chef_netbox']['packages'] = ['pkg-config', 'postgresql', 'redis-server', 'python3', 'python3-pip', 'python3-venv', 'python3-dev', 'build-essential', 'libxml2-dev', 'libxslt1-dev', 'libffi-dev', 'libpq-dev', 'libssl-dev', 'zlib1g-dev', 'nginx', 'libxmlsec1-openssl', 'libxmlsec1-dev', 'python3-xmlsec']

Thanks for your support @markkuleinio and @squintfox

[kennedymeadows]

If anyone comes across this, the root cause of this issue was fixed last week. Reverting all the apt dependencies back to the ones listed in the netbox documentation now works perfectly fine and plays nice with python3-saml.