Netbox upgrade from v4.2.4 to v4.5.2 fails on database migration

[osuewad]

NetBox Edition

NetBox Community

NetBox Version

v4.2.4 / v4.5.2

Python Version

3.12

Steps to Reproduce

Upgrade Ubuntu 22.04 to 24.04
Upgrade python 3.10 to 3.12

Other dependencies are met as per documentation

Stop netbox netbox-rq services
Run .\upgrade.sh

Extra info:
within /opt/netbox/netbox/netbox/configuration.py -> changed old database to databaseS format and also added API_TOKEN_PEPPERS:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'netbox',               # Database name
        'USER': 'netbox',               # PostgreSQL username
        'PASSWORD': 'redacted', # PostgreSQL password
        'HOST': 'redacted', # Database server
        'PORT': '5432',                     # Database port (leave blank for default)
        'CONN_MAX_AGE': 300,            # Max database connection age
    }

}

API_TOKEN_PEPPERS = {
    1: 'redacted',
}

Expected Behavior

Upgrade migration works without issue and upgrade is finished to v4.5.2

Observed Behavior

Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 111, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 114, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/loader.py", line 58, in __init__
    self.build_graph()
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/loader.py", line 235, in build_graph
    self.applied_migrations = recorder.applied_migrations()
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/recorder.py", line 89, in applied_migrations
    if self.has_table():
       ^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/recorder.py", line 63, in has_table
    with self.connection.cursor() as cursor:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 320, in cursor
    return self._cursor()
           ^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/dummy/base.py", line 20, in complain
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.

[pheus]

Thanks for the report (and for including the full traceback).

Could you help us narrow down whether something external is influencing the settings during the upgrade?

A couple quick questions/tests:

• Do you have any plugins installed? If so, could you temporarily disable them (e.g. clear PLUGINS / remove plugin config) and re-run the upgrade to see if the migration step succeeds?
• Can you try upgrading to v4.5.1 using the same environment (Ubuntu 24.04 / Python 3.12) and confirm whether migrations run successfully there?

If you can share the outcome of those two checks (plus any additional traceback/log output, if it changes), this would be great. Thanks!

[albert-flowerbed]

Can confirm that disabling plugins (PLUGINS = []) is a workaround !
Was using #PLUGINS = ['netbox_interface_synchronization', 'netbox_topology_views', 'netbox_lifecycle']

[osuewad]

@pheus Yes, we do have plugins, but before doing anything I did comment them out and add an empty field:

# Enable installed plugins. Add the name of each plugin to the list.
PLUGINS = [] #empty plugin line for updating
#PLUGINS = ["netbox_data_flows", "netbox_lifecycle", "netbox_topology_views"]

I re-pulled git for v.4.5.1. It seems to fail on the same step, now with a different traceback:

Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
    parser = self.create_parser(argv[0], argv[1])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 375, in create_parser
    self.add_arguments(parser)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 45, in add_arguments
    choices=tuple(connections),
            ^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/utils/connection.py", line 73, in __iter__
    return iter(self.settings)
                ^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/utils/connection.py", line 45, in settings
    self._settings = self.configure_settings(self._settings)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/utils.py", line 148, in configure_settings
    databases = super().configure_settings(databases)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/utils/connection.py", line 50, in configure_settings
    settings = getattr(django_settings, self.settings_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 81, in __getattr__
    self._setup(name)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 68, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 166, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/netbox/netbox/netbox/settings.py", line 277, in <module>
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: STORAGE_BACKEND and STORAGES are both set, remove the deprecated STORAGE_BACKEND setting.

Extra info of all installed versions:

Current Netbox version installed: "4.5.1"
Current Python3 version installed : Python 3.12.3
Redis server version installed: Redis server v=7.0.15
Max available Python3 version in apt: python3.12
Ubuntu version: Ubuntu 24.04.4 LTS
PostgreSQL server version:  14.20 (Ubuntu 14.20-0ubuntu0.22.04.1)

[pheus]

Thanks for testing and reporting back!

This doesn’t look like a DB migration regression in 4.5.2 after all. Your 4.5.1 run is now surfacing the underlying configuration issue:

STORAGE_BACKEND and STORAGES are both set, remove the deprecated STORAGE_BACKEND setting.

So the upgrade is failing because both the legacy STORAGE_BACKEND setting and the newer STORAGES setting are present at the same time. To proceed, you’ll want to remove STORAGE_BACKEND entirely and keep only the STORAGES configuration (per the current config example/docs).

Given you’ve already had to adjust DATABASE → DATABASES and add API_TOKEN_PEPPERS, I’d recommend one of these approaches:

• Best option: Start from the current configuration_example.py shipped with your target NetBox release and re-apply your customizations on top of it.
• Alternatively, carefully port your existing custom config forward, but double-check you’re not carrying forward any deprecated settings.

If you rebuild from the example, please make sure you bring over all required settings (e.g. SECRET_KEY, DATABASES, Redis config, ALLOWED_HOSTS, etc.) and then retry upgrade.sh.

Once STORAGE_BACKEND is removed, if you still hit any errors, feel free to paste the next traceback and we’ll take another look.