Errors when upgrading from 3.4.3 to 3.7.1 ( Migration account.0001_initial is applied before its dependency users.0004_netboxgroup_netboxuser)

[etigui]

Deployment Type

Self-hosted

NetBox Version

v3.7.1

Python Version

3.9

Steps to Reproduce

We have 2 instances, one prod with Netbox 3.4.3 and postgesql 13 and one test with Netbox 3.7.1 and postgesql 15. We are trying to upgrade from the prod to the test instance:

Prod instance (Netbox 3.4.3 with postgesql v13):

1. Dump production DB

Test instance (Netbox 3.7.1 with postgesql v15):

1. Drop the current DB

sudo -u postgres psql -c 'DROP DATABASE netbox'

3. Create DB (as in the doc) and import from dump (netbox v3.4.3)

sudo -u postgres psql -c 'CREATE DATABASE netbox'
sudo -u postgres psql -c 'ALTER DATABASE netbox OWNER TO netbox;'
sudo -u postgres psql -c '\connect netbox;' -c 'GRANT CREATE ON SCHEMA public TO netbox;'
sudo PGPASSWORD="<pass>" psql -U netbox -h localhost -d netbox < netbox.sql

5. Run upgrade.sh

PYTHON=/usr/bin/python3.9 ./upgrade.sh

7. Upgrade get stuck on DB migration

Expected Behavior

Migrate Netbox from 3.4.3 (postgesql v13) to 3.7.1 (postgesql v15)

Observed Behavior

$ PYTHON=/usr/bin/python3.9 ./upgrade.sh
...
Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/commands/migrate.py", line 120, in handle
    executor.loader.check_consistent_history(connection)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/loader.py", line 327, in check_consistent_history
    raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration account.0001_initial is applied before its dependency users.0004_netboxgroup_netboxuser on database 'default'.

[jeremystretch]

I can confirm that migrating from v3.4.3 to v3.7.1 works as expected. You don't say exactly how you're creating the database; I suspect you're attempting to load data into a partially-created database rather than a new one. The new database must be completely empty when loading the v3.4.3 dump.

[markkuleinio]

Have you tried upgrading to some earlier NetBox version first, like latest 3.5.x or 3.6.x, and then to 3.7.1, if that works for you?

This may help you in that: https://github.com/markkuleinio/netbox-freezes

[etigui]

I've tried Netbox 3.7.1 with postgesql 13 instead of 15, but I have not tried to upgrade to older Netbox version. I'll test this. Thanks

[markkuleinio]

Error was: Migration account.0001_initial is applied before its dependency users.0004_netboxgroup_netboxuser

Have you tried applying the users.0004_netboxgroup_netboxuser migration manually, and then running upgrade.sh again?

(Not that this should be necessary, but I guess everything can be tried now)

[markkuleinio]

(FWIW, I have upgraded two 3.3.x setups to 3.7.0/3.7.1 successfully with no problems)

[etigui]

I just migrated from v3.4.3 to v3.6.0. I have the exact same error:

$ PYTHON=/usr/bin/python3.9 ./upgrade.sh
....
Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/commands/migrate.py", line 120, in handle
    executor.loader.check_consistent_history(connection)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/loader.py", line 327, in check_consistent_history
    raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration account.0001_initial is applied before its dependency users.0004_netboxgroup_netboxuser on database 'default'.

FWI: The install without importing the prod DB work just fine. Once I import the prod DB and upgrade it's not working anymore.

[etigui]

Error was: Migration account.0001_initial is applied before its dependency users.0004_netboxgroup_netboxuser

Have you tried applying the users.0004_netboxgroup_netboxuser migration manually, and then running upgrade.sh again?

(Not that this should be necessary, but I guess everything can be tried now)

I have found some tutorials to manually migrate, but none worked as expected. Do you know a clean way to manually migrate users.0004_netboxgroup_netboxuser ?

[etigui]

Ok the issue was related to the relocation of the admin UI views on the version 3.6: https://github.com/netbox-community/netbox/releases/tag/v3.6.0

The migration of the new account app depends on the users app. So, as the migration was already done (as show before), this could not work.

I had to manually comment the dependency and run the migration again:

cat ./netbox/netbox/account/migrations/0001_initial.py
# Generated by Django 4.1.10 on 2023-07-30 17:49

from django.db import migrations


class Migration(migrations.Migration):

    initial = True

    #dependencies = [
    #    ('users', '0004_netboxgroup_netboxuser'),
    #]

    operations = [
        migrations.CreateModel(
            name='UserToken',
            fields=[
            ],
            options={
                'verbose_name': 'token',
                'proxy': True,
                'indexes': [],
                'constraints': [],
            },
            bases=('users.token',),
        ),
    ]
[