Netbox Upgrade Failing with psycopg2.errors.UniqueViolation errors.

[cryansharp]

I am attempting to upgrade Netbox from version 3.3.9 to the latest version and it continues to fail during the "Running migrations" stage. Here is the exact error:

Applying dcim.0162_unique_constraints...Traceback (most recent call last):
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
psycopg2.errors.UniqueViolation: could not create unique index "dcim_device_unique_name_site_tenant"
DETAIL: Key (lower(name::text), site_id, tenant_id)=(nexus 3048 tp, 21, 12) is duplicated.

These are the steps that I'm running:

1. cd /opt/netbox
2. git checkout master
3. git stash (to stash changes to choices.py)
4. git pull origin master
5. sudo ./upgrade.sh

Below is the full output of the error and a screenshot of the error in the Netbox UI is attached.

Operations to perform:
Apply all migrations: admin, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
Running migrations:
Applying wireless.0006_unique_constraints... OK
Applying wireless.0007_standardize_description_comments... OK
Applying wireless.0008_wirelesslan_status... OK
Applying virtualization.0033_unique_constraints... OK
Applying virtualization.0034_standardize_description_comments... OK
Applying tenancy.0008_unique_constraints... OK
Applying tenancy.0009_standardize_description_comments... OK
Applying ipam.0061_fhrpgroup_name... OK
Applying ipam.0062_unique_constraints... OK
Applying ipam.0063_standardize_description_comments... OK
Applying extras.0078_unique_constraints... OK
Applying extras.0079_scheduled_jobs... OK
Applying extras.0080_customlink_content_types... OK
Applying extras.0081_exporttemplate_content_types... OK
Applying extras.0082_savedfilter... OK
Applying dcim.0162_unique_constraints...Traceback (most recent call last):
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
psycopg2.errors.UniqueViolation: could not create unique index "dcim_device_unique_name_site_tenant"
DETAIL: Key (lower(name::text), site_id, tenant_id)=(nexus 3048 tp, 21, 12) is duplicated.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/netbox/netbox/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/init.py", line 446, in execute_from_command_line
utility.execute()
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/init.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/base.py", line 96, in wrapped
res = handle_func(*args, **kwargs)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/core/management/commands/migrate.py", line 349, in handle
post_migrate_state = executor.migrate(
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/migration.py", line 130, in apply
operation.database_forwards(
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/migrations/operations/models.py", line 1040, in database_forwards
schema_editor.add_constraint(model, self.constraint)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/base/schema.py", line 515, in add_constraint
self.execute(sql, params=None)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/base/schema.py", line 199, in execute
cursor.execute(sql, params)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/netbox/venv/lib64/python3.9/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
django.db.utils.IntegrityError: could not create unique index "dcim_device_unique_name_site_tenant"
DETAIL: Key (lower(name::text), site_id, tenant_id)=(nexus 3048 tp, 21, 12) is duplicated.

Any help would be appreciated. Thanks.

[candlerb]

psycopg2.errors.UniqueViolation: could not create unique index "dcim_device_unique_name_site_tenant"
DETAIL: Key (lower(name::text), site_id, tenant_id)=(nexus 3048 tp, 21, 12) is duplicated.

When upgrading from 3.3.x to 3.5.x, you first need to check the release notes for v3.4.0 and v3.5.0 for breaking changes.

In this case, you've been caught by the following change in v3.4:

* Device and virtual machine names are no longer case-sensitive. Attempting to create e.g. "device1" and "DEVICE1" within the same site will raise a validation error.

The Postgres error message is saying that the combination of (lowercased device name, site, tenant) must be unique, and it can't create the index if that condition is violated.

It looks like you have two devices with names that differ only in case, such as "Nexus 3048 tp" and "nexus 3048 tp", both in site id 21 with tenant id 12.

The solution will be to rename one of those devices. Then you should be able to run your migrations (unless there are other cases like this that you need to fix, of course)

[cryansharp]

Thank you! That was very helpful and resolved my problem.