Hot standby instance not logging in after 3.4.0 update

[martijnsnijderbs]

Hi Netbox,

Hot standby instance doesn't seem to work anymore after updating to 3.4.0 (coming from 3.3.9).

<class 'django.db.utils.InternalError'>

cannot execute UPDATE in a read-only transaction

Error above after logging in. Worked fine before, I have set the required parameters

MAINTENANCE_MODE = True
SESSION_FILE_PATH = '<path>'

Any ideas?

Python version: 3.8.10
NetBox version: 3.4.0

[kkthxbye-code]

Could you provide the full traceback? Should be in the logs, otherwise you can enable debug mode and it should be displayed in the browser.

[martijnsnijderbs]

Sure,

2022-12-16 11:01:59,658 django.request ERROR: Internal Server Error: /login/
Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.ReadOnlySqlTransaction: cannot execute UPDATE in a read-only transaction


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 92, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
  File "/opt/netbox/netbox/users/views.py", line 41, in dispatch
    return super().dispatch(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 142, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/netbox/netbox/users/views.py", line 87, in post
    if form.is_valid():
  File "/opt/netbox/netbox/utilities/forms/forms.py", line 69, in is_valid
    is_valid = super().is_valid()
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/forms/forms.py", line 200, in errors
    self.full_clean()
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/forms/forms.py", line 438, in full_clean
    self._clean_form()
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/forms/forms.py", line 459, in _clean_form
    cleaned_data = self.clean()
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/forms.py", line 217, in clean
    self.user_cache = authenticate(
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/views/decorators/debug.py", line 42, in sensitive_variables_wrapper
    return func(*func_args, **func_kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 77, in authenticate
    user = backend.authenticate(request, **credentials)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/backends.py", line 52, in authenticate
    if user.check_password(password) and self.user_can_authenticate(user):
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 115, in check_password
    return check_password(raw_password, self.password, setter)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/hashers.py", line 68, in check_password
    setter(password)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 113, in setter
    self.save(update_fields=["password"])
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 68, in save
    super().save(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/base.py", line 812, in save
    self.save_base(
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/base.py", line 863, in save_base
    updated = self._save_table(
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/base.py", line 976, in _save_table
    updated = self._do_update(
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/base.py", line 1040, in _do_update
    return filtered._update(values) > 0
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1216, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor = super().execute_sql(result_type)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.InternalError: cannot execute UPDATE in a read-only transaction

[kkthxbye-code]

I digged a little in the django.contrib.auth code and it's probably because django 4.1 increased the iteration count on the password hasher, so it rehashes the password and updates it on first login.

https://docs.djangoproject.com/en/4.1/releases/4.1/#django-contrib-auth

The solution here would be to log-in once on the production system with the user, then you should be able to use the hot standby again.

[martijnsnijderbs]

I cleared the cookies for my production instance, logged in once and tried logging in on the hot standby instance. Worked as expected again.

Thanks a lot!