Updating paperless-ngx LXC ends abruptly

[dimatx]

Seems like an issue with my LXC, but hoping you can shed some light on how to best troubleshoot what is causing the script to exit/fail. Is there a verbose/logging mode that can show where it's failing?

When I run:
bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/paperless-ngx-update.sh)"
in the paperless LXC, it stops here:

At that point the app is not running, when I reboot the LXC, the app is not functional and throws 500 error. Restoring from last backup restores the app to operating state.

[dimatx]

Sorry, just spotted this was already posted in the 'issues' section (#789). Didn't search there before posting in discussions.

However, in my case, I installed it using your script.

[tteck]

Would you please try the update script again

[tomdh76]

I had the same issue. Also installed paperless with the script but the update script failed. I had to "pip install celery" and "pip install django_celery_results", then it worked....

edit: well it worked partly. I had issues with incorrect databases so I am now downgrading...

[tteck]

Done!

[tomdh76]

Maybe dimatx can try the update script? My LXC is right now up to date already

[dimatx]

Slightly different result, but unfortunately still does not complete.

Now, instead of exiting out abruptly, it simply hangs on the "Updating paperless-ngx..." step. No CPU/RAM/IO 'spike' at all, so it seems like things fail quite early on before any real operations take place.

(On the positive front, it no longer breaks the existing install when I abort, so I can restart the LXC to continue using v1.80.)

Happy to try any parts of the script that helped you find the issue, @tomdh76. Maybe there's something else that is needed for a successful upgrade.

[dimatx]

Ran pip install -r requirements.txt (in /opt/paperless/)... And this is the full output:

root@paperless-ngx:/opt/paperless# pip install -r requirements.txt
Looking in indexes: https://pypi.python.org/simple, https://www.piwheels.org/simple
Ignoring backports.zoneinfo: markers 'python_version < "3.9"' don't match your environment
Collecting django-q
  Using cached django_q-1.3.9-py3-none-any.whl
Ignoring importlib-resources: markers 'python_version < "3.9"' don't match your environment
Ignoring zipp: markers 'python_version < "3.9"' don't match your environment
Requirement already satisfied: aioredis==1.3.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 3)) (1.3.1)
Requirement already satisfied: anyio==3.6.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 4)) (3.6.1)
Requirement already satisfied: arrow==1.2.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 5)) (1.2.2)
Requirement already satisfied: asgiref==3.5.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 6)) (3.5.2)
Requirement already satisfied: async-timeout==4.0.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 7)) (4.0.2)
Requirement already satisfied: attrs==21.4.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 8)) (21.4.0)
Requirement already satisfied: autobahn==22.6.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 9)) (22.6.1)
Requirement already satisfied: automat==20.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 10)) (20.2.0)
Requirement already satisfied: blessed==1.19.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 12)) (1.19.1)
Requirement already satisfied: certifi==2022.6.15 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 13)) (2022.6.15)
Requirement already satisfied: cffi==1.15.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 14)) (1.15.1)
Requirement already satisfied: channels==3.0.5 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 15)) (3.0.5)
Requirement already satisfied: channels-redis==3.4.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 16)) (3.4.1)
Requirement already satisfied: charset-normalizer==2.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 17)) (2.1.0)
Requirement already satisfied: click==8.1.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 18)) (8.1.3)
Requirement already satisfied: coloredlogs==15.0.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 19)) (15.0.1)
Requirement already satisfied: concurrent-log-handler==0.9.20 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 20)) (0.9.20)
Requirement already satisfied: constantly==15.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 21)) (15.1.0)
Requirement already satisfied: cryptography==37.0.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 22)) (37.0.4)
Requirement already satisfied: daphne==3.0.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 23)) (3.0.2)
Requirement already satisfied: dateparser==1.1.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 24)) (1.1.1)
Requirement already satisfied: deprecated==1.2.13 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 25)) (1.2.13)
Requirement already satisfied: deprecation==2.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 26)) (2.1.0)
Requirement already satisfied: django==4.0.6 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 27)) (4.0.6)
Requirement already satisfied: django-cors-headers==3.13.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 28)) (3.13.0)
Requirement already satisfied: django-extensions==3.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 29)) (3.2.0)
Requirement already satisfied: django-filter==22.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 30)) (22.1)
Requirement already satisfied: django-picklefield==3.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 31)) (3.1)
Requirement already satisfied: djangorestframework==3.13.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 33)) (3.13.1)
Requirement already satisfied: filelock==3.7.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 34)) (3.7.1)
Requirement already satisfied: fuzzywuzzy[speedup]==0.18.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 35)) (0.18.0)
Requirement already satisfied: gunicorn==20.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 36)) (20.1.0)
Requirement already satisfied: h11==0.13.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 37)) (0.13.0)
Requirement already satisfied: hiredis==2.0.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 38)) (2.0.0)
Requirement already satisfied: httptools==0.4.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 39)) (0.4.0)
Requirement already satisfied: humanfriendly==10.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 40)) (10.0)
Requirement already satisfied: hyperlink==21.0.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 41)) (21.0.0)
Requirement already satisfied: idna==3.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 42)) (3.3)
Requirement already satisfied: imap-tools==0.56.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 43)) (0.56.0)
Requirement already satisfied: img2pdf==0.4.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 44)) (0.4.4)
Requirement already satisfied: incremental==21.3.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 46)) (21.3.0)
Requirement already satisfied: inotify-simple==1.3.5 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 47)) (1.3.5)
Requirement already satisfied: inotifyrecursive==0.3.5 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 48)) (0.3.5)
Requirement already satisfied: joblib==1.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 49)) (1.1.0)
Requirement already satisfied: langdetect==1.0.9 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 50)) (1.0.9)
Requirement already satisfied: lxml==4.9.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 51)) (4.9.1)
Requirement already satisfied: msgpack==1.0.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 52)) (1.0.4)
Requirement already satisfied: numpy==1.23.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 53)) (1.23.1)
Requirement already satisfied: ocrmypdf==13.6.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 54)) (13.6.1)
Requirement already satisfied: packaging==21.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 55)) (21.3)
Requirement already satisfied: pathvalidate==2.5.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 56)) (2.5.0)
Requirement already satisfied: pdf2image==1.16.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 57)) (1.16.0)
Requirement already satisfied: pdfminer.six==20220524 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 58)) (20220524)
Requirement already satisfied: pikepdf==5.4.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 59)) (5.4.0)
Requirement already satisfied: pillow==9.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 60)) (9.2.0)
Requirement already satisfied: pluggy==1.0.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 61)) (1.0.0)
Requirement already satisfied: portalocker==2.5.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 62)) (2.5.1)
Requirement already satisfied: psycopg2==2.9.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 63)) (2.9.3)
Requirement already satisfied: pyasn1==0.4.8 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 64)) (0.4.8)
Requirement already satisfied: pyasn1-modules==0.2.8 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 65)) (0.2.8)
Requirement already satisfied: pycparser==2.21 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 66)) (2.21)
Requirement already satisfied: pyopenssl==22.0.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 67)) (22.0.0)
Requirement already satisfied: pyparsing==3.0.9 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 68)) (3.0.9)
Requirement already satisfied: python-dateutil==2.8.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 69)) (2.8.2)
Requirement already satisfied: python-dotenv==0.20.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 70)) (0.20.0)
Requirement already satisfied: python-gnupg==0.4.9 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 71)) (0.4.9)
Requirement already satisfied: python-levenshtein==0.12.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 72)) (0.12.2)
Requirement already satisfied: python-magic==0.4.27 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 73)) (0.4.27)
Requirement already satisfied: pytz==2022.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 74)) (2022.1)
Requirement already satisfied: pytz-deprecation-shim==0.1.0.post0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 75)) (0.1.0.post0)
Requirement already satisfied: pyyaml==6.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 76)) (6.0)
Requirement already satisfied: pyzbar==0.1.9 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 77)) (0.1.9)
Requirement already satisfied: redis==4.3.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 78)) (4.3.4)
Requirement already satisfied: regex==2022.3.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 79)) (2022.3.2)
Requirement already satisfied: reportlab==3.6.11 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 80)) (3.6.11)
Requirement already satisfied: requests==2.28.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 81)) (2.28.1)
Requirement already satisfied: scikit-learn==1.1.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 82)) (1.1.1)
Requirement already satisfied: scipy==1.8.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 83)) (1.8.1)
Requirement already satisfied: service-identity==21.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 84)) (21.1.0)
Requirement already satisfied: setuptools==63.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 85)) (63.2.0)
Requirement already satisfied: six==1.16.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 86)) (1.16.0)
Requirement already satisfied: sniffio==1.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 87)) (1.2.0)
Requirement already satisfied: sqlparse==0.4.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 88)) (0.4.2)
Requirement already satisfied: threadpoolctl==3.1.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 89)) (3.1.0)
Requirement already satisfied: tika==1.24 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 90)) (1.24)
Requirement already satisfied: tqdm==4.64.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 91)) (4.64.0)
Requirement already satisfied: twisted[tls]==22.4.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 92)) (22.4.0)
Requirement already satisfied: txaio==22.2.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 93)) (22.2.1)
Requirement already satisfied: typing-extensions==4.3.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 94)) (4.3.0)
Requirement already satisfied: tzdata==2022.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 95)) (2022.1)
Requirement already satisfied: tzlocal==4.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 96)) (4.2)
Requirement already satisfied: urllib3==1.26.10 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 97)) (1.26.10)
Requirement already satisfied: uvicorn[standard]==0.18.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 98)) (0.18.2)
Requirement already satisfied: uvloop==0.16.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 99)) (0.16.0)
Requirement already satisfied: watchdog==2.1.9 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 100)) (2.1.9)
Requirement already satisfied: watchfiles==0.16.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 101)) (0.16.0)
Requirement already satisfied: wcwidth==0.2.5 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 102)) (0.2.5)
Requirement already satisfied: websockets==10.3 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 103)) (10.3)
Requirement already satisfied: whitenoise==6.2.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 104)) (6.2.0)
Requirement already satisfied: whoosh==2.7.4 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 105)) (2.7.4)
Requirement already satisfied: wrapt==1.14.1 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 106)) (1.14.1)
Requirement already satisfied: zope.interface==5.4.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 108)) (5.4.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/logging.py", line 177, in emit
    self.console.print(renderable, overflow="ignore", crop=False, style=style)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/rich/console.py", line 1673, in print
    extend(render(renderable, render_options))
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/rich/console.py", line 1305, in render
    for render_output in iter_render:
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/logging.py", line 134, in __rich_console__
    for line in lines:
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/rich/segment.py", line 249, in split_lines
    for segment in segments:
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/rich/console.py", line 1283, in render
    renderable = rich_cast(renderable)
  File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/rich/protocol.py", line 36, in rich_cast
    renderable = cast_method()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/self_outdated_check.py", line 130, in __rich__
    pip_cmd = get_best_invocation_for_this_pip()
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/entrypoints.py", line 58, in get_best_invocation_for_this_pip
    if found_executable and os.path.samefile(
  File "/usr/lib/python3.9/genericpath.py", line 101, in samefile
    s2 = os.stat(f2)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/pip3.9'
Call stack:
  File "/usr/local/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/main.py", line 70, in main
    return command.main(cmd_args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 101, in main
    return self._main(args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 223, in _main
    self.handle_pip_version_check(options)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/req_command.py", line 190, in handle_pip_version_check
    pip_self_version_check(session, options)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/self_outdated_check.py", line 236, in pip_self_version_check
    logger.warning("[present-rich] %s", upgrade_prompt)
  File "/usr/lib/python3.9/logging/__init__.py", line 1454, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/lib/python3.9/logging/__init__.py", line 1585, in _log
    self.handle(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 1595, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 1657, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 948, in handle
    self.emit(record)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/logging.py", line 179, in emit
    self.handleError(record)
Message: '[present-rich] %s'
Arguments: (UpgradePrompt(old='22.2.2', new='22.3.1'),)

[dimatx]

Was able to address the requirements 'error' (FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/pip3.9') by running "pip3 install --upgrade pip", but the original issue still persists.

[tteck]

I just installed Paperless-ngx v1.9.2
Then using the update script successfully updated to v1.10.2

[dimatx]

OK, running line by line, when I run Line 11 (pip install -r requirements.txt), the error seems related to @tomdh76 comments.... it is:

Collecting mysqlclient==2.1.1
  Downloading mysqlclient-2.1.1.tar.gz (88 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.1/88.1 kB 9.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      /bin/sh: 1: mysql_config: not found
      /bin/sh: 1: mariadb_config: not found
      /bin/sh: 1: mysql_config: not found
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-78tlo10n/mysqlclient_973d5759549341b8be9feed22e7f915f/setup.py", line 15, in <module>
          metadata, options = get_config()
        File "/tmp/pip-install-78tlo10n/mysqlclient_973d5759549341b8be9feed22e7f915f/setup_posix.py", line 70, in get_config
          libs = mysql_config("libs")
        File "/tmp/pip-install-78tlo10n/mysqlclient_973d5759549341b8be9feed22e7f915f/setup_posix.py", line 31, in mysql_config
          raise OSError("{} not found".format(_mysql_config_path))
      OSError: mysql_config not found
      mysql_config --version
      mariadb_config --version
      mysql_config --libs
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[tteck]

Try apt install default-libmysqlclient-dev

[dimatx]

Needed to run "apt-get install libmariadb-dev-compat" (one of @tomdh76's commands). Then ran the full script. It worked. Thanks again!

[tteck]

The update script should have installed that 🤔

if [ "$(dpkg -l | awk '/libmariadb-dev/ {print }'|wc -l)" != 2 ]; then apt-get install libmariadb-dev; fi &>/dev/null
if [ "$(dpkg -l | awk '/libmariadb-dev-compat/ {print }'|wc -l)" != 1 ]; then apt-get install libmariadb-dev-compat; fi &>/dev/null

[tteck]

forgot the -y tag
libmariadb-dev-compat adds libmariadb-dev
Should be fixed now