Upgrade from 3.13.x with Khepri enabled: exception exit: {{user_already_exists,<<"guest">>} #13853
-
Community Support Policy
RabbitMQ version used4.1.0 Erlang version used27.3.x Operating system (distribution) usedwindows server How is RabbitMQ deployed?Windows installer rabbitmq-diagnostics status outputStatus of node rabbit@nbpli-wcsatp1 ... OS PID: 1168 Plugins Enabled plugin file: c:/Users/mbillman/AppData/Roaming/RabbitMQ/enabled_plugins
Data directory Node data directory: c:/Users/mbillman/AppData/Roaming/RabbitMQ/db/rabbit@nbpli-wcsatp1-mnesia Config files
Log file(s)
Alarms (none) Tags (none) Memory Total memory used: 0.0867 gb other_proc: 0.0263 gb (30.35 %) File Descriptors Total: 0, limit: 65439 Free Disk Space Low free disk space watermark: 0.05 gb Totals Connection count: 0 Listeners Interface: [::], port: 15672, protocol: http, purpose: HTTP API Logs from node 1 (with sensitive values edited out)2025-05-05 11:46:29.257000-05:00 [info] <0.211.0> Starting RabbitMQ 4.1.0 on Erlang 27.3.3 [jit] Logs from node 2 (if applicable, with sensitive values edited out)No response Logs from node 3 (if applicable, with sensitive values edited out)No response rabbitmq.confn/a - using base config Steps to deploy RabbitMQ clusterused windows installers to upgrade Steps to reproduce the behavior in questionCannot get any user to log in... For instance: C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.0\sbin>rabbitmqctl authenticate_user qc r@bbitMQ C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.0\sbin>rabbitmqctl list_users C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.0\sbin>rabbitmqctl list_permissions advanced.configNo response Application codeNo response Kubernetes deployment fileNo response What problem are you trying to solve?Cannot get anything to log in...even when creating users. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 4 replies
-
These are pretty self-explanatory. There is a CLI command for troubleshooting authentication: If you use characters such as |
Beta Was this translation helpful? Give feedback.
-
|
I've tried all of those...as well as simplifying the change password to something very simple as follows: C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.0\sbin>rabbitmqctl change_password qc qc C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.0\sbin>rabbitmqctl authenticate_user qc qc |
Beta Was this translation helpful? Give feedback.
-
|
It works for everyone else, so I'm afraid it's something environment-specific and only you can investigate what might be going on. RabbitMQ logs when a user's credentials are updated. Enable debug logging, take a traffic capture to see if your CLI tools even connect to the node you think they connect. |
Beta Was this translation helpful? Give feedback.
-
|
2025-05-05 12:46:03.949000-05:00 [debug] <0.749.0> Will make sure that processes of 1 virtual hosts are running on all reachable cluster nodes What is strange is I upgraded multiple versions (old version on this test box), but no matter what I do, i can't get anything to log in - not even the default guest account |
Beta Was this translation helpful? Give feedback.
-
|
and now everytime i restart i get: 2025-05-05 12:51:59.160000-05:00 [error] <0.210.0> exception exit: {{user_already_exists,<<"guest">>}, |
Beta Was this translation helpful? Give feedback.
-
|
That likely means that the node assumes to be a blank node while it is not. Assuming that Mnesia is used, the logic for detecting it looks like this: it simply checks for Mnesia files on disk. For Khepri, a dedicated Khepri function is used, here is what it looks like. @qcsoftware513 which one do you use? Note that you can delete the Depending on how this node is used, you must as well I don't recall seeing something like this, and even the largest (many thousands of clusters) users haven't reported it, so my best guess is that something prevents the node from reading some of its files — a common problem on Windows with security tools — and that confuses it, leading it to seed the default data. |
Beta Was this translation helpful? Give feedback.
-
|
@qcsoftware513 we have figured out what happens. Khepri in 4.x has a completely different data model than in 3.13.x. In 3.13.x, it was an experimental feature flag, so upgrades from 3.13.x nodes with Khepri are only possible if you use a Blue Green deployment technique or (for development nodes) after wiping the node's data. |
Beta Was this translation helpful? Give feedback.
@qcsoftware513 we have figured out what happens.
Khepri in 4.x has a completely different data model than in 3.13.x. In 3.13.x, it was an experimental feature flag, so upgrades from 3.13.x nodes with Khepri are only possible if you use a Blue Green deployment technique or (for development nodes) after wiping the node's data.