boot-time definition import deletes unspecified queues

[lc-guy]

Documentation says:

Definition import happens after plugin activation. This means that definitions related to plugins (e.g. dynamic Shovels, exchanges of a custom type, and so on) can be imported at boot time.

The definitions in the file will not overwrite anything already in the broker.

My definitions file specifies a vhost, some queues, and some bindings.

If I manually create a new queue in that vhost, then restart rabbitmq (thus doing the boot-time definition import again), the queue I just added is now gone, which is contrary to the behavior the documentation specifies.

[lukebakken]

Hello,

For us to assist we'll have to reproduce this issue. Can you provide the following:

• The definitions file you're using
• Exact steps you're using to "manually create a new queue"

My guess is that you are not creating a durable queue, in which case restarting RabbitMQ will delete it.

Thanks!

[michaelklishin]

I cannot reproduce.

With a reset node and two virtual hosts, one user that has permissions to both and the following
definitions file

{
  "rabbit_version": "3.11.9",
  "rabbitmq_version": "3.11.9",
  "product_name": "RabbitMQ",
  "product_version": "3.11.9",
  "users": [
    {
      "name": "username1",
      "password_hash": "sy/lmee4hXXMhVo3Hhgxfdh+CzdKn6jk+TqmKrEeT4WCKr+L",
      "hashing_algorithm": "rabbit_password_hashing_sha256",
      "tags": [
        "monitoring"
      ],
      "limits": {}
    },
    {
      "name": "guest",
      "password_hash": "YHcoYk8MdM+WUQwI5nM/kjTAp8yHKCSlH1/HrboiCS2vm7aW",
      "hashing_algorithm": "rabbit_password_hashing_sha256",
      "tags": [
        "administrator"
      ],
      "limits": {}
    },
    {
      "name": "username2",
      "password_hash": "hVjeyjreCuxLEF9wukBH8QA1oz1fUzNIRNwGnpwdvdWp4Yeh",
      "hashing_algorithm": "rabbit_password_hashing_sha256",
      "tags": [
        "policymaker"
      ],
      "limits": {}
    }
  ],
  "vhosts": [
    {
      "name": "vh2"
    },
    {
      "name": "/"
    },
    {
      "name": "vh1"
    },
    {
      "name": "vh3"
    }
  ],
  "permissions": [
    {
      "user": "username1",
      "vhost": "vh2",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "guest",
      "vhost": "/",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "guest",
      "vhost": "vh2",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "guest",
      "vhost": "vh1",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "guest",
      "vhost": "vh3",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "username1",
      "vhost": "vh1",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    },
    {
      "user": "username2",
      "vhost": "vh3",
      "configure": ".*",
      "write": "^$",
      "read": ".*"
    }
  ],
  "topic_permissions": [],
  "parameters": [],
  "global_parameters": [
    {
      "name": "internal_cluster_id",
      "value": "rabbitmq-cluster-id-efXejVWxQC1GN1HYrHeI1g"
    }
  ],
  "policies": [],
  "queues": [],
  "exchanges": [],
  "bindings": []
}

I start a node with the following config file

load_definitions = /path/to/definitions_2023-3-1.json

and after node boot I see definition import traces in the log:

2023-03-01 09:00:53.871354+04:00 [info] <0.640.0> Asked to import definitions. Acting user: rmq-internal
2023-03-01 09:00:53.871426+04:00 [info] <0.640.0> Importing concurrently 3 users...
2023-03-01 09:00:53.873082+04:00 [info] <0.628.0> Created user 'username1'
2023-03-01 09:00:53.873090+04:00 [info] <0.630.0> Created user 'username2'
2023-03-01 09:00:53.873098+04:00 [info] <0.628.0> Successfully set user tags for user 'username1' to [monitoring]
2023-03-01 09:00:53.873110+04:00 [info] <0.630.0> Successfully set user tags for user 'username2' to [policymaker]
2023-03-01 09:00:53.873152+04:00 [info] <0.640.0> Importing concurrently 4 vhosts...
2023-03-01 09:00:53.873301+04:00 [info] <0.630.0> Adding vhost 'vh1' (description: '', tags: [])
2023-03-01 09:00:53.873334+04:00 [info] <0.628.0> Adding vhost 'vh2' (description: '', tags: [])
2023-03-01 09:00:53.873354+04:00 [info] <0.631.0> Adding vhost 'vh3' (description: '', tags: [])
2023-03-01 09:00:53.873989+04:00 [info] <0.631.0> Applying default limits to vhost '<<"vh3">>': []
2023-03-01 09:00:53.873992+04:00 [info] <0.630.0> Applying default limits to vhost '<<"vh1">>': []
2023-03-01 09:00:53.874044+04:00 [info] <0.628.0> Applying default limits to vhost '<<"vh2">>': []

but all originally created users, virtual hosts and permissions are still there.

[michaelklishin]

rabbitmqctl list_vhosts --formatter=pretty_table
Listing vhosts ...
┌───────────────────┐
│ name              │
├───────────────────┤
│ manually-created1 │
├───────────────────┤
│ vh2               │
├───────────────────┤
│ manually-created2 │
├───────────────────┤
│ /                 │
├───────────────────┤
│ vh1               │
├───────────────────┤
│ vh3               │
└───────────────────┘

rabbitmqctl list_users --formatter=pretty_table
Listing users ...
┌──────────────────┬───────────────┐
│ user             │ tags          │
├──────────────────┼───────────────┤
│ username1        │ monitoring    │
├──────────────────┼───────────────┤
│ guest            │ administrator │
├──────────────────┼───────────────┤
│ username2        │ policymaker   │
├──────────────────┼───────────────┤
│ manually-created │ monitoring    │
└──────────────────┴───────────────┘

[michaelklishin]

I cannot find anything related to deletions in the rabbit_definitions module.

My best guess is that you have declared some of your entities as transient and they do not survive broker restart. That said, virtual hosts and users can only be "transient" if you use RAM nodes,
a very rarely used and deprecated feature.

[lc-guy]

The issue is related to the container image I was using (bitnami/rabbitmq). Sorry for the inconvenience and thanks for the help, I'll check whether the issue is downstream first next time.

[lukebakken]

Did you open an issue to which we can link here?

[lc-guy]

Yes, at bitnami/containers#25799

[lukebakken]

Thanks!