GET /vhosts output contains empty list + stats

[WilliamDEdwards]

Sometimes, GET /vhosts contains either or both of the following objects:

• Empty list.
• Stats.

However, I would expect only vhost objects to be returned.

Is this a known bug (could not find an already reported bug on the mailing list nor on GitHub), or expected (could not find this in the documentation)?

Particularities: we create and delete several vhosts per second (CI environment).

Example:

[
  [],
  {
    "recv_oct": 16733,
    "recv_oct_details": {
      "rate": 1125
    },
    "send_oct": 25709,
    "send_oct_details": {
      "rate": 1750.4
    }
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "Default virtual host",
    "metadata": {
      "description": "Default virtual host",
      "tags": []
    },
    "name": "/",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "testing.cluster1-borg-gw0",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "testing.cluster1-gw0",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "stopped"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "testing.cluster2-borg-gw0",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "testing.cluster2-gw0",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu421-gw5",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu422-gw5",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu423-gw5",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu424-gw5",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "stopped"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu425-gw5",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu822-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu823-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu824-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu825-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu826-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu827-gw1",
    "tags": [],
    "tracing": false
  },
  {
    "cluster_state": {
      "rabbit@61493a239619": "running"
    },
    "description": "",
    "metadata": {
      "description": "",
      "tags": []
    },
    "name": "uu828-gw1",
    "tags": [],
    "tracing": false
  }
]

[michaelklishin]

This is an effect of metric cleanup, periodic stats emission, concurrent virtual host definitions, and the fact that in Erlang, an empty "object" and an empty list are historically the same thing.
In other words, stats are still represented as lists of tuples and the JSON generator cannot know which one do you mean to encode without explicit hints.

Virtual hosts are not meant to be created and deleted several times a second.

[WilliamDEdwards]

So the empty list represents stats for a virtual host that does not have stats yet because it was just created. But why does this stats object end up in the output of GET /vhosts? AFAICS, it is not documented, and it is only present sometimes.

[michaelklishin]

The only solution I can think of is filtering out such virtual hosts (that don't have any metrics emitted for them because they were created fractions of a second ago). Empty metric objects would not be very useful anyway.

Unfortunately, so far I cannot reproduce.

So consider reworking your tests to not churn through virtual hosts so much.

[WilliamDEdwards]

I have worked around this by skipping objects equal to [] or containing recv_oct. Thank you for the hint!