Message data corrupted with route protocol metadata [v2.12.4]

[superlevure]

Observed behavior

Messages pulled by JetStream consumers occasionally get corrupted: their data field is polluted with NATS route protocol metadata (account name, subject, reply-to inbox, header sizes).

Here's an extract of the corrupted data field from one such message:

default _INBOX.CF4pDsWxqdzVqqgMiqQgeJ.CF4pDsWxqdzVqqgMiqyuzE9c2a 79\r\n{"stream":"stream-name","seq":243179203}\r\nhiYC1ujaLYGGLQR0459 63 33288\r\nNATS/1.0\r\nNats-Msg-Id: d78a0eae-9114-4654-8308-95fdba25e0b7\r\n\r\n\n$d78a0eae-911

Another example in the same message:

\x0c\x10\x18N\xba\x1c\xa8&\x07\x88\xd2\x1e1\xa3\x80\x18z"\xe6\n\x00\x00\x01\x01\x08\n\xe5\xc1RVz-\xf8\xc7RMSG default _INBOX.CF4pDsWxqdzVqqgMiqQgeJ.CF4pDsWxqdzVqqgMiqzhQ9a2ae 79\r\n{"stream":"stream-name","seq":243179982}\r\nmand_ascii\x12\x08\x08\x02\x10\x0

Another example in a different message:

xfe\x9e\xb3\x19\xf2\xaaHMSG default subject.name _INBOX.pAx6wvYhiYC1ujaLYGGHkQ.pAx6wvYhiYC1ujaLYGGLQR0459 63 33288\r\nNATS/1.0\r\nNats-Msg-Id: d78a0eae-9114-4654-8308-95fdba25e0b7\r\n\r\n\n$d78a0eae-9114-4654-8308-95fdba25e0b7\x10\x01\x1a

The last example looks like a raw inter-server route HMSG frame:
<account> <subject> <reply-to> <header_len> <total_len>\r\n<headers>\r\n\r\n<payload>.

We also observe the following errors in the server logs:

[7] 2026/02/16 14:20:25.737892 [ERR] 10.36.19.14:57120 - cid:2244 - "v2.12.0:python3:consumer-name" -
 "default/user:default" - processPub Parse Error: "$JS.ACK.XXXX"
[7] 2026/02/16 14:20:26.610960 [ERR] 10.36.15.29:35066 - rid:156330 - processRoutedMsgArgs Parse Error: '[$NRG.R.Ka0xTZUu 25]'
[...]
[7] 2026/02/16 13:08:24.310818 [ERR] 10.36.15.29:40918 - rid:1684 - processRoutedHeaderMsgArgs Parse Error: '[default _INBOX.e8lEZecUcoy5szW4MQ0cok
  $JS.ACK.XXXX]'
[7] 2026/02/16 13:08:24.311522 [INF] 10.36.15.29:40918 - rid:1684 - Router connection closed: Protocol Violation - Remote: nats-1

For context, no WebSocket clients are involved here. We had to manually term the corrupted messages to get back to a stable state.

Expected behavior

• Messages delivered to consumers should contain only the original published payload
• Route connections between servers should not encounter parse errors or protocol violations

Server and client version

• nats server: v2.12.4
• nats client (Python): v2.12.0

Host environment

Kubernetes based deployment, using the official helm chart. 3-node NATS cluster.

Steps to reproduce

We haven't been able to reproduce this consistently. It occurs intermittently in our production cluster under normal load.

[MauriceVanVeen]

Could you share more about your setup? Do you use account imports/exports?

[superlevure]

Of course:

• We don't use account imports/exports
• This issue happened to us on server v2.10.26, v2.12.3 and v2.12.4 already
• Those consumers connect to NATS via TCP, not WebSocket
• The stream is replicated over our three nodes (R3)

Let me know what other type of information could be useful.

[wallyqs]

@superlevure do you have compression enabled in the routes, or leafnodes/gateways?

[MauriceVanVeen]

The $JS.ACK and subjects are redacted, but what do you use for the subjects? Do they contain any "interesting" symbols at all, like @ for example?

[superlevure]

erlevure do you have compression enabled in the routes, or leafnodes/gateways?

No we don't use compression, and no leafnodes/gateways either.

The $JS.ACK and subjects are redacted, but what do you use for the subjects? Do they contain any "interesting" symbols at all, like @ for example?

The subject looks like this a.b.c-d.e.f.a06f57d1-9f99-4990-9f55-6fcfeac2d1ab.g-h, it only contains letters, numbers, - and .. The total length is 105 characters.