Hardening: Normalize Content-Length headers to remove leading zeros for downstream compatibility

[sivaadityacoder]

Problem: Envoy currently accepts Content-Length headers containing leading zeros (e.g., Content-Length: 010) and forwards the raw string value to upstream clusters without normalization.

Context & Impact: While Envoy correctly parses 010 as decimal 10, various downstream backends (specifically those using legacy C strtol with base 0, or older Python parsers) may interpret leading zeros as Octal indicators.

Envoy: 010 -> 10 bytes.

Backend: 010 -> 8 bytes.

This ambiguity creates a potential desync/smuggling vector (CL.CL) when Envoy is placed in front of such vulnerable backends.

RFC Reference: RFC 9110 (and historically RFC 7230 Section 3.3.2) states that a sender MUST NOT send a Content-Length header field value that contains a leading zero unless the value is 0. As Envoy acts as the sender to the upstream, forwarding the raw value preserves this ambiguity.

Proposed Solution: Envoy should normalize the Content-Length header by stripping leading zeros (converting 010 to 10) before forwarding it to the upstream. This ensures that the upstream receives a canonical decimal integer, eliminating the risk of octal misinterpretation.

Reproduction: I have a reproduction script (vulnerable_backend.py) that demonstrates this desync against a Gunicorn-style backend if needed for verification.

[ravenblackx]

As previously discussed in the related security thread, the given RFC references are false, those sections don't say that, nor as far as I could tell do any other sections.

The most relevant genuine RFC quote to this issue is RFC 8941 section 3.3.1 which says

While it is possible to serialize Integers with leading zeros (e.g., "0002", "-01") and signed zero ("-0"), these distinctions may not be preserved by implementations.

Which gives envoy permission to normalize, but clearly no requirement nor even a preference.

My suggestion here is that at the very least if we were to implement this it should be a configurable behavior, because altering a request when not required to do so tends to be surprising (i.e. if an upstream does mishandle leading-zero content-length requests, it would be surprising for that behavior to change on the introduction of a proxy that isn't explicitly tasked with modifying the request).

Currently envoy isn't doing anything incorrect according to the spec, only the hypothetical upstream is. But if envoy were to normalize this header it still wouldn't be doing anything incorrect.

I would also suggest that if we were to normalize this integer in a header, we should under the same circumstances in the same way normalize all similar integer values (e.g. the integers in range and content-range headers)

[yanavlasov]

I agree with @ravenblackx that there is no prohibition of leading 0s in HTTP standard. We can add an option to strip leading 0s from content-length at least, since it is the sensitive header, but could also add the same behavior for other known integer headers.

[ravenblackx]

My additional thought on this is that we should probably put this sort of thing in an http filter extension, rather than adding more cluttery config in non-obvious places (otherwise would it be part of http common config, part of a codec config, part of the listener config?)

Regardless of which one, putting it into any common config would slow down the proxy for all users by at least one additional check per request, vs. putting it in a filter would only have any performance impact for people who explicitly configure that filter.

Having an "optional http sanitizations for poorly behaved upstreams" filter as a common place to put things like this seems like a sensible compromise (vs. having one filter for each optional sanitization would also be okay but would involve a lot of boilerplate).

(Other reasonable optional sanitizations suitable for such a filter that I'm aware of include rejecting percent-encoded invalid utf8 in the path, and rejecting headers that upset the grpc library, e.g. headers ending with -bin that aren't either valid-ish¹ base64 or prefixed by a null-byte)

Footnotes

1. grpc accepts base64 with omitted padding bytes, which is not technically valid base64. ↩

[yanavlasov]

It is sure is a very marginal feature and requires a server implementation that is troublesome to begin with, even without Envoy in the picture. Also crosses into WAF territory. I would also lean toward either an extension with this sort of edge case modification or even a LUA filter if upstream has this problem.