Request-Id header

[chriskapp]

Hi,

so our system currently issues an "X-Request-Id" which is basically a simple UUID 4 which we add as header to every response. Based on this request id we can get details about a specific request which can be used for debugging etc. I would like to gather some feedback whether it would be useful to create a simple RFC where we specify such a "Request-Id" header. Our system uses the "X-Request-Id" but there are also other versions i.e. Github uses "X-Github-Request-Id". So please let me know your thoughts, Iam not sure whether it is too simple for an RFC but on the other hand I could imagine that for some tooling it would be handy to have such a standard.

[richsalz]

One thing you might want to consider: a request comes in, goes through a CDN for some processing, and then ultimately goes to the origin. Do we need to tag multiple 'requests'?

[richsalz]

Thanks for the pointer. Your draft might want to say, basically, what you just wrote above.

[chriskapp]

Thanks for the response and the pointer to the spec, I was not aware of it. I have skimmed though the spec and it looks to me, that it is targeted to a debug use case where you can activate tracing and then the server handles those headers i.e. for a complex microservice architecture. The idea with this header is more that every response also on a production API always returns this id so that actual end users i.e. can get this request id and ask our support what happened with this request. So is my understanding of the spec correct or could it also handle this case? Otherwise regarding the hopping case I would suggest that only the origin system generates such a request id and all other services simply redirect the id.

[asbjornu]

Although Trace Context can be used in a complex microservice deployment scenario with several intermediary hops between the client and origin server, I believe it can also be used to serve your simple use-case @chriskapp.

What is nice about Trace Context is that if you start out simple, you can easily extend its use once your app becomes more complex. If you, for instance, want to start placing a message on a queue for every HTTP request, the Trace Context can carry over to the queue protocol and be picked up by the consuming party at the other end.

[cjaccino]

I'm happy to have learned of Trace Context, but I have a few concerns.

• It seems quite a bit more complicated than other HTTP headers. It even has a W3C-managed registry!
• The W3C isn't registering these new headers with IANA. Turf wars maybe? Whatever the SOP might be, we would have better outcomes if the W3C would contribute headers and other common protocol elements to IANA's registries, where we can see them in one place.

My biggest concern is that simplicity is important for use cases I'm thinking of, and Trace Context seems to have gone a bit beyond. It seems that it would functionally work. A firm's API gateway might apply these headers when a request is received and apply the appropriate handling internally, enabling it to track its request through its own systems. But for the inter-firm scenario, it may appear to be unnecessary complexity; an API provider organization will be concerned with the marketability of its APIs.

Some scenarios:

• API provider would like to provide an optional correlator in a Correlator header. If the API consumer does not use it, one will be created and provided back. This supports the ability for the API consumer to extend its own correlation functionality into the provider organization for end-to-end tracing. There may be some guidelines for randomness and length, etc.
• Beyond the response being provided synchronously for the same request, API consumer may want something to validate that the response really is for its request. Perhaps it is concerned about mishandling within or compromise of the origin server. The correlator provides weak, but valuable, assurance that does not exist without a correlator.
• The previous point becomes more important in HTTP/2 multiple-response scenarios; the correlator may be used to identify that the multiple responses were for the same request.
• In an asynchronous response (webhooks, etc), the correlator might be used to match requests with anticipated responses (202 Accept scenarios).
• For secure handover of handling of responses, a requestor's correlator might be cryptographically generated, making it possible for a trusted alternate response handler to accept messages based on their ability to cryptographically validate a request.
• Multiple parties may be involved in the handling of the transaction, each with their own need to produce a correlator.

To satisfy the need for simplicity while enabling the use cases, I look to the Via header for inspiration. Each intermediary can add a Via header, and the header can be collapsed.

I'm avoiding "Request-Id" not to challenge the name, but to avoid confusion in the thread.

Correlator: S2Qojn5Jc7kdZU28UDUiGX3m
Correlator: OkC0s/xeEMBl7tmrTrYVtw9D
Correlator: KUwtEZvNO/XTp2XfVCu5RBtjy5eK2ArZ

Can be
Correlator: S2Qojn5Jc7kdZU28UDUiGX3m, OkC0s/xeEMBl7tmrTrYVtw9D, KUwtEZvNO/XTp2XfVCu5RBtjy5eK2ArZ

Intermediaries can annotate other headers by appending a delimiter and extending them, but not changing them.

Correlator: S2Qojn5Jc7kdZU28UDUiGX3m
Correlator: S2Qojn5Jc7kdZU28UDUiGX3m; Good morning Dave

Does it seem like this is a different case?

[darrelmiller]

While the W3C Distributed Tracing is fairly sophisticated, the reasons for that are not artificial. The specification was heavily influenced by the practical requirements of commercial application performance management products. If you dig under the covers of the OpenTelemetry effort https://opentelemetry.io/ you will see that many of the collectors rely on the W3C Tracing specification. Having open source standardized libraries that take advantage of this W3C standard means that many people can take advantage of the standard without having to know the underlying mechanics.

For scenarios like distributed tracing where many network components and connectors need to agree on a common mechanism, to get any value, the adoption of the tech is more important than the engineering elegance. Considering the traction that W3C Distributed Tracing and OpenTelemetry have, I would be reluctant at this point to try and create something better. Looking at the registry of implementations gives a hint to the adoption success https://opentelemetry.io/registry/