Refactor HttpMessage into generalized templated types

[jschmidt-icinga]

This adds generalized IncomingHttpMessage and OutgoingHttpMessage templates that support different types of streams (via a std::variant) and can both be used for either requests or responses.

The tacked on metadata from the old HttpRequest and server connection from the old HttpServerConnection have been moved to HttpApi(Request|Response) classes that derive from the above generalized message types.

The implementation has been left in the source file so templating the classes doesn't blow up the compile times too much. This has the caveat that any new code that wants to use different instances of the template will have to add their instantiation to httpmessage.cpp, but I think it's better this way.

I wanted to do something like this initially in #10516, but as at that time nothing else would have used these classes and the PR was already huge I tried to reduce complexity.

This is useful for #10685, and maybe #10668. Since this doesn't have any functional changes it's better to have it here and review/merge it separately.

PS: I've split up the commits into one that only renames Http(Request|Response) to HttpApi(Request|Response) and one that adds the actual generalization of the underlying class, for easier review. I also suggest to turn on "Hide Whitespace" as it makes the diff much easier to read.

[yhabteab]

The implementation has been left in the source file so templating the classes doesn't blow up the compile times too much. This has the caveat that any new code that wants to use different instances of the template will have to add their instantiation to httpmessage.cpp, but I think it's better this way.

What does that mean for users that for example don't want to use boost::beast::multi_buffer as a buffer type for the body? I don't like the fact that this buffer type can return multiple buffers on prepare calls, which makes it harder to work with in my PR with zero-copy serialization. The beast flat_buffer on the other hand, always returns a continues memory region represented by a single buffer on prepare calls. So, would users that want to use flat_buffer instead of multi_buffer have to add their own instantiation of the OutgoingHttpMessage template to this file? That seems a bit inconvenient!

Is there a better way? I'm not saying we should move the implementation to the header file, but maybe we can find a middle ground?

[jschmidt-icinga]

What does that mean for users that for example don't want to use boost::beast::multi_buffer as a buffer type for the body? [...] So, would users that want to use flat_buffer instead of multi_buffer have to add their own instantiation of the OutgoingHttpMessage template to this file? That seems a bit inconvenient!

Yes, you then add another explicit instantiation to httpmessage(.hpp|.cpp). We're not writing a library here, we can afford to just add instantiations as needed. It amounts to one (for the explicit instantiation), two (if you add an extern template), or three (if you're also adding a typedef) lines per new type in a user's PR.

I'd add the definitions to the header, but I really think it might start impacting compile times as more and more other translation units include this header (either directly or transitively). And since we don't add new instances of this class all the time (like std::map or std::vector etc.) but probably just a couple if any in years to come, I think this is fine compromise.

The beast flat_buffer on the other hand, always returns a continues memory region represented by a single buffer on prepare calls.

🤦 I actually meant to use that in the generic OutgoingHttp(Response|Request), specifically because it makes the least assumptions and is the one you used it in your PR. Somehow I got confused, forgot and used multi_buffer there again. So I'll change the aforementioned types to use the flat_buffer instead and you don't have to add anything to your PR.

[yhabteab]

Awesome 🎉!

curl -ksSiu root:icinga 'https://localhost:5665/v1/config/files/example-cmdb/3a8d9807-6869-4512-b2e6-8e96b81e19a6/startup.log?pretty=1'

HTTP/1.1 200 OK
Server: Icinga/v2.15.0-231-g1505f09ed
Content-Type: application/octet-stream
Content-Length: 1898

[2026-01-23 09:19:51 +0100] information/cli: Icinga application loader (version: v2.15.0-231-g1505f09ed; debug)
[2026-01-23 09:19:51 +0100] information/cli: Loading configuration file(s).
[2026-01-23 09:19:51 +0100] warning/config: Ignoring directory '/Users/yhabteab/Workspace/icinga2/prefix/etc/icinga2/zones.d/master' for unknown zone 'master'.
[2026-01-23 09:19:51 +0100] warning/config: Ignoring directory '/Users/yhabteab/Workspace/icinga2/prefix/etc/icinga2/zones.d/satellite' for unknown zone 'satellite'.
[2026-01-23 09:19:51 +0100] warning/config: Ignoring directory '/Users/yhabteab/Workspace/icinga2/prefix/var/lib/icinga2/api/zones/master' for unknown zone 'master'.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Committing config item(s).
[2026-01-23 09:19:51 +0100] information/ApiListener: My API identity: mbp-yhabteab
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 ApiListener.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 9 CheckCommands.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 2 ApiUsers.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 Endpoint.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 CheckerComponent.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 2 Hosts.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 IcingaApplication.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 Zone.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 Service.
[2026-01-23 09:19:51 +0100] information/ConfigItem: Instantiated 1 NotificationComponent.
[2026-01-23 09:19:51 +0100] information/ScriptGlobal: Dumping variables to file '/Users/yhabteab/Workspace/icinga2/prefix/var/cache/icinga2/icinga2.vars'
[2026-01-23 09:19:51 +0100] information/cli: Finished validating the configuration file(s).

[julianbrost]

This is useful for #10685

Without having looked into this in more detail, I don't understand the motivation right away. Can you please explain how this change fits into the big picture?

[jschmidt-icinga]

Without having looked into this in more detail, I don't understand the motivation right away. Can you please explain how this change fits into the big picture?

@yhabteab Added a HttpRequestWriter class that does essentially the same as what was previously HttpResponse, just derived from boost::beast::http::request instead of response, plus two additional methods for accessing the buffer directly and minus the added metadata for API-requests.

Before this PR you had to essentially either use the exact same types as HttpServerConnection, or, like the perfdata writers and console commands, use bare bones boost::beast types, which would be a lot of boiler-plate when you want to use streaming, like in #10685.

Since the code for streaming does not distinguish between requests and responses, it makes sense to template the types the same way boost::beast::http::message<> is templated into response and requests. And then I derived from these generalized classes to tack on the metadata for the API types, that #10685 (or potentially others) doesn't need.

PS: personally I maybe wouldn't have merged it immediately and given you a chance to look at it too.

[yhabteab]

PS: personally I maybe wouldn't have merged it immediately and given you a chance to look at it too.

I wanted to do it too and let it open for around an 1hour after my approval but nobody cared to step up ;). Seriously, I wanted to answer Julian's question after I rebased my PR and make use of this, so that he can get the prove what this is useful for but thanks anyway. It's nothing that can't be undone, so 🤷!

[julianbrost]

So the otel writer is supposed to use chunked encoding (which we recently implemented for our REST API responses), so that was generalized to support it for both sending requests and responses. And otel probably allows plaintext HTTP as well, so the underlying stream type was made a template as well? Anything more that I've missed?

[yhabteab]

So the otel writer is supposed to use chunked encoding (which we recently implemented for our REST API responses), so that was generalized to support it for both sending requests and responses.

Yes. The Protobuf messages are streamed in chunks, so I had to add a kinda exact replica of the existing HttpRespone class in my PR for requests.

And otel probably allows plaintext HTTP as well, so the underlying stream type was made a template as well?

Not just otel but all the other writers too but our API requests/responses on the other hand use always TLS stream. But yes, that's exactly the reason.

Anything more that I've missed?

Nope, that pretty much summs it up.