Skip to content

🎨 💥 Change msg::combine to msg::overlay #764

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 3, 2025
Merged

🎨 💥 Change msg::combine to msg::overlay #764

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 47 additions & 11 deletions docs/message.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ using msg1_defn = extend<header_defn, "msg", type_f::with_required<1>, payload_f
using msg2_defn = extend<header_defn, "msg", type_f::with_required<2>, payload_f>;
----

==== Combining and packing messages
==== Overlaying and packing messages

It is sometimes useful to combine multiple message definitions, to avoid
repetition. For example, adding a payload message to a header:
Expand All @@ -176,13 +176,22 @@ using data_f = field<"data", std::uint32_t>::located<at{0_dw, 15_msb, 7_lsb}>;
using payload_defn = message<"payload", data_f>;

using msg_defn = extend<
combine<"msg", header_defn, payload_defn>,
overlay<"msg", header_defn, payload_defn>,
type_f::with_required<1>>;

// resulting message layout:
// byte |0 |1 |
// bit |01234567|01234567|
// field |header |payload |
Copy link
Contributor

@mjcaisse-intel mjcaisse-intel Sep 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: is data_f specified incorrectly above? Should it be:

using data_f = field<"data", std::uint32_t>::located<at{0_dw, 15_msb, 8_lsb}>;

optionally, if the intent is to show overlap the table can be rendered differently.

----

The combined definition incorporates all the fields of the messages. And as
shown, the combination might typically be `extend`​ed with a constraint on the
header field.
The resulting definition incorporates all the fields of the messages.
And as shown, the combination might typically be `extend`​ed with a constraint on
the header field.

NOTE: It is possible to have overlapping message fields! Fields just determine
which parts of the data are read/written, and overlapping field definitions are
sometimes useful.

Other times it is useful to automatically concatenate or `pack` messages
together, where the field locations in each message start at 0.
Expand All @@ -193,25 +202,24 @@ using type_f = field<"type", std::uint32_t>::located<at{0_dw, 5_msb, 0_lsb}>;
using header_defn = message<"header", type_f>;

// note: data_f collides with type_f under a naive combination
using data_f = field<"data">, std::uint32_t>::located<at{0_dw, 7_msb, 0_lsb}>;
using data_f = field<"data", std::uint32_t>::located<at{0_dw, 7_msb, 0_lsb}>;
using payload_defn = message<"payload", data_f>;

using msg_defn = extend<
pack<"msg", std::uint8_t, header_defn, payload_defn>,
type_f::with_required<1>>;

// resulting message layout:
// byte 0 1
// bit 01234567 01234567
// field |type|xx |data |
// byte |0 |1 |
// bit |012345|67|01234567|
// field |type |xx|data |
----

The second parameter to `pack` (`std::uint8_t` in the example above) defines how
the messages are packed together - in this case, each subsequent message is
byte-aligned.

CAUTION: After combining or packing messages, the fields inside them may have
moved!
CAUTION: After packing messages, the fields inside them may have moved!

Any matchers defined on the original fields may cause problems when matching
against raw data, because they will be looking in the wrong place. (Matching
Expand All @@ -238,6 +246,34 @@ using msg_defn = extend<
using new_data_f = msg_defn::field_t<"data">;
----

==== Relaxed messages

During prototyping, it can be useful to specify message types, but not worry
about where they are located yet. The compiler can automatically place them in
storage for us, and this is what `relaxed_message` is for.

[source,cpp]
----
// just prototyping: we want field types, but we don't care about layout yet
using type_f = field<"type", std::uint8_t>;
using data_f = field<"data", std::uint32_t>;
using msg_defn = relaxed_message<"msg", type_f, data_f>;

// msg_defn has both fields, at unspecified locations
----

[source,cpp]
----
// we want to fix the type, but we don't care about the rest
using type_f = field<"type", std::uint8_t>::located<at{0_dw, 3_msb, 0_lsb}>;
using field0_f = field<"f0", std::uint8_t>;
using field1_f = field<"f1", std::uint16_t>;
using field2_f = field<"f2", std::uint32_t>;
using msg_defn = relaxed_message<"msg", type_f, field0_f, field1_f, field2_f>;

// msg_defn has type as the first 4 bits; the other fields are at unspecified locations
----

==== Owning vs view types

An owning message uses underlying storage: by default, this is a `std::array` of
Expand Down
12 changes: 6 additions & 6 deletions include/msg/message.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -699,26 +699,26 @@ using shifted_msgs =
msg_offsets<AlignTo, Msgs...>,
stdx::type_list<Msgs..., msg::message<"end">>>;

template <stdx::ct_string Name, stdx::envlike Env> struct combiner {
template <stdx::ct_string Name, stdx::envlike Env> struct overlayer {
template <typename... Fields> using fn = msg::message<Name, Env, Fields...>;
};

template <stdx::ct_string Name> struct combine_q {
template <stdx::ct_string Name> struct overlay_q {
template <typename... Msgs>
requires(sizeof...(Msgs) > 0)
using fn = boost::mp11::mp_apply_q<
combiner<Name, stdx::append_env_t<typename Msgs::env_t...>>,
overlayer<Name, stdx::append_env_t<typename Msgs::env_t...>>,
boost::mp11::mp_append<typename Msgs::fields_t...>>;
};
} // namespace detail

template <stdx::ct_string Name, typename... Msgs>
requires(sizeof...(Msgs) > 0)
using combine = typename detail::combine_q<Name>::template fn<Msgs...>;
using overlay = typename detail::overlay_q<Name>::template fn<Msgs...>;

template <stdx::ct_string Name, typename AlignTo, typename... Msgs>
requires(sizeof...(Msgs) > 0)
using pack = boost::mp11::mp_apply_q<detail::combine_q<Name>,
using pack = boost::mp11::mp_apply_q<detail::overlay_q<Name>,
detail::shifted_msgs<AlignTo, Msgs...>>;

namespace detail {
Expand All @@ -739,7 +739,7 @@ struct field_locator<Name, Env, Fields...> {
using unlocated_fields = boost::mp11::mp_second<fields>;

using located_msg =
boost::mp11::mp_apply_q<combiner<Name, stdx::env<>>, located_fields>;
boost::mp11::mp_apply_q<overlayer<Name, stdx::env<>>, located_fields>;

using auto_fields =
boost::mp11::mp_sort<unlocated_fields, field_size_sort_fn>;
Expand Down
12 changes: 6 additions & 6 deletions test/msg/message.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -614,7 +614,7 @@ TEST_CASE("shift all fields in a message", "[message]") {
STATIC_REQUIRE(std::is_same_v<defn, expected_defn>);
}

TEST_CASE("combine messages", "[message]") {
TEST_CASE("overlay messages", "[message]") {
using f1 = field<"f1", std::uint32_t>::located<at{23_msb, 16_lsb}>;
using f2 = field<"f2", std::uint32_t>::located<at{15_msb, 0_lsb}>;
using m1 = message<"m1", f1, f2>;
Expand All @@ -623,19 +623,19 @@ TEST_CASE("combine messages", "[message]") {
using f4 = field<"f4", std::uint32_t>::located<at{15_msb, 0_lsb}>;
using m2 = message<"m2", f3, f4>;

using defn = combine<"defn", m1, m2::shifted_by<1, std::uint32_t>>;
using defn = overlay<"defn", m1, m2::shifted_by<1, std::uint32_t>>;
using expected_defn =
message<"defn", f1, f2, f3::shifted_by<1, std::uint32_t>,
f4::shifted_by<1, std::uint32_t>>;
STATIC_REQUIRE(std::is_same_v<defn, expected_defn>);
}

TEST_CASE("combine 1 message", "[message]") {
TEST_CASE("overlay 1 message", "[message]") {
using f1 = field<"f1", std::uint32_t>::located<at{15_msb, 0_lsb}>;
using f2 = field<"f2", std::uint32_t>::located<at{23_msb, 16_lsb}>;
using m1 = message<"m1", f1, f2>;

using defn = combine<"defn", m1>;
using defn = overlay<"defn", m1>;
using expected_defn = message<"defn", f1, f2>;
STATIC_REQUIRE(std::is_same_v<defn, expected_defn>);
}
Expand Down Expand Up @@ -771,14 +771,14 @@ TEST_CASE("supplement message environment", "[message]") {
STATIC_REQUIRE(custom(new_defn::env_t{}) == 18);
}

TEST_CASE("combine appends environments", "[message]") {
TEST_CASE("overlay appends environments", "[message]") {
using env1_t = stdx::make_env_t<custom, 17>;
using m1 = message<"m1", env1_t>;

using env2_t = stdx::make_env_t<custom, 18>;
using m2 = message<"m2", env2_t>;

using defn = combine<"defn", m1, m2>;
using defn = overlay<"defn", m1, m2>;
STATIC_REQUIRE(custom(defn::env_t{}) == 18);
}

Expand Down
Loading