Add endian adaptors and allow setting default stream byte order (#19)

Author: Chaosvex

README.md

@@ -59,7 +59,7 @@ auto serialise(const UserPacket& packet) {
     return buffer;
 }
 ```
-By default, Hexi will try to serialise basic structures such as our `UserPacket` if they meet requirements for being safe to directly copy the bytes. Now, for reasons of portability, it's not recommended that you do things this way unless you're positive that the data layout is identical on the system that wrote the data. Not to worry, this is easily solved. Plus, we didn't do any error handling. All in good time.
+By default, Hexi will try to serialise basic structures such as our `UserPacket` if they meet requirements for being safe to directly copy the bytes. Now, for reasons of portability, it's not recommended that you do things this way unless you're positive that the data layout is identical on the system that wrote the data. Not to worry, this is easily solved. Plus, we didn't do any error or endianness handling. All in good time.
 
 <img src="docs/assets/frog-remember.png" alt="Remember these two classes, if nothing else!">
 
@@ -146,15 +146,15 @@ struct UserPacket {
     std::string username;
     uint64_t timestamp;
     uint8_t has_optional_field;
-    uint32_t optional_field;  // pretend this is big endian in the protocol
+    uint32_t optional_field;  // pretend this is big-endian in the protocol
 
     // deserialise
     auto& operator>>(auto& stream) {
         stream >> user_id >> username >> timestamp >> has_optional_field;
 
         if (has_optional_field) {
-            stream >> optional_field;
-            hexi::endian::big_to_native_inplace(optional_field);
+			// fetch explicitly as big-endian value
+            stream >> hexi::endian::from_big(optional_field);
         }
 
         // we can manually trigger an error if something went wrong
@@ -167,7 +167,8 @@ struct UserPacket {
         stream << user_id << username << timestamp << has_optional_field;
 
         if (has_optional_field) {
-            stream << hexi::endian::native_to_big(optional_field);
+			// write explicitly as big-endian value
+            stream << hexi::endian::to_big(optional_field);
         }
 
         return stream;
@@ -194,7 +195,14 @@ void read() {
 
 auto handle_user_packet(std::span<const char> buffer) {
     hexi::buffer_adaptor adaptor(buffer);
-    hexi::binary_stream stream(adaptor);
+
+	/**
+	 * hexi::endian::little tells the stream to convert to/from
+	 * little-endian unless told otherwise by using the endian
+	 * adaptors. If no argument is provided, it does not perform
+	 * any conversions by default.
+	 */
+    hexi::binary_stream stream(adaptor, hexi::endian::little);
 
     UserPacket packet;
     stream >> packet;
@@ -208,10 +216,21 @@ auto handle_user_packet(std::span<const char> buffer) {
 }
 ```
 
-Because `binary_stream` is a template, it's easiest to allow the compiler to perform type deduction magic.
+This example is fully portable and is even independent of the platform byte order. By specifying the endianness of the stream, it'll automagically convert all endian-sensitive data to the requested byte order.
+The default argument is `hexi::endian::native`, which will perform no conversions, while `hexi::endian::big` and `hexi::endian::little` will perform conversions if required.
+
+If your protocol contains mixed endianness, you can use the endian adaptors to specify the desired byte order when streaming
+the data, as shown in the above example. 
+
+Best of all, because this is handled by templates, there is zero runtime cost if no conversion is required
+(i.e. that is the native byte order matches the requested byte order) and constant values can be converted
+at compile-time. For example, specifying `hexi::endian::little` on a little-endian platform will generate zero
+code. 
+
+`docs/examples/endian.cpp` provides examples for byte order handling functionality.
 
-If you want the function bodies to be in a source file, it's recommended that you provide your own `using` alias for your `binary_stream` type.
-The alternative is to use the polymorphic equivalents, `pmc::buffer_adaptor` and `pmc::binary_stream`, which allow you to change the underlying buffer type at runtime but at the cost of virtual call overhead and lacking some functionality that doesn't mesh well with polymorphism.
+As for the serialisation functions, if you want the function bodies to be in a source file, it's recommended that you provide your own `using` alias for your `binary_stream` type.
+The alternative is to use the polymorphic equivalents, `pmc::buffer_adaptor` and `pmc::binary_stream`, which allow you to change the underlying buffer type at runtime but at the potential cost of virtual call overhead (devirtualisation not withstanding) and lacking some functionality that doesn't mesh well with polymorphism.
 
 How you structure your code is up to you, this is just one way of doing it.
 

docs/examples/endian.cpp

@@ -7,35 +7,54 @@
 int main() {
 	std::vector<char> buffer;
 	hexi::buffer_adaptor adaptor(buffer);
-	hexi::binary_stream stream(adaptor);
+	hexi::binary_stream stream(adaptor); // performs no conversions by default
+	
+	{
+		hexi::binary_stream be_stream(adaptor, hexi::endian::big);    // will convert to/from big endian by default
+		hexi::binary_stream be_stream(adaptor, hexi::endian::little); // will convert to/from little endian by default
+		hexi::binary_stream be_stream(adaptor, hexi::endian::native); // default, will not convert by default
+	}
 
-	{ // serialise foo & bar as big/little endian
+	{ // serialise foo & bar as big/little endian with put
 		const std::uint64_t foo = 100;
 		const std::uint32_t bar = 200;
-		stream.put<hexi::endian::conversion::native_to_big>(foo);
-		stream.put<hexi::endian::conversion::native_to_little>(bar);
+		stream.put<hexi::endian::to_big>(foo);
+		stream.put<hexi::endian::to_little>(bar);
 	}
 
 	{ // deserialise foo & bar as big/little endian
 		std::uint64_t foo = 0;
 
 		// write to existing variable or return result
-		stream.get<hexi::endian::conversion::big_to_native>(foo);
+		stream.get<hexi::endian::to_big>(foo);
 		std::ignore = stream.get<std::uint32_t, hexi::endian::conversion::little_to_native>();
 	}
 
 	{ // stream integers as various endian combinations
-		stream << hexi::endian::native_to_big(9000);
-		stream << hexi::endian::big_to_native(9001); // over 9000
+		stream << hexi::endian::to_big(9000);
+		stream << hexi::endian::to_little(9001); // over 9000
 		stream << hexi::endian::native_to_little(9002);
 		stream << hexi::endian::little_to_native(9003);
 	}
 
+	{ // retrieve stream integers as big or little endian
+		std::uint64_t foo;
+		stream >> hexi::endian::from_big(foo);
+		stream >> hexi::endian::from_little(foo);
+	}
+
 	{ // convert endianness inplace
 		int foo = 10;
 		hexi::endian::native_to_big_inplace(foo);
 		hexi::endian::big_to_native_inplace(foo);
 		hexi::endian::little_to_native_inplace(foo);
 		hexi::endian::native_to_little_inplace(foo);
 	}
+
+	{ // retrieve converted value
+		auto foo = hexi::endian::native_to_big_inplace(1);
+		auto bar = hexi::endian::big_to_native_inplace(2);
+		auto baz = hexi::endian::little_to_native_inplace(3);
+		auto qux = hexi::endian::native_to_little(4);
+	}
 }

include/hexi/binary_stream.h

@@ -36,7 +36,11 @@ using namespace detail;
 		}                                                         \
 	}
 
-template<byte_oriented buf_type, std::derived_from<except_tag> exceptions = allow_throw_t>
+template<
+	byte_oriented buf_type,
+	std::derived_from<except_tag> exceptions = allow_throw_t,
+	std::derived_from<endian::storage_tag> endianness = endian::as_native_t
+>
 class binary_stream final {
 public:
 	using size_type          = typename buf_type::size_type;
@@ -109,11 +113,23 @@ class binary_stream final {
 		: buffer_(source),
 		  read_limit_(read_limit) {};
 
+	explicit binary_stream(buf_type& source, exceptions)
+		: binary_stream(source, 0) {}
+
+	explicit binary_stream(buf_type& source, endianness)
+		: binary_stream(source, 0) {}
+
+	explicit binary_stream(buf_type& source, exceptions, endianness)
+		: binary_stream(source, 0) {}
+
 	explicit binary_stream(buf_type& source, size_type read_limit, exceptions)
 		: binary_stream(source, read_limit) {}
 
-	explicit binary_stream(buf_type& source, exceptions)
-		: binary_stream(source, 0) {}
+	explicit binary_stream(buf_type& source, size_type read_limit, endianness)
+		: binary_stream(source, read_limit) {}
+
+	explicit binary_stream(buf_type& source, size_type read_limit, exceptions, endianness)
+		: binary_stream(source, read_limit) {}
 
 	binary_stream(binary_stream&& rhs) noexcept
 		: buffer_(rhs.buffer_), 
@@ -136,9 +152,22 @@ class binary_stream final {
 		return data.operator<<(*this);
 	}
 
+	template<std::derived_from<endian::adaptor_in_tag_t> endian_func>
+	binary_stream& operator<<(endian_func adaptor) requires writeable<buf_type> {
+		const auto converted = adaptor.convert();
+		write(&converted, sizeof(converted));
+		return *this;
+	}
+
+	binary_stream& operator<<(const arithmetic auto& data) requires writeable<buf_type> {
+		const auto converted = endian::storage_in(data, endianness{});
+		write(&converted, sizeof(converted));
+		return *this;
+	}
+
 	template<pod T>
-	requires (!has_shl_override<T, binary_stream>)
-	binary_stream& operator <<(const T& data) requires writeable<buf_type> {
+	requires (!has_shl_override<T, binary_stream> && !arithmetic<T>)
+	binary_stream& operator<<(const T& data) requires writeable<buf_type> {
 		write(&data, sizeof(T));
 		return *this;
 	}
@@ -221,10 +250,10 @@ class binary_stream final {
 	 * 
 	 * @param data The element to be written to the stream.
 	 */
-	template<endian::conversion conversion>
-	void put(const arithmetic auto& data) requires writeable<buf_type> {
-		const auto swapped = endian::convert<conversion>(data);
-		write(&swapped, sizeof(data));
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	void put(const endian_func& adaptor) requires writeable<buf_type> {
+		const auto swapped = adaptor.convert();
+		write(&swapped, sizeof(swapped));
 	}
 
 	/**
@@ -357,8 +386,23 @@ class binary_stream final {
 		return data.operator>>(*this);
 	}
 
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	binary_stream& operator>>(endian_func adaptor) requires writeable<buf_type> {
+		STREAM_READ_BOUNDS_ENFORCE(sizeof(adaptor.value), *this);
+		buffer_.read(&adaptor.value, sizeof(adaptor.value));
+		adaptor.value = adaptor.convert();
+		return *this;
+	}
+
+	binary_stream& operator>>(arithmetic auto& data) requires writeable<buf_type> {
+		STREAM_READ_BOUNDS_ENFORCE(sizeof(data), *this);
+		buffer_.read(&data, sizeof(data));
+		endian::storage_out(data, endianness{});
+		return *this;
+	}
+
 	template<pod T>
-	requires (!has_shr_override<T, binary_stream>)
+	requires (!has_shr_override<T, binary_stream> && !arithmetic<T>)
 	binary_stream& operator>>(T& data) {
 		STREAM_READ_BOUNDS_ENFORCE(sizeof(data), *this);
 		buffer_.read(&data, sizeof(data));
@@ -394,11 +438,11 @@ class binary_stream final {
 	 * 
 	 * @param The destination for the read value.
 	 */
-	template<endian::conversion conversion>
-	void get(arithmetic auto& dest) {
-		STREAM_READ_BOUNDS_ENFORCE(sizeof(dest), void());
-		buffer_.read(&dest, sizeof(dest));
-		dest = endian::convert<conversion>(dest);
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	void get(endian_func& adaptor) {
+		STREAM_READ_BOUNDS_ENFORCE(sizeof(adaptor.value), void());
+		buffer_.read(&adaptor.value, sizeof(adaptor));
+		adaptor.value = adaptor.convert();
 	}
 
 	/**

include/hexi/endian.h

@@ -131,4 +131,57 @@ constexpr auto convert(arithmetic auto value) -> decltype(value) {
 	};
 }
 
+struct adaptor_in_tag_t {};
+struct adaptor_out_tag_t {};
+
+#define ENDIAN_ADAPTOR(name, func, tag, ref) \
+template<arithmetic T>                       \
+struct name final : tag {                    \
+	T ref value;                             \
+    name(T ref t) : value(t) {}              \
+	auto convert() -> T {                    \
+		return func(value);                  \
+	}                                        \
+};
+
+#define ENDIAN_ADAPTOR_OUT(name, func) ENDIAN_ADAPTOR(name, func, adaptor_out_tag_t, &)
+#define ENDIAN_ADAPTOR_IN(name, func)  ENDIAN_ADAPTOR(name, func, adaptor_in_tag_t,   )
+
+ENDIAN_ADAPTOR_IN(to_big,       native_to_big)
+ENDIAN_ADAPTOR_IN(to_little,    native_to_little)
+ENDIAN_ADAPTOR_OUT(from_big,    big_to_native)
+ENDIAN_ADAPTOR_OUT(from_little, little_to_native)
+
+struct storage_tag {};
+struct as_big_t final : storage_tag {};
+struct as_little_t final : storage_tag {};
+struct as_native_t final : storage_tag {};
+
+constexpr static as_big_t big {};
+constexpr static as_little_t little {};
+constexpr static as_native_t native {};
+
+inline auto storage_in(const arithmetic auto& value, as_native_t) {
+	return value;
+}
+
+inline auto storage_in(const arithmetic auto& value, as_little_t) {
+	return native_to_little(value);
+}
+
+inline auto storage_in(const arithmetic auto& value, as_big_t) {
+	return native_to_big(value);
+}
+
+inline void storage_out(arithmetic auto& value, as_native_t) {}
+
+inline void storage_out(arithmetic auto& value, as_little_t) {
+	return little_to_native_inplace(value);
+}
+
+inline void storage_out(arithmetic auto& value, as_big_t) {
+	return big_to_native_inplace(value);
+}
+
+
 } // endian, hexi

include/hexi/pmc/binary_stream_reader.h

@@ -128,6 +128,14 @@ class binary_stream_reader : virtual public stream_base {
 		return data.operator>>(*this);
 	}
 
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	binary_stream_reader& operator>>(endian_func adaptor) {
+		enforce_read_bounds(sizeof(adaptor.value));
+		buffer_.read(&adaptor.value, sizeof(adaptor.value));
+		adaptor.value = adaptor.convert();
+		return *this;
+	}
+
 	template<pod T>
 	requires (!has_shr_override<T, binary_stream_reader>)
 	binary_stream_reader& operator>>(T& data) {
@@ -233,11 +241,11 @@ class binary_stream_reader : virtual public stream_base {
 	 * 
 	 * @param The destination for the read value.
 	 */
-	template<endian::conversion conversion>
-	void get(arithmetic auto& dest) {
-		enforce_read_bounds(sizeof(dest));
-		buffer_.read(&dest, sizeof(dest));
-		dest = endian::convert<conversion>(dest);
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	void get(endian_func& adaptor) {
+		enforce_read_bounds(sizeof(adaptor.value));
+		buffer_.read(&adaptor.value, sizeof(adaptor.value));
+		adaptor.value = adaptor.convert();
 	}
 
 	/**

include/hexi/pmc/binary_stream_writer.h

@@ -55,6 +55,14 @@ class binary_stream_writer : virtual public stream_base {
 		return data.operator<<(*this);
 	}
 
+	template<std::derived_from<endian::adaptor_in_tag_t> endian_func>
+	binary_stream_writer& operator<<(endian_func adaptor) {
+		const auto converted = adaptor.convert();
+		buffer_.write(&converted, sizeof(converted));
+		total_write_ += sizeof(converted);
+		return *this;
+	}
+
 	template<pod T>
 	requires (!has_shl_override<T, binary_stream_writer>)
 	binary_stream_writer& operator<<(const T& data) {
@@ -142,9 +150,9 @@ class binary_stream_writer : virtual public stream_base {
 	 * 
 	 * @param data The element to be written to the stream.
 	 */
-	template<endian::conversion conversion>
-	void put(const arithmetic auto& data) {
-		const auto swapped = endian::convert<conversion>(data);
+	template<std::derived_from<endian::adaptor_out_tag_t> endian_func>
+	void put(const endian_func& adaptor) {
+		const auto swapped = adaptor.convert();
 		write(&swapped, sizeof(swapped));
 	}