Unpacking binary data in C++

[jimmy-swa]

This isn't an issue necessarily with Pybind11 but more of a C++ noobie's cry for help. I would like to be able to pass packed binary data from Python into C++ and then copy it into a basic struct. I tried reading through #2517 but just don't have a good enough understanding to get this to work.

Python

import struct
some_values = [2, 2, 2, 2, 1, 1, 8, 8]
struct.pack("B"*8, *some_values)  # b'\x02\x02\x02\x02\x01\x01\x08\x08'

C++

struct my_struct {
  uint8_t v1;
  uint8_t v2;
  uint8_t v3;
  uint8_t v4;
  uint8_t v5;
  uint8_t v6;
  uint8_t v7;
  uint8_t v8;
};

My binding function takes an argument py::arg("my_bytes") with type py::bytes which appear to be passed in correctly:

// the bytes received are: b'\x02\x02\x02\x02\x01\x01\x08\x08' with size 8
std::cout << "the bytes received are: " << my_bytes << " with size " << sizeof(my_bytes) << '\n';

// results in unexpected values in v (which change during each run)
my_struct v;
std::memcpy(&v, &my_bytes, sizeof(my_bytes));

What am I missing?

[jimmy-swa]

Okay, after a lot of head banging I figured this out. Instead of accepting a py::bytes in my binding function, I'm using std::string (ref). Using std::memcpy to assign to my struct works as expected.

[jagerman]

You might prefer to still take a py::bytes argument so that you cannot get accidentally pass a string from the Python side, then in C++ convert the py::bytes to a string or string_view:

std::string content{my_bytes};
// or:
std::string_view content{my_bytes};
// and then:
std::memcpy(&v, content.data(), content.size());

The string_view requires compiling in C++17 mode, but if you are it's preferable as it avoid an extra intermediate copy of the data.

[jimmy-swa]

Great tip, thank you