Send a binary data example from server to client

[NuclearRazor]

Hi everyone!

I use server example from the master branch of the mongoose library (examples/websocket-server/main.c).

All works fine, but I want to know, how to send integers as binary data to client for example? With WEBSOCKET_OP_BINARY flag.
Or it will always be a "string" (I mean, I should always to use mg_str call).

My dummy example:

// Copyright (c) 2020 Cesanta Software Limited
// All rights reserved
//
// Example Websocket server. See https://mongoose.ws/tutorials/websocket-server/


#include "mongoose.h"

static const char *s_listen_on = "ws://localhost:8000";
static const char *s_web_root = ".";

//static struct mg_rpc *s_rpc_head = NULL;

// This RESTful server implements the following endpoints:
//   /websocket - upgrade to Websocket, and implement websocket echo server
//   /api/rest - respond with JSON string {"result": 123}
//   any other URI serves static files from s_web_root
static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_OPEN) {
    // c->is_hexdumping = 1;
  } else if (ev == MG_EV_HTTP_MSG) {
    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    if (mg_http_match_uri(hm, "/websocket")) {
      // Upgrade to websocket. From now on, a connection is a full-duplex
      // Websocket connection, which will receive MG_EV_WS_MSG events.
      mg_ws_upgrade(c, hm, NULL);
    } else if (mg_http_match_uri(hm, "/rest")) {
      // Serve REST response
      mg_http_reply(c, 200, "", "{\"result\": %d}\n", 123);
    } else {
      // Serve static files
      struct mg_http_serve_opts opts = {.root_dir = s_web_root};
      mg_http_serve_dir(c, ev_data, &opts);
    }
  } else if (ev == MG_EV_WS_MSG) {
    // Got websocket frame. Received data is wm->data. Echo it back!

    int payload = 123;
    int payload_size = 4; //len is in bytes, right?

    mg_ws_send(c, payload, payload_size, WEBSOCKET_OP_BINARY);

    //mg_ws_send(c, wm->data.ptr, wm->data.len, WEBSOCKET_OP_TEXT);
  }
  (void) fn_data;
}


int main(void) {
  struct mg_mgr mgr;  // Event manager
  mg_mgr_init(&mgr);  // Initialise event manager
  printf("Starting WS listener on %s/websocket\n", s_listen_on);
  mg_http_listen(&mgr, s_listen_on, fn, NULL);  // Create HTTP listener
  for (;;) mg_mgr_poll(&mgr, 1000);             // Infinite event loop
  mg_mgr_free(&mgr);
  return 0;
}

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
GNU Make 4.2.1
Built for x86_64-redhat-linux-gnu

[scaprile]

Check the WebSocket server example and tutorial.
Check the MQTT over WebSocket client example and tutorial; when sending binary protocols, you need to "wrap" your message (add "message boundaries") so the other end can identify it as a whole message

[scaprile]

You won't send an integer, you will send bytes
Binary data (bytes) is sent using mg_ws_send()
As you can see in the docs, you specify a pointer and a length, you are not passing a pointer; your compiler should probably have issued a warning

    int payload = 123;
    mg_ws_send(c, &payload, sizeof(payload), WEBSOCKET_OP_BINARY);

Anyway, I'd use JSON and send as text

[NuclearRazor]

Hm, thanks, but can't read incoming data. Maybe I need to "decode" in some way not using just a mg_ws_message type?

Part of server code (other parts are the same as on my intial comment):

int payload = 123;
mg_ws_send(c, &payload, sizeof(payload), WEBSOCKET_OP_BINARY);

Part of client code (all parts are the same as websocket-client example, except this):

 else if (ev == MG_EV_WS_MSG) {
    // When we get echo response, print it
    struct mg_ws_message *wm = (struct mg_ws_message *) ev_data;
    printf("GOT ECHO REPLY: [%d] [%d]\n", (int) wm->data.len, (int)wm->data.ptr);
  }

Got only:
GOT ECHO REPLY: [4] data: [31725874]

[NuclearRazor]

Maybe I need to "unpack" mg_ws_message?

[NuclearRazor]

Sorry, will try with memcpy, printf will not work for binary data.

[NuclearRazor]

void *value = malloc(wm->data.len);
memcpy(value, wm->data.ptr, wm->data.len);
printf("value = %d\n",* (int*)value);

It works great on the client side, thank you very much!

[cpq]

It is a bad idea to send native data types over the network.
Your client can be 32-bit, and a server can be 64-bit. Or vice versa. Or they may have a different endianness. Don't do that.
A significantly more robust way is to encode the data. The simplest encoding is JSON, and Mongoose has native support for it.
So on the sending side, do:

mg_ws_printf(c, "{%Q: %d}", "value", 123);   // Sends a {"value": 123}  websocket message

On the receiving side,

struct mg_ws_message *wm = ev_data;
long value = mg_json_get_long(wm->data, "$.value", -1);  // value is 123

You can do a less verbose enconding - just send a number in a string form: mg_ws_printf(c, "%d", value), and decode on the receiving side as mg_json_get_long(wm->data, "$", -1)

[NuclearRazor]

Thank you very much for more example. Yeah, know about endianness, but this code would be a part of "local services" and I just wanted to know how to work with binary data. Was not so obvious for me.

[cpq]

void *value = malloc(wm->data.len);
memcpy(value, wm->data.ptr, wm->data.len);
printf("value = %d\n",* (int*)value);

That is an overkill. There is no need to copy the received data in order to parse it:

int value = * (int *) wm->data.ptr;  // Assumes that wm->data.len >= sizeof(int)

[TomzBench]

Our embedded device is sending cbor data which is way smaller than JSON. HTTP supports binary just fine. cbor handles machine endianess, and content-length in the header means i dont need to wrap anything. The suggested answer is not correct and OP has legitimate question. "Just use json" is not an answer

[TomzBench]

I used mg_send and it works. but its a little wonky to require the end user to write the http protocol specific stuff in there like HTTP/1.0 200 .... There should be a public http api for sending binary data.

[scaprile]

@TomzBench This is necroposting. Mongoose has experienced about 16 release upgrades from that date. The response given is more than correct.
If you have a problem yourself, I suggest you open a question, I appreciate you trying not to add more noise by searching and reusing a question, though.
Sending binary data is fine with Mongoose, for HTTP: you can check our file transfer tutorials: https://mongoose.ws/documentation/tutorials/http/file-uploads/ ; for WebSocket, my answer above is valid.
I'm closing this, if you still have issues, please tell us your case and we'll find the best solution for you. Thank you.

[TomzBench]

I looked at the current API and thought this post was still relevant, and I also decided to not add new noise as you noticed.

I see in other examples that mongoose is ok with users needing to know the nitty gritty details of http. (For example the http client get request has users writing http protocol stuff).

I don't mind the low level nature of mongoose, it's probably part of what makes it popular and versatile.

My suggestion is that a light weight wrapper sending binary data similar to mg_http_reply would just be a nice touch here.

There are other use cases besides json, especially in embedded.

[scaprile]

Yeah, been doing embedded for a living since before the world would end due to the Y2K bugs all over. As Mongoose has one release every quarter (that would make 12 releases, not 16 as I said), it is usually better to not reuse old threads. The other reason is that the OP was wanting to send binary variables using WebSocket.

The fact is that you are the one who knows what you send.
If you can serve a file, then do it: craft a file in memory and use mg_http_serve_file()
Otherwise, you have to respond, so you need to provide a response code; and you are the one who knows what is what you're sending, so you need to provide the MIME type.
mg_http_reply can handle binary, just use "%.*s", len, data
If you need finer control, use mg_send()
If you don't know how much data you'll send beforehand, use chunks
https://mongoose.ws/documentation/#best-practices

All of that is covered through our tutorials, https://mongoose.ws/documentation/#http , and if something is not, we'll add it.
In our experience, most people (including myself, that used to save every byte by writing Z80 assembly code), prefer JSON. It is easier to handle at the frontend, and the overhead is usually minimal in comparison, not to mention that the more expensive component in the chain is programmer time, usually frontend programmer time. And you save binary parsing issues.

Off-topic, but not that off, around 7 years ago I was designing an IoT architecture, before Amazon and Google provided their own specifics, and I went with a nice compact format that was just like JSON but packed in binary, and it had support for all platforms. When discussing this with the cloud guys (we'd be using Amazon), it turned out that JSON was cheaper. We would use more bandwidth, but we saved breaking the pipe to add the decompressor, which was a more expensive operation.
With the amount of overhead in HTTP over TLS over TCP over IP, usually JSON vs anything else is negligible, and you gain portability, ease of debugging and maintenance through readability, and save programmer time. Just my 2 cents, "never follow ... and advice".

[TomzBench]

Yes - i am in a similar boat. We are currently using CBOR. And now to support integrations easier, we are considering this cbor/json proxy service that would sit in the middle between clients and present a json interface. The proxy would use code generation to translate the data between formats, but still a pain in the ass, and probably would just convert the firmware to use JSON instead.

But we're considering a proxy service anyway because we can provide nicer api's that the firmware can't do, beyond just data formats. So if we have the proxy anyway, whats the big deal?

Which is better? who knows in the long run, probably sticking with JSON. But when your down a whole of design decisions sometimes you are just where you are and cant change stuff on a dime.

Mongoose shouldn't care about where people are per say, and the best practices are a nice recommendation - and with out worrying to much about other peoples problems, I still think a nice wrapper around bin data is nice touch for a cleaner api experience. for the similar reason why mg_http_reply is. (in the case where you do know the content length ahead of time). So i use mg_send.

mg_send is just as fine. So - consider this some feedback you may freely ignore! and not meant to come of as a harsh critque as it may have originally sounded.