|
| 1 | +--- |
| 2 | +title: Bye bye JSON! Welcome Protocol Buffers! |
| 3 | +slug: bye-bye-json-welcome-protocol-buffers |
| 4 | +tags: [tech] |
| 5 | +--- |
| 6 | + |
| 7 | +Developers are familiar working with REST services and its implementation in your client |
| 8 | +application. REST services are most simple method of data exchange between the client |
| 9 | +and server. Well, it’s real simple text based communication technique, and easier to learn |
| 10 | +and debug, and many tools like Postman, Insomnia also exists to play with REST endpoints. |
| 11 | +I also did all those funnier things in my early days of career. |
| 12 | + |
| 13 | +## Story of JSON |
| 14 | + |
| 15 | +```json |
| 16 | +{"status":"OK","message":"Hello JSON!"} |
| 17 | +``` |
| 18 | + |
| 19 | +In the JSON object the characters like `{ } [ ] , : "` doesn’t possess any kind of data. |
| 20 | +Instead it helps the serializer to format the data, so that it can be decoded and structured |
| 21 | +at the end point. The keywords helps to format data to a much more meaningful data for a |
| 22 | +newbie who is reading the JSON document for the first time. For example, `OK` `Hello JSON!` |
| 23 | +are the exact data wrapped in the above JSON object, but it looks like a sentence, and doesn’t |
| 24 | +convey the exact meaning of the JSON object. |
| 25 | + |
| 26 | +For getting things more clear, let’s do an exercise. Count the number of characters in the |
| 27 | +above JSON object. It contains a total of **39** characters including spaces. We have 1 left |
| 28 | +curly bracket, 8 quotation marks, 2 colons, 1 comma and 1 right curly bracket, so **13** |
| 29 | +characters in total. The keywords occupies a total of 6 + 7 = **13** characters. The data holds |
| 30 | +2 + 11 = **13** characters. Hence the data of two strings are wrapped up in a JSON object of |
| 31 | +**26** characters. |
| 32 | + |
| 33 | +Let’s summarize: |
| 34 | + |
| 35 | +``` |
| 36 | +JSON object length: 39 bytes |
| 37 | +Information length: 13 bytes |
| 38 | +Non-information length: 26 bytes /* Wastage */ |
| 39 | +``` |
| 40 | + |
| 41 | +## Protocol Buffers |
| 42 | + |
| 43 | +In my earlier times of career, REST and SOAP are the most popular data exchange mechanisms existed, |
| 44 | +and most devs and companies encourages to learn and use them. And the time flies, the game has moved |
| 45 | +to the next level of faster and efficient binary data serialization techniques, and I met Protocol |
| 46 | +Buffers. |
| 47 | + |
| 48 | +Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing |
| 49 | +structured data. For Protocol buffers, everything is a message. Message is equivalent to **class** |
| 50 | +or **structure** in programming languages. We declare message types with unique names and gives a list |
| 51 | +of fields like this. Let’s define a message in _greeting.proto_ |
| 52 | + |
| 53 | +```proto3 |
| 54 | +syntax = "proto3"; |
| 55 | +
|
| 56 | +message Greeting { |
| 57 | + string status = 1; |
| 58 | + string message = 2; |
| 59 | +} |
| 60 | +``` |
| 61 | + |
| 62 | +The `syntax = "proto3";` sentence tells the compiler that you’re using the version 3 of protocol |
| 63 | +buffers. In the message body, you can define the fields associated with the message. It supports |
| 64 | +unsigned and signed integers, floats, doubles, byte-arrays, strings, booleans, enums and user defined |
| 65 | +messages. In the above example, _don’t confuse it like assigning a number to a string_. It’s just |
| 66 | +field numbers, that represent the order of those fields in serialized data. |
| 67 | + |
| 68 | +What’s next? Get protocol buffer compiler `protoc` from [google/protobuf](https://github.com/google/protobuf) |
| 69 | +and use it to generate some protobuf classes and methods equivalent to the message. You can generate |
| 70 | +code for **any programming language you want.** All you have to do is simply use that classes for data |
| 71 | +serialization and deserialization. For demonstration, I will use Javascript here. |
| 72 | + |
| 73 | +This is how to generate protobuf classes: |
| 74 | + |
| 75 | +```sh |
| 76 | +protoc --js_out=import_style=commonjs,binary:. greeting.proto |
| 77 | +``` |
| 78 | + |
| 79 | +That’s it! `protoc` has generated `greeting_pb.js` from `greeting.proto` for you. Now you can use them |
| 80 | +anywhere you want, like this: |
| 81 | + |
| 82 | +```js |
| 83 | +var pb = require('./greeting_pb') |
| 84 | + |
| 85 | +// Serialization |
| 86 | +var data = { status: 'OK', message: 'Hello JSON!' } |
| 87 | +var msg = new pb.Greeting(); |
| 88 | +msg.setStatus(data.status) |
| 89 | +msg.setMessage(data.message) |
| 90 | +var bytes = msg.serializeBinary(); |
| 91 | + |
| 92 | +// Deserialization |
| 93 | +var msg2 = pb.Greeting.deserializeBinary(bytes) |
| 94 | +console.log(msg2.getStatus(), msg2.getMessage()) |
| 95 | +``` |
| 96 | + |
| 97 | +The serialized data you got is `UInt8Array` . Let’s see how it looks like: |
| 98 | + |
| 99 | +``` |
| 100 | +10 2 79 75 18 11 72 101 108 108 111 32 74 83 79 78 33 |
| 101 | + O K H e l l o J S O N ! |
| 102 | +``` |
| 103 | + |
| 104 | +The serialized data is only just 17 bytes long! Also we can see that there is no more `"status"` |
| 105 | +and `"message"` literals. They are nicely obfuscated by protobuf object, and only they can understand |
| 106 | +the serialized message. |
| 107 | + |
| 108 | +In summary, |
| 109 | + |
| 110 | +``` |
| 111 | +Serialized data length: 17 bytes |
| 112 | +Information length: 13 bytes |
| 113 | +Non-information length: 4 bytes /* In JSON it is 26 */ |
| 114 | +``` |
| 115 | + |
| 116 | +Also, for deserialization, the Protocol Buffers feels so easier, because it only has to remove those |
| 117 | +unnecessary bytes from the serialized data. Hence, the data exchange will become much faster and cost |
| 118 | +effective. If you’re planning to build an IoT based project or mobile apps, Protocol Buffers is just |
| 119 | +for you. |
0 commit comments