Skip to content
StephanDollberg edited this page Sep 16, 2014 · 20 revisions

Creating and using protocols

To create a protocol you have inherit from another protocol via the [Curiously recurring template pattern] (https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern). This basically means to inherit from class that has your class as a template argument. Then you have to implement certain callback methods that will be called once data arrives on your socket.

If we have a look at the following example everything should become clear:

#include <twisted/reactor.hpp>
#include <twisted/basic_protocol.hpp>
#include <twisted/default_factory.hpp>

// [1] CRTP happening here
struct echo_protocol : twisted::basic_protocol<echo_protocol> {
    // [2] this message will be called once a certain amount of data received
    void on_message(const_buffer_iterator begin, const_buffer_iterator end) { 
        // [3] use the send_message function to reply to your peer
        send_message(begin, end);
    }
};

int main() {
    // [4] create a reactor/event loop
    twisted::reactor reac;
    // [5] listen on tcp port 50000 with your echo protocol
    reac.listen_tcp(50000, twisted::default_factory<echo_protocol>());
    // we could add more protocols here
    // [6] run all your protocols
    reac.run();
}

We see a lot of stuff going on here. [1] shows the basic struct/class definition on how to defined a protocol. We inherit from the most basic protocol - the basic_protocol. The basic_protocol will call the on_message(const_buffer_iterator, const_buffer_iterator) function whenever data is available on your connection. The the two iterators build the range that contains the data that was received [2]. In [3] we see how to reply data - by using the send_message(iterator, iterator) function. The two iterators here build the range that will be sent. In this example we simply send the data we received back to the sender - a simple echo protocol.

To get our event loop running we need a reactor [4]. In [5] we tell our reactor on which ports we want to listen and which protocol should be used for each port. We also see that we pass a factory. Factories will be used to create instances of your protocol. In the next chapter factories will be explained more in depth. Finally we tell our event loop to run and accept connections [6].

Factories

If we take look at the declaration of the reactor::listen_tcp function:

template <typename ProtocolFactory>
void listen_tcp(int port, ProtocolFactory factory);

We see that it takes a ProtocolFactory as shown in the first example. A ProtocolFactory is a simple function object that once called creates an instance of the protocol you want to use for the given port.

The twisted::default_factory<ProtocolType>() convenience function will create a factory that default constructs your protocol. So basically we could rewrite the first example to:

reac.listen_tcp(50000, [] () { return echo_protocol(); });

Custom factories, or a simple lambda, can be useful if you want to pass arguments to your protocol.

Transport Types

Currently there are two kinds of transport types - TCP and SSL. The respective functions are:

template <typename ProtocolFactory>
void listen_tcp(int port, ProtocolFactory factory);

template <typename ProtocolFactory>
void listen_ssl(int port, ProtocolFactory factory, ssl_options&& ssl_opts);

You already know the first one. The ssl version takes an extra argument that contains the ssl_options. ssl_options is a typedef for the boost::asio::ssl_context. So you can configure everything that is supported there. Note that we explicitly take a rvalue-reference here as we have to take ownership of the context.

The header <twisted/ssl_options.hpp> provides a convenience function default_ssl_options:

ssl_options default_ssl_options(std::string certificate_filename,
                                       std::string key_filename,
                                       std::string password);

This function will create an options object with SSL3 enabled, the given certificate-chain file and the password for a PEM key file.

Using predefined protocols

There are already some predefined protocols that you can use to implement your application logic.

Basic Protocol

This protocol provides on_message callback which is called whenever data is received.

Line Receiver

The twisted::line_receiver implements string based delimiter parsing. The callback line_received will be called whenever a stream of data is terminted with a certain delimiter.

#include <twisted/byte_receiver.hpp>

struct line_receiver_echo
    : twisted::line_receiver<line_receiver_echo> {

    // delimter is passed as constructor argument to the linereciever
    line_receiver_echo() : line_receiver("\r\n\r\n") {}

    // Note: the range [begin, end) doesn't contain the delimiter
    void line_received(const_buffer_iterator begin, const_buffer_iterator end) {
        // will send the range [begin, end) and append the delimiter
        send_line(begin, end);
    }
}; 

Sending the sequence "AAA\r\n\r\nBBB\r\n\r\nCCC" to the above protocol would call line_received twice and reply with "AAA\r\n\r\n" and "BBB\r\n\r\n".

Note that there is also a default delimiter of "\r\n".

Byte Receiver

The twisted::byte_receiver protocol parses packages in terms of N bytes. You can dynamically update the package size with set_package_size. Whenever N bytes were received the bytes_received callback is called.

#include <twisted/byte_receiver.hpp>

struct byte_receiver_update_test 
    : twisted::byte_receiver<byte_receiver_update_test> {
    // initial package size of 2
    byte_receiver_update_test() : byte_receiver(2) {}

    // std::distance(begin, end) == 2
    void bytes_received(const_buffer_iterator begin, 
                        const_buffer_iterator end) {
        // there is no special send function for the byte_receiver
        send_message(begin, end);

        // the next package size is 20
        set_package_size(20);
    }
};
Mixed Receiver

The twisted::mixed_receiver protocol is a combination of the line_receiver and the byte_receiver. You can activate the different modes via set_byte_mode and set_line_mode. The mixed_receiver offers the interfaces and callbacks from both the line_receiver and byte_receiver.

#include <twisted/mixed_receiver.hpp>

struct mixed_receiver_test 
    : twisted::mixed_receiver<mixed_receiver_test> {
    // we start in line mode 
    // and set the package size to 5 and the delimiter to "\r\n\r\n"
    mixed_receiver_test() : mixed_receiver("\r\n\r\n") {
        set_package_size(5);
    }

    void line_received(const_buffer_iterator begin, 
                       const_buffer_iterator end) {
        send_line(begin, end);
        // switch to byte mode
        // package size is 5 as set in the constructor
        set_byte_mode();
    }

    void bytes_received(const_buffer_iterator begin, 
                        const_buffer_iterator end) {
        send_message(begin, end);
        // switch to line mode - package size will be kept 
        // so that we can switch back to byte mode 
        // without resetting the package size
        set_line_mode();
    }
};

Using deferreds - aka async callbacks

Deferreds as they are called in twisted are callbacks that you want to have executed after a certain time or simply in a thread in the reactor thread pool.
Currently there are three different types of callbacks which all have some quite important differences. Their signature looks like this:

template <typename Callable, typename... Args>
void call_from_thread(Callable&& callable, Args&&... args);

template <typename Duration, typename Callable>
void call_later(Duration duration, Callable callable);

template <typename Callable>
void call(Callable callable);

call_from_thread allows you to dispatch a callable to a thread in the reactor thread pool. This is useful if you want to execute something that is independent from the protocol instance you dispatched it from. Note that this also means that you may not refer to any instance from the protocol in the callback. The protocol instance might no longer be alive and in addition it would not be thread safe.

void on_message(const_buffer_iterator begin,
                const_buffer_iterator end) {
    // do stuff
    call_from_thread([] () { call_third_party_api(); });
}

call will dispatch the callable to a reactor thread. However, using call you can operate on the protocol instance from which you are dispatching the callable. This also means that you can send date data in the callable. The operations will also be synchronized and the lifetime of the protocol will be extended until the callback is called. Note that however your peer could have disconnected in the meantime. This means that if you are operating on the connection state(e.g.: sending data) you should check whether the connection is alive by using is_connected.

void on_message(const_buffer_iterator begin,
                const_buffer_iterator end) {
    // do stuff
    call([this] () { 
        auto result = call_third_party_api();
        // if peer is still connected forward to peer
        if(is_connected()) {
            send_message(result.begin(), result.end());
        }
    });
}

call_later is basically the same as call just that the callable will be called after the given time has elapsed. The duration can be of either std::chrono or boost::chrono type.

void on_message(const_buffer_iterator begin,
                const_buffer_iterator end) {
    // do stuff
    // wait 10 seconds before calling the api
    call_later(std::chrono::seconds(10), [this] () { 
        // same as in `call`
    });
}

Advanced

Creating non-leaf protocols

So far we have only implemented application layer protocols - leaf protocols - that implement application logic and from which won't be inherited from. For better encapsulation you can also implement non-leaf protocols like line_receiver or the byte_receiver. You only need to keep the CRTP pattern alive and provide your new interface functions. Let's have a look at an imaginary json protocol that sends an receives data as a json encoded string delimited by a linefeed. We do also use a magic json library.

// CRTP: take child protocol type as template argument 
//  and pass it to the parent
template <typename ChildProtocol>
struct json_receiver : public twisted::line_receiver<ChildProtocol> {
    // set our line delimiter
    json_receiver() : line_receiver("\n") {}

    void line_received(const_buffer_iterator begin,
                       const_buffer_iterator end) {
        JsonMapType json_map = json::decode(begin, end);
        // this provides the json_received interface 
        //  function for the child protocols
        // Note that it is important to use `this_protocol()` 
        //  to resolve the CRTP
        this_protocol().json_received(json_map);
    }

    // provide the send interface
    void send_json(const JsonMapType& json_map) {
        auto serialized_buffer = json::encode(json_map);
        send_line(serialized_buffer.begin(), serialized_buffer.end())
    }
};

The user could then use it like this:

struct application_logic : json_receiver<application_logic> {
    void json_received(JsonMapType map) {
        auto reply = application_logic(map);
        send_json(reply);
    }
};

This provides a clear separation of application and protocol logic.

The buffer interface - aka using the protocol_core

Clone this wiki locally