Communication through Network via NetGate

Message delivery among different ActorSystems is carried out by NetGate. An ActorSystem can associate with multiple NetGates which communicate with different remote ActorSystems. The basic philosophy of communication via NetGate is that, if actor A wants to send a message to a remote actor B, A first serializes and sends the message to a Sender actor in the NetGate, the Sender forwards the message bytes to the Receiver actor in the NetGate at the side of B, the Receiver forwards the message to B, then B deserializes the messages and processes it.

For the default implementation of NetGate in ZAF, the Sender/Receiver is essentially PUSH/PULL sockets. If a NetGate connects with multiple peer NetGates, it creates multiple pairs of PUSH/PULL sockets, one for each peer, so that the message delivery can be parallelized.

Actor Registration and Actor Lookup

That sending a message to a remote actor is the same as we send a message to a local actor, where what we need is a Actor handle for the remote actor. However, the remote actor is spawned in a remote ActorSystem, how do we get the Actor handle of it? The solution is actor registraion and actor lookup.

A local actor can register itself to a local NetGate, by sending a regisration message with code NetGate::ActorRegistration and the name of the actor. In this way, the local actor is exposed to the outside world. To obtain an actor registered in a remote NetGate, we send a actor lookup message to the local NetGate with code NetGate::ActorLookupReq, the url of the remote NetGate and the name of the actor we want. Once found, the local NetGate will reply a message with code NetGate::ActorLookupRep and the actor. Instead of sending and receiving the messages with specified codes, it's suggested to use NetGateClient to interact with NetGate.

// In ActorSystem A
NetGate gate{actor_system, "x.y.z.w", port};
Actor actor = actor_system.spawn<NewActor>();
gate.register_actor("new_actor", actor); // register the new_actor in NetGate("x.y.z.w:port") with name "new_actor"

// In ActorSystem B
NetGate gate{actor_system, "a.b.c.d", port};
Actor actor = actor_system.spawn(
  [&, client = NetGateClient{gate.actor()}](ActorBehavior& self) {
    // send a NetGate::ActorLookupReq to local NetGate
    client.lookup_actor(self, "x.y.z.w:port", "new_actor");
    // wait for the NetGateActorLookupRep from local NetGate
    self.receive_once({
      // process the reply
      client.on_lookup_actor_reply([&](std::string remote_net_gate_url, std::string actor_name, Actor remote_actor) {
        // `remote_actor` can be used to send messages to the `new_actor` in ActorSystem A.
        ...
      });
    });
  });

You may also check the example here.

ActorInfo

Actor can be safely delivered among actors in the same ActorSystem, no matter whether Actor refers to a local one or a remote once. However, Actor cannot be delivered among different ActorSystems, due to many reasons, e.g., we do not know which NetGate will be used for communication with the delivered Actor and we do not know if the NetGate to be used is connected with the NetGate on the side of the delivered Actor.

Therefore, to deliver an Actor from one ActorSystem to another, we have two steps:

1. When sending an Actor, convert the Actor to ActorInfo and send the ActorInfo. This is done by Actor::to_actor_info, which requires the actor that asks for the ActorInfo, because it makes difference if it's a remote actor or a local actor that asks for a remote actor or a local actor.

2. When receiving an ActorInfo, convert the ActorInfo to Actor before sending messages to the Actor. This is done by sending a mesasge to the NetGate with code NetGate::RetrieveActorReq and the ActorInfo, where we will receive a message with code NetGate::RetrieveActorRep and the Actor.

Instead of sending and receiving the messages with specified codes, it's suggested to use NetGateClient to interact with NetGate. Here is an example that delivers Actor a in ActorSystem A from b in the same ActorSystem to c in ActorSystem B.

// In ActorSystem A
Actor a = actor_system.spawn<ActorA>();
Actor b = actor_system.spawn([&](ActorBehavior& self) {
  self.receive_once({
    AskForActorInfo - [&]() {
      this->reply(ReplyWithActorInfo, a.to_actor_info(this->get_current_sender_actor()));
    }
  });
});
gate.register("b", b);

// In ActorSystem B
NetGate gate{actor_system, "a.b.c.d", port};
Actor c = actor_system.spawn(
  [&, client = NetGateClient{gate.actor()}](ActorBehavior& self) {
    Actor b = ... ; // supposed we have fetched the remote actor named "b" in ActorSystem A.
    self.request(b, AskForActor).on_reply({
      ReplyWithActorInfo - [&](ActorInfo info) {
        // Send a request `NetGate::RetrieveActorReq` to local NetGate
        client.retrive_actor(Requester{self}, info).on_reply({
          // Wait for a response `NetGate::RetrieveActorRep` from local NetGate
          client.on_retrieve_actor_reply([&](ActorInfo, Actor remote_a) {
            // remote_a can be used to send messages to Actor a in ActorSystem A
            ...
          });
        });
      }
    });
  });