Ticket #117 : Add clustering documentation

Ticket #118 : Add docker compose file to publish EventMesh cluster with two peers

Author: habarthierry-hue

README.md

@@ -10,7 +10,7 @@ For project documentation, please visit [docs](https://simpleidserver.github.io/
 
 | Command                     | Description                             |
 | --------------------------  | --------------------------------------- |
-| psake publishWebsite 		| Publish website                       	|
+| psake publishWebsite 		  | Publish website                       	|
 | psake publishDockerCI       | Build and publish docker images         |
 | psake packTemplate          | Build template package                  |
 

docs/documentation/eventmesh/architecture.md

@@ -1 +1,3 @@
-# Architecture
+# Architecture
+
+**TODO**

docs/documentation/eventmesh/clustering.md

@@ -28,7 +28,7 @@ To ensure fault tolerance, EventMesh replicates queues across peers using the [R
 Data is divided into partitions. Each partition has a number of replicas. Among the replica set, a leader is determined by the raft protocol which takes in requests and performs all of the processing.
 All other peers are passive followers. When the leader becomes unavailable, the followers transparently select a new leader.
 
-Eah peer in the cluster my be both leader and follower at the same time of different partitions.
+Each peer in the cluster may be both leader and follower at the same time of different partitions.
 
 In EventMesh, the partition key corresponds to the `groupId` or the `topic` name
 
@@ -42,11 +42,35 @@ An EventMesh cluster can be formed in a number of ways :
 * Using etcd-based discovery (via a plugin).
 * Using in-memory list on nodes (enabled by default).
 
-In order to have an up and running cluster one plugin must be enabled.
+In order to have an up and running cluster one of the plugin above must be enabled.
 For more information please refer to the `Plugins` chapter.
 
 # Quick start
 
 In this tutorial, we are going to install two EventMesh servers with the plugin `DiscoveryEtcd` enabled via Docker.
 
-TODO
+Download the docker compose file 
+
+```
+https://raw.githubusercontent.com/simpleidserver/FaasNet/master/src/Samples/FaasNet.EventMesh.EtcdCluster/docker-compose.yml
+```
+
+Open a command prompt and execute the following command 
+
+```
+docker-compose up
+```
+
+Two peers will be deployed and are listening the ports `4000` and `4001`.
+
+Always in a command prompt, add a new VPN 
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_vpn --name=vpn --port=4001
+```
+
+Check if the VPN is correctly replicated in the second node by executing the following command 
+
+```
+FaasNet.EventMeshCTL.CLI.exe get_all_vpn --port=4002
+```

docs/documentation/eventmesh/installserverondocker.md

@@ -0,0 +1,14 @@
+# Installing on Docker
+
+This guide describes how EventMesh peer node can be installed on Docker.
+
+## Deploy a windows service
+
+> [!IMPORTANT]
+> Docker must be installed on your machine.
+
+Open a command prompt and execute the following command line. An EventMesh peer node will be deployed and listening the port 4000.
+
+```
+docker run --name eventmesh -p 4000:4000/udp eventmesh
+```

docs/documentation/eventmesh/installserveronwindows.md

@@ -1,6 +1,6 @@
 # Installing on Windows Manually
 
-This guide describes how EventMesh server can be installed and configured manually on Windows.
+This guide describes how EventMesh peer node can be installed and configured manually on Windows.
 
 ## Download zip
 

docs/documentation/eventmesh/pluginamqp.md

@@ -20,13 +20,13 @@ The ZIP file can be downloaded [here]().
 | maxFrameSize      | Largest frame size that the sending peer is able to accept on this connection.              | 1000          |
 | maxChannel        | The channel-max value is the highest channel number that can be used on the connection      | 1000          |
 | sessionLinkCredit | Current maximum number of messages that can be handled at the receiver endpoint of the link | 255           |
-| eventMeshVpn      | EventMesh server VPN                                                                        | default       |
-| eventMeshUrl      | EventMesh server URL                                                                        | localhost     |
-| eventMeshPort     | EventMesh server Port                                                                       | 4000          |
+| eventMeshVpn      | EventMesh peer VPN            	                                                          | default       |
+| eventMeshUrl      | EventMesh peer URL      	                                                                  | localhost     |
+| eventMeshPort     | EventMesh peer Port       	                                                              | 4000          |
 
 ## Quick start
 
-Once you have an up and running EventMesh server with `ProtocolAmqp` plugin enabled, you can start using any client compliant with the [AMQP 1.0 protocol](http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-complete-v1.0-os.pdf).
+Once you have an up and running EventMesh peer with `ProtocolAmqp` plugin installed, you can start using any client compliant with the [AMQP 1.0 protocol](http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-complete-v1.0-os.pdf).
 
 ### Configure client and VPN
 
@@ -59,9 +59,9 @@ If the plugin is not yet configured, it can be enabled like this
 FaasNet.EventMeshCTL.CLI.exe enable_plugin --name=ProtocolAmqp
 ```
 
-Its configuration can be updated either by [using CLI](cli.md) or by updating the configuration file `appsettings.json`.
+Its configuration can be updated either by [using CLI](cli.md) or by updating the configuration file `plugin.json`.
 
-Don't forget that the EventMesh server must be restarted, otherwise the changes are not taken into account.
+Don't forget that the EventMesh peer must be restarted, otherwise the changes are not taken into account.
 
 When the configuration is finished, a client can be created and can start publishing message.
 

docs/documentation/eventmesh/pluginsinkamqp.md

@@ -22,67 +22,63 @@ The ZIP file can be downloaded [here]().
 | amqpTopicName     | Name of the topic exchange                | amq.topic          	|
 | amqpUserName      | AMQP username                             | guest          		|
 | amqpPassword      | AMQP password                             | guest          		|
-| eventMeshUrl      | EventMesh server URL                      | localhost     		|
-| eventMeshPort     | EventMesh server Port                     | 4000         			|
-| eventMeshVpn      | EventMesh server VPN                      | default       		|
+| eventMeshUrl      | EventMesh peer URL                     	| localhost     		|
+| eventMeshPort     | EventMesh peer Port                    	| 4000         			|
+| eventMeshVpn      | EventMesh peer VPN                     	| default       		|
 | clientId      	| Client identifier used to publish message | publishClientId       |
 
 ## Quick start
 
-Once the plugin `SinkAMQP` is configured and enabled, you can deploy an AMQP 1.0 server such as RabbitMQ.
+Once you have an up and running EventMesh peer with `SinkAMQP` plugin installed, you can deploy an AMQP 1.0 server such as RabbitMQ.
 
-In this tutorial we will explain how to deploy a local RabbitMQ server with the plugin `rabbitmq_amqp1_0` enabled via Kubernetes.
+In this tutorial we will explain how to deploy a local RabbitMQ server with the plugin `rabbitmq_amqp1_0` enabled via Docker.
 
-### Deploy RabbitMQ via Kubernetes
+### Configure client and VPN
 
-Open a command prompt and install the RabbitMQ Cluster Kubernetes Operator
+Before going further, a Virtual Private Network (VPN) and one client must be configured.
+Those information will be used to publish message.
 
-```
-kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"
-```
-
-Deploy RabbitMQ
+Open a command prompt and create a topic named `default` :
 
 ```
-kubectl apply -f "https://raw.githubusercontent.com/simpleidserver/FaasNet/master/src/Samples/FaasNet.EventMesh.AmqpSink/rabbitmq.yml"
+FaasNet.EventMeshCTL.CLI.exe add_vpn --name=default
 ```
 
-When the RabbitMQ server is running, fetch the default credentials username/password from the Kubernetes secrets.
-Both values are encoded in base64 and must be decoded.
+Add a client `publishClientId`, as the name suggests, it will be used to publish message.
 
 ```
-kubectl get secret hello-world-default-user -o jsonpath='{.data.user}'
-kubectl get secret hello-world-default-user -o jsonpath='{.data.password}'
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=publishClientId --publish_enabled=true --subscription_enabled=false
 ```
 
-### Update plugin configuration
+### Deploy RabbitMQ via Docker
 
-Always in a command prompt, use the CLI to update the username and password with the values obtained from the previous step.
+Download the Docker file
 
 ```
-FaasNet.EventMeshCTL.CLI.exe update_plugin_configuration --name=SinkAMQP --key=amqpUserName --value=<USERNAME>
-FaasNet.EventMeshCTL.CLI.exe update_plugin_configuration --name=SinkAMQP --key=amqpPassword --value=<PASSWORD>
+https://raw.githubusercontent.com/simpleidserver/FaasNet/master/Samples/FaasNet.EventMesh.AmqpSink/AmqpServerDockerFile
 ```
 
-Use the AMQP port `30007`.
+Open a command prompt and build the docker image
 
 ```
-FaasNet.EventMeshCTL.CLI.exe update_plugin_configuration --name=SinkAMQP --key=amqpPort --value=30007
+docker build -t amqpserver -f AmqpServerDockerFile .
 ```
 
-And use the client identifier `publishClient`.
+Run RabbitMQ peer with the plugin `rabbitmq_amqp1_0` enabled.
 
 ```
-FaasNet.EventMeshCTL.CLI.exe update_plugin_configuration --name=SinkAMQP --key=clientId --value=publishClient
+docker run --name rabbitmq -p 5672:5672 -p 15672:15672 amqpserver
 ```
 
 ### Enable the plugin
 
-Enable the plugin and restart the EventMesh server to take into account your changes.
+Enable the plugin and restart the EventMesh peer to take into account your changes.
+
+By default, the configuration is correct and there is no need to update it.
 
 ```
 FaasNet.EventMeshCTL.CLI.exe enable_plugin --name=SinkAMQP
 ```
 
-Now the EventMesh server is running, it should be able to capture all the events coming from the exchange name `amq.topic`.
-You can publish some messages in RabbitMQ and check if they are captured by the EventMesh server.
+Now the EventMesh peer is running, it should be able to capture all the events coming from the exchange name `amq.topic`.
+You can publish some messages in RabbitMQ and check if they are captured by the EventMesh peer.

docs/documentation/eventmesh/pluginsinkkafka.md

@@ -21,46 +21,55 @@ The ZIP file can be downloaded [here]().
 | kafkaGroupId 			| Client group identifier 					 | EventMeshKafka    |
 | kafkaMetadataTimeout  | Timeout in MS to fetch metadata from Kafka | 2 seconds         |
 | kafkaTopicTimerMS     | Timer in MS used to fetch kafka topics     | 5 seconds         |
-| eventMeshUrl      	| EventMesh server URL                       | localhost     	 |
-| eventMeshPort     	| EventMesh server Port                      | 4000         	 |
-| eventMeshVpn      	| EventMesh server VPN                       | default       	 |
+| eventMeshUrl      	| EventMesh peer URL                       	 | localhost     	 |
+| eventMeshPort     	| EventMesh peer Port                      	 | 4000         	 |
+| eventMeshVpn      	| EventMesh peer VPN                       	 | default       	 |
 | clientId      		| Client identifier used to publish message  | publishClientId   |
 
 ## Quick start
 
-Once the plugin `SinkKafka` is configured and enabled, you can deploy an Apache Kafka server.
+Once you have an up and running EventMesh peer with `SinkKafka` plugin installed, you can deploy an Apache Kafka server.
 
 In this tutorial we will explain how to deploy Apache Kafka by using Docker.
 
-### Deploy Apache Kafka via Docker
+### Configure client and VPN
 
-Download the file
+Before going further, a Virtual Private Network (VPN) and one client must be configured.
+Those information will be used to publish message.
+
+Open a command prompt and create a topic named `default` :
 
 ```
-https://raw.githubusercontent.com/simpleidserver/FaasNet/master/Samples/FaasNet.EventMesh.KafkaSink/docker-compose.yml
+FaasNet.EventMeshCTL.CLI.exe add_vpn --name=default
 ```
 
-Open a command prompt and execute the command below to deploy Apache Kafka
+Add a client `publishClientId`, as the name suggests, it will be used to publish message.
 
 ```
-docker-compose up
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=publishClientId --publish_enabled=true --subscription_enabled=false
 ```
 
-### Update plugin configuration
+### Deploy Apache Kafka via Docker
 
-Use the client identifier `publishClient`.
+Download the file
 
 ```
-FaasNet.EventMeshCTL.CLI.exe update_plugin_configuration --name=SinkKafka --key=clientId --value=publishClient
+https://raw.githubusercontent.com/simpleidserver/FaasNet/master/Samples/FaasNet.EventMesh.KafkaSink/docker-compose.yml
+```
+
+Open a command prompt and execute the command below to deploy Apache Kafka
+
+```
+docker-compose up
 ```
 
 ### Enable the plugin
 
-Enable the plugin and restart the EventMesh server to take into account your changes.
+Enable the plugin and restart the EventMesh peer to take into account your changes.
 
 ```
 FaasNet.EventMeshCTL.CLI.exe enable_plugin --name=SinkKafka
 ```
 
-Now the EventMesh server is running, it should be able to capture all the events coming from Apache Kafka.
-You can publish some messages in Apache Kafka and check if they are captured by the EventMesh server.
+Now the EventMesh peer is running, it should be able to capture all the events coming from Apache Kafka.
+You can publish some messages in Apache Kafka and check if they are captured by the EventMesh peer.

docs/documentation/eventmesh/pluginsinkvpnbridge.md

@@ -19,9 +19,88 @@ The ZIP file can be downloaded [here]().
 | jobId      			| Job identifier              				 	| VpnBridge    	 	|
 | bridgeTimerMS 		| Timer in MS to get bridge servers      		| 5 seconds	     	|
 | bridgeGroupId 		| Group identifier used to subscribe to a topic | VpnBridgeGroupId  |
-| eventMeshUrl      	| EventMesh server URL                       	| localhost     	|
-| eventMeshPort     	| EventMesh server Port                      	| 4000         	    |
-| eventMeshVpn      	| EventMesh server VPN                       	| default       	|
+| eventMeshUrl      	| EventMesh peer URL                       	    | localhost     	|
+| eventMeshPort     	| EventMesh peer Port                      	    | 4000         	    |
+| eventMeshVpn      	| EventMesh peer VPN                       	    | default       	|
 | clientId      		| Client identifier used to publish message  	| publishClientId   |
 
-TODO
+## Quick start
+
+To establish a link between two VPN, we are going to use Docker to deploy two EventMesh cluster.
+
+Download the Docker compose file
+
+```
+https://raw.githubusercontent.com/simpleidserver/FaasNet/master/src/Samples/FaasNet.EventMesh.TwoCluster/docker-compose.yml
+```
+
+Open a command prompt and execute the following command 
+
+```
+docker-compose up
+```
+
+Two EventMesh clusters will be deployed and will listen the ports `4001` and `4002`.
+
+### Configure the client, VPN and the bridge
+
+Add a new VPN `default` in both cluster.
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_vpn --name=default --port=4001
+FaasNet.EventMeshCTL.CLI.exe add_vpn --name=default --port=4002
+```
+
+Add a client `publishClient` in the cluster `4002`. It will be used by the CLI to publish message
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=publishClient --publish_enabled=true --subscription_enabled=false --port=4002
+```
+
+Add a client `subscribeClient` in the cluster `4002`. It will be used by the cluster `4001` to receive message.
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=subscribeClient --publish_enabled=false --subscription_enabled=true --port=4002
+```
+
+Add a client `publishClientId` in the cluster `4001`. It will be used by the SinkVpnBridge to publish message.
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=publishClientId --publish_enabled=true --subscription_enabled=false --port=4001
+```
+
+
+Add a client `subscribeClient` in the cluster `4001`. It will be used by the CLI to receive message.
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_client --vpn=default --identifier=subscribeClient --publish_enabled=false --subscription_enabled=true --port=4001
+```
+
+Add a bridge between both VPN
+
+```
+FaasNet.EventMeshCTL.CLI.exe add_vpn_bridge --vpn=default --tvpn=default --turn=evtmeshnode2 --tport=4000 --tid=subscribeClient --port=4001
+```
+
+### Configure and enable the plugin
+
+Enable the plugin and restart the EventMesh cluster to take into account your changes.
+
+```
+FaasNet.EventMeshCTL.CLI.exe enable_plugin --name=SinkVpnBridge --port=4001
+```
+
+Now both EventMesh cluster are running, the cluster `4001` should be able to capture all the events coming from the cluster `4002`.
+
+Publish a message TODO
+
+```
+FaasNet.EventMeshCTL.CLI.exe publish_message --vpn=default --identifier=publishClient --topic=topic --message=hello --port=4002
+```
+
+
+Read the message TODO
+
+```
+FaasNet.EventMeshCTL.CLI.exe read_message --vpn=default --identifier=subscribeClient --topic=topic --port=4001
+```