chunked docs

Author: mneedham

doc/asciidoc/consumer/index.adoc

@@ -1,3 +1,4 @@
+[[consumer]]
 == Neo4j Streams Consumer
 
 Is the Kafka Sink that ingest the data directly into Neo4j

doc/asciidoc/index.adoc

@@ -5,7 +5,24 @@
 :sectid:
 :sectlinks:
 
-== Motivation
+
+[abstract]
+--
+This is the user guide for Neo4j Streams version {docs-version}, authored by the Neo4j Labs Team.
+--
+
+The guide covers the following areas:
+
+* <<introduction>> -- An introduction to Neo4j Streams
+* <<producer>> -- Sends transaction event handler events to a Kafka topic
+* <<consumer>> -- Ingests events from a Kafka topic into Neo4j
+* <<procedures>> -- Procedures for consuming and producing Kafka events
+* <<docker>> -- Docker Compose files for local testing
+* <<kafka-connect>> -- Kafka Connect Sink plugin 
+
+
+[[introduction]]
+== Introduction
 
 Many user and customers want to integrate Kakfa and other streaming solutions with Neo4j.
 Either to ingest data into the graph from other sources.
@@ -21,7 +38,8 @@ The project is composed of several parts:
 
 In the next releases we will add Kafka-Connect connectors as well.
 
-== Installation
+[[installation]]
+=== Installation
 
 Download the latest release jar from https://github.com/neo4j-contrib/neo4j-streams/releases/latest
 
@@ -37,7 +55,8 @@ kafka.bootstrap.servers=localhost:9092
 For each module there are additional configs that are explained in the individual sections.
 
 
-== Build locally
+[[build-locally]]
+==== Build locally
 
 ----
 mvn clean install
@@ -53,4 +72,7 @@ include::consumer/index.adoc[]
 
 include::procedures/index.adoc[]
 
-include::docker/index.adoc[]
+
+include::docker/index.adoc[]
+
+include::kafka-connect/index.adoc[]

doc/asciidoc/kafka-connect/index.adoc

@@ -0,0 +1,92 @@
+[[kafka-connect]]
+== Kafka Connect
+
+image::neo4j-loves-confluent.png[Neo4j Loves Confluent]
+
+
+Kafka Connect, an open source component of Apache Kafka, is a framework for connecting Kafka with external systems such as databases, key-value stores, search indexes, and file systems.
+
+The Neo4j Streams project provides a Kafka Connect plugin that can be installed into the Confluent Platform enabling:
+
+- Ingest data from Kafka topics directly into Neo4j via templated Cypher queries;
+- Stream Neo4j transaction events (*coming soon*).
+
+=== Plugin installation
+
+You can choose your preferred way in order to install the plugin
+
+==== Download and install the plugin via Confluent Hub client
+
+If you are using the provided compose file you can easily install the plugin by using the Confluent Hub.
+
+Once the compose file is up and running you can install the plugin by executing the following command:
+
+[source,bash]
+----
+<confluent_platform_dir>/bin/confluent-hub install neo4j/kafka-connect-neo4j:1.0.0
+----
+
+When the installation will ask:
+
+[source,bash]
+----
+The component can be installed in any of the following Confluent Platform installations:
+----
+
+Please prefer the solution `(where this tool is installed)` and then go ahead with the default options.
+
+Following an example:
+
+image::confluent-hub-client.jpg[Installation via Confluent Hub Client]
+
+At the end of the process the plugin is automatically installed.
+
+==== Download the zip from the Confluent Hub
+
+Please go to the Confluent Hub page of the plugin:
+
+https://www.confluent.io/connector/kafka-connect-neo4j-sink/
+
+And click to the **Download Connector** button.
+
+Once you downloaded the file please place it into your Kafka Connect `plugins` dir.
+
+==== Build it locally
+
+Download the project from Github:
+
+    git clone https://github.com/neo4j-contrib/neo4j-streams.git
+
+Go into the `neo4j-streams` directory:
+
+    cd neo4j-streams
+
+Build the project by running the following command:
+
+    mvn clean install
+
+Inside the directory `<neo4j-streams>/kafka-connect-neo4j/target/component/packages` you'll find a file named `neo4j-kafka-connect-neo4j-<VERSION>.zip`, please place it into your Kafka Connect `plugins` dir.
+
+=== Docker compose file
+
+Inside the directory `/kafka-connect-neo4j/docker` you'll find a compose file that allows you to start the whole testing environment:
+
+.docker-compose.yml
+[source,yaml]
+----
+include::../../../kafka-connect-neo4j/docker/docker-compose.yml[]
+----
+
+include::../../../kafka-connect-neo4j/docker/readme.adoc[]
+
+=== Monitor via Confluent Platform UI
+
+The Kafka Monitoring UI can be found at http://<localhost>:9021/management/connect
+
+image::confluent-metrics.jpg[Confluent Importing Metrics]
+
+They show up properly in my topic, and then are added to Neo4j via the sink.
+
+Below you see the data that has been ingested into Neo4j. During my testing I got up to more than 2M events.
+
+image::confluent-imported-data.jpg[Confluent Platform Management]

doc/asciidoc/procedures/index.adoc

@@ -1,3 +1,4 @@
+[[procedures]]
 == Procedures
 
 The Streams project comes out with a list of procedures.

doc/asciidoc/producer/index.adoc

@@ -1,3 +1,4 @@
+[[producer]]
 == Neo4j Streams Producer
 
 Is the transaction event handler events that sends data to a Kafka topic