|
| 1 | += Exposing HTTP processors |
| 2 | +:description: Expose NiFi ListenHTTP processor by creating Service and Ingress objects, allowing external tools to trigger workflows or send data. |
| 3 | + |
| 4 | +You can use the `ListenHTTP` processor to start a HTTP webserver and listen for incoming connections. |
| 5 | + |
| 6 | +In this case `POST` requests are being send to NiFi, which acts as a webhook to receive data here. |
| 7 | +You should also be able to serve `GET` requests by using the `HandleHttpRequest` processor, however this is currently not demonstrated in this guide. |
| 8 | + |
| 9 | +== 1. Create `ListenHTTP` processor |
| 10 | + |
| 11 | +Let's start by creating a `ListenHTTP` processor: |
| 12 | + |
| 13 | +1. Set `Base Path` to an empty string |
| 14 | +2. Set `Listening Port` to `8042`. |
| 15 | +3. Set `Record Reader` to an `JsonTreeReader` and `Record Writer` to an `JsonRecordSetWriter` instance. |
| 16 | + This guide assumes that JSON documents are being posted to the listener. For other document formats, change the record reader and writer accordingly. |
| 17 | + |
| 18 | +You should end up with something similar to |
| 19 | + |
| 20 | +image:listen-http-1.png[A ListenHTTP processor] |
| 21 | + |
| 22 | +== 2. Expose `ListenHTTP` processor |
| 23 | + |
| 24 | +Afterwards you need to expose the processor to the outside world. |
| 25 | + |
| 26 | +For that to work, first create a Service object as follows. |
| 27 | +In this guide, the NifiCluster is called `simple-nifi`. The name of the Nifi cluster must match the value of the `app.kubernetes.io/instance` as shown below. |
| 28 | + |
| 29 | +[source,yaml] |
| 30 | +---- |
| 31 | +apiVersion: v1 |
| 32 | +kind: Service |
| 33 | +metadata: |
| 34 | + name: simple-nifi-listen-http # Update according to NifiCluster name |
| 35 | +spec: |
| 36 | + type: ClusterIP |
| 37 | + selector: |
| 38 | + app.kubernetes.io/component: node |
| 39 | + app.kubernetes.io/instance: simple-nifi # Update according to NifiCluster name |
| 40 | + app.kubernetes.io/name: nifi |
| 41 | + ports: |
| 42 | + - name: http |
| 43 | + port: 8042 |
| 44 | + protocol: TCP |
| 45 | + targetPort: 8042 |
| 46 | +---- |
| 47 | + |
| 48 | +In case you don't have an ingress controller, you can set the Service type to `LoadBalancer` or `NodePort` instead and should be ready to go. |
| 49 | +If you are using an ingress controller, an Ingress could look something like |
| 50 | + |
| 51 | +[source,yaml] |
| 52 | +---- |
| 53 | +apiVersion: networking.k8s.io/v1 |
| 54 | +kind: Ingress |
| 55 | +metadata: |
| 56 | + name: simple-nifi-listen-http # Update according to NifiCluster name |
| 57 | +spec: |
| 58 | + rules: |
| 59 | + - host: simple-nifi-listen-http.my.corp # Update to your host |
| 60 | + http: |
| 61 | + paths: |
| 62 | + - pathType: Prefix |
| 63 | + path: / |
| 64 | + backend: |
| 65 | + service: |
| 66 | + name: simple-nifi-listen-http # Update to your Service name |
| 67 | + port: |
| 68 | + number: 8042 |
| 69 | +---- |
| 70 | + |
| 71 | +=== 3. Route |
| 72 | + |
| 73 | +The next step is to handle different kind of messages coming in, based on the HTTP path. |
| 74 | + |
| 75 | +First, create a `RouteOnAttribute` processor and connect it to the `success` output of the `ListenHTTP` processor as shown below. |
| 76 | +Start the `ListenHTTP` processor. |
| 77 | + |
| 78 | +image:listen-http-2.png[´RouteOnAttribute connected to ListenHTTP processor] |
| 79 | + |
| 80 | +The `ListenHTTP` processor should now generate a FlowFile for every incoming HTTP request. |
| 81 | +You can test this by calling `curl --verbose --data '{"hello":"NiFi"}' https://simple-nifi-listen-http.my.corp`, you should get a `HTTP/2 200` response. |
| 82 | + |
| 83 | +TIP: If you get a `503 Service Temporarily Unavailable`, this probably means your Ingress controller was not able to |
| 84 | +reach your `ListenHTTP` processor. Check that the processor is running and configured correctly. |
| 85 | + |
| 86 | +This should result in one FlowFile being queued, as shown in the picture above and below. |
| 87 | + |
| 88 | +image:listen-http-3.png[Resulting FlowFile] |
| 89 | + |
| 90 | +In the `RouteOnAttribute` processor, add a field called `/webhook/foo` with the value of `${"restlistener.request.uri":equals('/webhook/foo')}`. |
| 91 | +You can replace `/webhook/foo` with whatever URL your HTTP service should be reachable. |
| 92 | +This guide also added `/webhook/bar` and `/webhook/baz` in a similar way. |
| 93 | + |
| 94 | +The `RouteOnAttribute` processor now has one `unmatched` output as well as one for every field you defined. |
| 95 | +You can connect a processor for every HTTP path you routed on. |
| 96 | + |
| 97 | +The result should look something like below and should allow you to consume many different HTTP `POST` requests. |
| 98 | + |
| 99 | +image:listen-http-4.png[Result flow] |
0 commit comments