-
Notifications
You must be signed in to change notification settings - Fork 28
HTTPRequestAsync operator
A new operator that supports all HTTP 1.1 request methods (GET, POST, PUT, DELETE, etc.) and is designed to work well with HTTP(S) REST services in asynchronous manner.
- Current operators/native functions support HTTP requests in synchronous manner only.
- Streaming pipelines as well as other streams applications are required to perform HTTP/REST calls to external services (such as alerting or activating external systems) and yet, avoid back pressure.
-
The asynchronous call approach can make HTTP/REST requests possible without blocking the thread. The responses will be handled as callbacks, in asynchronous manner when they arrive.
-
Operator makes an HTTP(S) request for each tuple arriving on its input port.
-
The request is placed on a queue(s) and the calling thread is released
-
Each queue has a working thread that sends out the HTTP request with a callback to handle the response, keeping the input tuple data.
-
When the response arrives, it will call the callback with the response parameters (including the input tuple data)
-
The operator will provide an output section to allow output manipulations
param url: "http://api.example.com/scores/" + team // url can be any expression (team - attribute of an input tuple) -
Connection configuration (authentication, retries, timeouts etc.) is driven by operator parameters and/or application configurations JSON support will be the default and primary focus. Multiple common authentication schemes will be supported (e.g. BASIC, certificate, ...). Lessons learnt from existing operators will be applied (e.g. extra headers, and look at existing issues requesting new functionality).