HTTPRequest operator

HTTPRequest operator proposal

Goal

A new operator that supports all HTTP 1.1 request methods (GET, POST, PUT, DELETE, etc.) and is designed to work well with HTTP(S) REST services including Bluemix services.

Issues

• Interacting with Bluemix services is not always supported by existing operators and/or functions in com.ibm.streamsx.inet, some examples:
  • Updating secure gateway destinations requires PUTand a non-standard authorization header - https://www.ng.bluemix.net/docs/api/content/api/securegateway/swagger/index.html
  • Similar request for Twitter: https://github.com/IBMStreams/streamsx.inet/issues/210
• Some comments on the existing operators: https://github.com/IBMStreams/streamsx.inet/issues/105
• Current operators do not (all?) support certification based authentication
• Not well integrated with JSON: https://github.com/IBMStreams/streamsx.inet/issues/187

All existing issues should be looked at when designing this operator.

Use cases

Commons use cases for an single invocation of the operator.

1. Interaction with a single REST service, e.g. Bluemix secure gateway or Twitter. Authentication & connection configuration is common to all requests, multiple methods may be executed against the same REST service (same or different URLs, e.g. different resources in the same service).
2. Execution of requests against internet services (single or multiple URLs) that do not require authentication (public facing data), most likely GETs.

Functionality concepts

• Operator makes an HTTP(S) Request for each tuple arriving on its input port.
• HTTP response becomes an output tuple on an optional output port or an optional error port(?).
• Details of the specific request may be fixed or come from the input tuple (e.g. URL may be fixed but the method may be an attribute in the tuple, or both URL and method can come from the attribute)
• Connection configuration (authentication, retries, timeouts etc.) is driven by operator parameters and/or application configurations
• JSON support will be the default and primary focus.
• Multiple common authentication schemes will be supported (e.g. BASIC, certificate, ...).
• Lessons learnt from existing operators will be applied (e.g. extra headers).