update readme [ci skip]

Author: HatsuneMiku3939

README.md

@@ -8,7 +8,37 @@
 
 [Fluentd](http://fluentd.org/) input plugin to pull log from rest api.
 
-Many of modern server application offer status reporting API via http (even 'fluentd' too). This plugin will help to gathering status log from these status api.
+Many of modern server application offer status reporting API via http (even 'fluentd' too).
+This plugin will help to gathering status log from these status api.
+
+* [Installation](#installation)
+* [Usage](#usage)
+	* [Basic Usage](#basic-usage)
+	* [Monitor Status Code](#monitoring-http-status-code-only)
+	* [Override User Agent](#override-user-agent)
+	* [HTTP Basic Auth](#http-basic-auth)
+	* [HTTP Proxy](#http-proxy)
+	* [Logging Response Header](#logging-http-response-header)
+	* [Custom Request Header](#custom-request-header)
+	* [HTTPS Support](#https-support)
+* [Configuration](#configuration)
+	* [tag](#tag-string-required)
+	* [url](#url-string-required)
+	* [agent](#agent-string-optional-default-fluent-plugin-http-pull)
+	* [interval](#interval-time-required)
+	* [format](#format-required)
+	* [status_only](#status_only-bool-optional-default-false)
+	* [http_method](#http_method-enum-optional-default-get)
+	* [timeout](#timeout-time-optional-default-10s)
+	* [proxy](#proxy-string-optional-default-nil)
+	* [user](#user-string-optional-default-nil)
+	* [password](#password-string-optional-default-nil)
+	* [response_header](#response_header-section-optional-default-nil)
+	* [request_header](#response_header-section-optional-default-nil)
+	* [verify_ssl](#verify_ssl-bool-optional-default-true)
+	* [ca_path](#ca_path-string-optional-defualt-nil)
+	* [ca_file](#ca_file-string-optional-defualt-nil)
+* [Copyright](#copyright)
 
 ## Installation
 
@@ -32,87 +62,227 @@ And then execute:
 $ bundle
 ```
 
-## Example
+## Usage
 
-You can found more examples in `test/plugin/test_in_http_pull.rb`
+### basic usage
+
+In your Fluentd configuration, use @type http_pull.
+`tag`, `url`, `interval`, `format` is mandatory configuration.
+
+```
+<source>
+	@type http_pull
+
+	tag status
+	url http://your-infrastructure/api/status.json
+	interval 1s
+
+	format json
+</source>
+
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+In this example, a response of your infrastructure parsed as json.
+A result contains `url`, `status`, `message`.	`message` is a response of your infrastructure.
+
+```
+{
+	"url": "http://your-infrastructure/api/status.json",
+	"status": 200,
+	"message": {
+		// response of your infra structure
+	}
+}
+```
+
+You can found more examples in this document.
 
 ### Monitoring http status code only
+
+If you need only http status code, not response body, You can turn off response
+body parser to set `status_only` is true. Remember that `format` is mandatory.
+In this case, you must set `format` is none.
+
+
 ```
 <source>
-  @type http_pull
+	@type http_pull
 
-  tag test
-  url http://www.google.com
-  interval 1s
+	tag fluentd.status
+	url http://your-infrastructure/healthcheck
+	interval 1s
 
-  format none
-  status_only true
+	status_only true
+	format none
 </source>
 
-<match test>
-  @type stdout
-</match>
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/healthcheck","status":200}
+```
+
+### Override User Agent
+
+You can set the User Agent by specifying the `agent` option.
 
-# 2017-05-17 21:40:40.413219000 +0900 test: {"url":"http://www.google.com","status":200}
-# 2017-05-17 21:40:41.298215000 +0900 test: {"url":"http://www.google.com","status":200}
-# 2017-05-17 21:40:42.310993000 +0900 test: {"url":"http://www.google.com","status":200}
-# 2017-05-17 21:40:43.305947000 +0900 test: {"url":"http://www.google.com","status":200}
 ```
+<source>
+	@type http_pull
+
+	tag status
+	url http://your-infrastructure/api/status.json
+	interval 1s
+
+	format json
+	agent infrastructure_monitor
+</source>
+
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+### HTTP basic auth
+
+If your infrastructure is protected by HTTP basic auth,
+You can use `user`, `password` options to provide authentication information.
 
-### Monitoring fluentd itself
 ```
 <source>
-  @type monitor_agent
+	@type http_pull
+
+	tag status
+	url http://your-infrastructure/api/status.json
+	interval 1s
 
-  port 24220
+	format json
+	user foo
+	passwrd bar
 </source>
 
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+### HTTP proxy
+
+You can send your requests via proxy server.
+
+```
 <source>
-  @type http_pull
+	@type http_pull
 
-  tag fluentd.status
-  url http://localhost:24220/api/plugins.json
-  interval 1s
+	tag status
+	url http://your-infrastructure/api/status.json
+	proxy http://your-proxy-server:3128
+	interval 1s
 
-  format json
+	format json
 </source>
 
-<match fluentd.status>
-  @type stdout
-</match>
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+### Logging HTTP response header
+
+If you wish to monitoring not only response body but also response header,
+provide name of header in `response_header` sections.
 
-# 2017-05-17 21:41:47.872951000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
-# 2017-05-17 21:41:48.955316000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
-# 2017-05-17 21:41:50.033628000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
-# 2017-05-17 21:41:51.107372000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
 ```
+<source>
+	@type http_pull
+
+	tag status
+	url http://your-infrastructure/api/status.json
+	interval 1s
+
+	format json
 
+	<response_header>
+		header Content-Type
+	</response_header>
+
+	<response_header>
+		header Content-Length
+	</response_header>
+</source>
+
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+
+### Custom request header
+
+The custom request header also supported.
+This will help you when your infrastructure needs authentication headers or something like that.
 
-### Monitoring elasticsearch cluster health
 ```
 <source>
-  @type http_pull
+	@type http_pull
+
+	tag status
+	url http://your-infrastructure/api/status.json
+	interval 1s
 
-  tag es.cluster.health
-  url http://localhost:9200/_cluster/health
-  interval 1s
+	format json
 
-  format json
+	<request_header>
+		header API_ACCESS_KEY
+		value hatsune
+	</request_header>
+
+	<response_header>
+		header API_ACCESS_KEY_SECRET
+		value miku
+	</response_header>
 </source>
 
-<match es.cluster.health>
-  @type stdout
-</match>
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+
+### HTTPS support
+
+If your infrastructure has https endpoints, just use https url for monitoring them.
+
+#### self-signed SSL
+
+
+If your infrastructure has https endpoints secured by self signed certification,
+you can provide custom certification file via `ca_path`, `ca_file` option.
 
-# 2017-05-17 12:49:09.886298008 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
-# 2017-05-17 12:49:10.669431296 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
-# 2017-05-17 12:49:11.668789668 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
-# 2017-05-17 12:49:12.668789849 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
+```
+<source>
+	@type http_pull
+
+	tag status
+	url https://your-infrastructure/api/status.json
+	interval 1s
+	ca_path /somewhere/ca/stored
+	ca_file /somewhere/ca/stored/server.crt
+
+	format json
+</source>
+
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
+```
+
+And, disable SSL verification also supported. (not recommended)
+
+```
+<source>
+	@type http_pull
+
+	tag status
+	url https://your-infrastructure/api/status.json
+	interval 1s
+	verify_ssl false
+
+	format json
+</source>
+
+# 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
 ```
 
 ## Configuration
 
-### Basic options
+### Basic Configuration
 
 #### tag (string) (required)
 
@@ -143,7 +313,7 @@ for more detail.
 
 If `status_only` is true, body is not parsed.
 
-###E http_method (enum) (optional, default: :get)
+#### http_method (enum) (optional, default: :get)
 
 The http request method for each requests. Avaliable options are listed below.
 
@@ -157,13 +327,13 @@ If `status_only` is true, `http_method` was override to `head`
 
 The timeout of each request.
 
-### Proxy options`
+### Proxy Configuration
 
 #### proxy (string) (optional, default: nil)
 
 The HTTP proxy URL to use for each requests
 
-### Basic auth options
+### Basic Auth Configuration
 
 #### user (string) (optional, default: nil)
 
@@ -173,7 +343,7 @@ The user for basic auth
 
 The password for basic auth
 
-### Req/Resp header options
+### Req/Resp Header Configuration
 
 #### response_header (section) (optional, default: nil)
 
@@ -183,7 +353,7 @@ The name of response header for capture.
 
 The name, value pair of custom reuqest header.
 
-### SSL options
+### SSL Configuration
 
 #### verify_ssl (bool) (optional, default: true)
 
@@ -195,33 +365,10 @@ The absolute path of directory where ca_file stored. Should be used with `ca_fil
 
 #### ca_file (string) (optional, defualt: nil)
 
-The Absolute path of ca_file. Should be used with `ca_path`.
-
-
-## In case of remote error
-
-### Can receive response from remote
-
-```
-{
-  "url": url of remote
-  "status": status code of response
-  "error": "RestClient::NotFound: 404 Not Found" or something similar
-}
-```
-
-### All the other case
-
-```
-{
-  "url": url of remote
-  "status": 0
-  "error": "Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 12345" or something similar
-}
-```
+The absolute path of ca_file. Should be used with `ca_path`.
 
 ## Copyright
 
 * Copyright(c) 2017- filepang
 * License
-  * Apache License, Version 2.0
+	* Apache License, Version 2.0