Allow Kibana to connect to all search nodes directly when using a true ES cluster

[rwaight]

In a true ES cluster, Kibana can be configured to leverage High availability across multiple Elasticsearch nodes:

Kibana can be configured to connect to multiple Elasticsearch nodes in the same cluster. In situations where a node becomes unavailable, Kibana will transparently connect to an available node and continue operating. Requests to available hosts will be routed in a round robin fashion.

In kibana.yml:

elasticsearch.hosts:
  - http://elasticsearch1:9200
  - http://elasticsearch2:9200

It looks like there would need to be an update to this code (in /salt/kibana/etc/kibana.yml) to something similar to the following:

{%- if TRUECLUSTER is sameas true %}
  {%- some syntax to pull an array of ES hosts %}
  elasticsearch.hosts: [ "https://array:9200", "https://of:9200", "https://elasticsearch:9200", "https://nodes:9200", "https://within:9200", "https://securityonion:9200", ]
{%- else %}
  elasticsearch.hosts: [ "https://{{ ES }}:9200" ]
{%- endif %}

If there is a mechanism to grab the array of search nodes, then I'd be happy to put in a pull request.

[TOoSmOotH]

I am talking to @jertel about this. The main issue here is we would need soc to support this as well. There is also a complete overhaul of the elasticsearch state that will allow more flexibility coming in .80 for customizing elaticsearch via pillar. Kibana is on this same list to be completely overhauled. This will remove the need to use local versions of the configs. We will see if Kibana makes .90 but in the meantime I will make this an issue we can track.

[TOoSmOotH]

#5424