ILM/Cluster settings

[Shak3]

Add options to setup ILM along side curator.
---ILM could be used to setup rollover/closing index at X days and until ILM can delete based on hard drive space keep using curator for that.
---Rollover using ILM will allow the shards to stay at the recommended 45-50G size. Home setup shards get around maybe 20 megs, I have seen production shard size get ~300G

[TOoSmOotH]

Shard settings can be modified in the global.sls so you can keep your shard sizes to what they need to be. Right now in order to clean up disk space properly, we require manual control over how indices are handled. We are looking at ILM but that cleanup process is assuming that the machine is running just ES whereas in a standalone situation you have PCAP also filling up the disk.

An ILM feature I am most interested in is the ability to combine older indices to reduce the memory footprint when transitioning it to a warm index. This would allow longer retention times but will require more gear. So stay tuned on ILM!

[paziman]

Also I'd like to see option to reconfigure ILM to a tiered disks. For example 'hot' in nvme and frozen in classic cheap hard drive. No need to push frozen data into a expensive media. By using frozen media in cheap lvm storage one can expand the log retention time.

[Shak3]

Yeh I am currently setting the shards after 3 or so days to get the average size of each index. Could just only setup rollover in a distributed setup. so you don't have to worry about the PCAP/ES issue on the stand alone. Adding shards vs just rolling over at a designated size has the same issue if you don't add Search nodes and that is the amount of shards per node/JVM.