2.3.100 -- information no longer processing?

[RagnarSTS]

I also have a completely empty kibana and such after the upgrade completed with no data points after the install completed.
No errors noted on the upgrade.
so-status shows everything started; though seems slower than usual to start things up.
Did the tweak mentioned in another thread despite not having capital letters - my test install is named "seco2"
#7073
After doing the update above the highstate completed without issue. Rather than restarting all individual docker containers just did a reboot...still nothing.
tcpdump definitely shows data still hitting the interface. Stenographer seems to still be making pcap files normally.
soup.log

Attaching soup.log as instructed.

[m0duspwnens]

Looking at the log, the hostname of the manager does appear to be SECO2, so the changes from the previous discussion would have been needed. Following the change to ssl/init.sls, you will need to run salt \* state.apply ssl -l info on the manager to apply the ssl state to all the nodes. Following this, it would be ok to reboot to force the docker containers to reload.

Once the manager has come back online, and running so-status on it shows everything as OK then run the following on the manager: salt-call state.apply kibana.so_savedobjects_defaults -l info.

[RagnarSTS]

Did everything twice just to make sure; no errors noted but still zero information processing through the stack.

[RagnarSTS]

Still nothing processing.
Kind of fun to watch the certificates re-generate.
Nothing in elastic; kibana pops up with a million "no_shard_available_action_exception"

[m0duspwnens]

@RagnarSTS If you haven't rebuilt, the latest hotfix should resolve your issue. If you are still receiving the Kibana error regarding shards, you can run salt-call state.apply kibana.so_savedobjects_defaults -l info on your manager node once the hot fix has been applied and all other nodes have had a chance to run highstate.

[RagnarSTS]

Hotfix applied; state.apply completed, highstate completed, reboot completed ... still zero ingestion after about 30 minutes of letting it try to catch up.
I think my biggest issue with the new docker world is the lack of simple logs in /var/log. Life used to be simple.

[m0duspwnens]

Do you have just a standalone node? The logs are in /opt/so/log and it would be helpful to start with Filebeat, then Logstash, then Elasticsearch logs.

[RagnarSTS]

Aye it is a standalone test box that lasted much longer than anticipated. Apologies I've already reformatted the drives and about 40% of the way through a new setup.
One of these days we'll be ready for production. :)