Network maintenance caused so-steno and so-elasticsearch to go missing

[steelerguy]

There was network maintenance that brought down the interfaces on a sensor this weekend for a few hours.

I went to look at alerts yesterday and noticed there were none. so-status showed that so-steno was in a bad state. I took the easy way out and decided to reboot the machine since there was no so-steno-start. I figured a reboot would start everything up in the cleanest way. When the machine came back up, many of the containers were in WAIT_START for a long time, including so-steno and so-elasticsearch. They did get to a STARTING state but eventually went to MISSING along with so-zeek.

I have verified that the interfaces that were down for maintenance are back up and I can do a tcpdump on bond0 and the traffic is there. It like a laucher process is trying to make connections to the internet which is not possible because this is an airgap install.

Any ideas what could be going on? This is version 2.3.130 and was working fine up until the mentioned maintenance. The airgap situation makes it a bit hard to post logs and such.

[reyesj2]

You can run this is the steno wrapper

sudo so-pcap-start

As for services not starting back up try running a highstate to rebuild the containers

sudo salt '*' state.highstate

Also helpful if you can grab some logs from: even if you can't export the log just a summary of whats displayed

docker logs <container in question>