Pcaps and Kibana Logs

[noob2020-alt]

Hello, I have a few questions?

security onion version - 2.3.110
Deployment - standalone
Use purpose - collect logs and pcaps for analysis

1. How can we transfer the sec onion /nsm/pcaps files into actual caps via the cli and NOT the gui of the SOC?
   Any attempts at using so-export-pcap leads to an error due to tcpdump not being able to recognize the file-type in /nsm/pcaps/

so-steno does work, but stenographer is not installed
Using the docker command yields the same results
' sudo docker exec -it so-steno stenoread "YourStenoQueryHere" -w /tmp/new.pcap '
' sudo so-pcap-export "YourStenoQueryHere" output '
both commands do not work.

2. Where can I find the logs that are in Kibana on the system
   As a backup plan, where would I be able to pull the logs for Kibana if I wanted to transfer the logs from one machine to another?
   Ex: I have machine 1 that collected the data;
   I want to copy data from machine 1 to machine 2 (which has a new SOC) and see the logs in Kibana.
   Same with the pcaps:

Thank you

[dougburks]

security onion version - 2.3.110

Security Onion 2.3.110 is over a year old and is therefore outdated. Please consider upgrading to the latest version.

1. How can we transfer the sec onion /nsm/pcaps files into actual caps via the cli and NOT the gui of the SOC?
   Any attempts at using so-export-pcap leads to an error due to tcpdump not being able to recognize the file-type in /nsm/pcaps/

First, please note that it's not so-export-pcap but instead so-pcap-export.

From https://docs.securityonion.net/en/2.3/stenographer.html#command-line:
You can also access packet capture from the command line of the box where the pcap is stored using a steno query as defined at https://github.com/google/stenographer#querying. In the following examples, replace “YourStenoQueryHere” with your actual steno query.

So take a look at https://github.com/google/stenographer#querying for valid query syntax.

2. Where can I find the logs that are in Kibana on the system

From https://docs.securityonion.net/en/2.3/kibana.html:
Kibana is a free and open user interface that lets you visualize your Elasticsearch data

You can read more about Elasticsearch at:
https://docs.securityonion.net/en/2.3/elasticsearch.html

[noob2020-alt]

@dougburks Thanks a bunch. I will try that, and greatly appreciate your response.

[noob2020-alt]

@dougburks Thank you again for the tips. We got it working. I do have a couple of questions regarding this discussion before closing out.
I want to move the pcaps from one sensor to another.

Would taking the files from /nsm/pcap and /nsm/pcapindex be ok to do, or would I need to transfer more than just the two folders?

Ex:

sensor 1 is needed for new setup (standalone)
sensor 2 will get an scp of old data on sensor1 (standalone)

We scp from sensor1:/nsm/pcap and /nsm/pcapindex folders to sensor2

Would sensor2 (standalone) be able to read the pcaps?

if not, would you mind telling how to do it correct, if it can be done at all?

Thank you

V/R

Jay

[dougburks]

I haven't tried that. You might want to set up a couple of test machines and try it out.

[noob2020-alt]

@dougburks Tried and worked

Copying the /nsm/pcap and /nsm/pcapindex folders to a new build works.
pcaps were able to be added to a new pcap directory and pull it from there.

There was an issue where copying external pcaps into the new original pcap folder resulted in the copied files being immediately removed. The conclusion was a security or permissions issue. No further analysis was done there.

The work around was to import the entire directory "/nsm/pcap" onto the new build AFTER changing the name of the new builds' /nsm/pcap folder to pcap.old. Same for the pcapindex directory.

Ex: scp -r root@standalone:/nsm/pcap .

So a lengthy scp -r of the directories allowed the both pcap directories to come in, as is.
The pcap directory took the longest of the two due to the pcapindex being smaller files.

We were able to query the pcaps using the gui, verified their original date, downloaded a pcap and viewed all data in Wireshark successfully.

In conclusion, it can be done with just the two directories listed above, at a bare minimum. Best for space and time.
A suggestion would be to

1. ensure the new system's IP matches the old IP
2. Ensure the file permissions match the original ones upon adding to the new build (i.e. steno...:steno...)

The device "manager" IP can be changed to pull data (ssh, rsync, etc.) from any other device while retaining the installed gui IP configuration.
The IP should be changed using the "nmtui" command.
It's best to reboot the system as opposed to using the systemctl or service commands. Some issues were found when not rebooting.

An assumption can be made that this would work on standalone and distributed environments, however, only the standalone method has been tested.

On the distributed environments, the pcaps are on the forward node so the process may differ a little, or not.

One final suggestion would be to just copy the entire /nsm directory over if the device transfer isn't too taxing and there's enough storage space.

Thanks for your helps with this. I greatly appreciate it.

v/r
Jay