Best approach for ingesting static (historical) firewall log files into Security Onion

[Divyanshu0313]

Version

2.4.200

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Eval

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

12

RAM

32

Storage for /

500 GB

Storage for /nsm

500 GB

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I’m currently working on a Security Onion deployment and need guidance on an offline / historical ingestion use case.

My requirement is to ingest static firewall log files (not live syslog) into Security Onion and have them:

• Properly parsed
• Visible in dashboards
• Reflected correctly in timelines and alerts (where applicable for static data)

Context:

• The firewall logs already exist as files (for example, FortiGate-style traffic logs).
• This is not real-time monitoring; the goal is forensic analysis and SOC-style investigation on historical logs inside Security Onion.

Security Onion is already installed and running successfully. However, I’m unclear on:

• The recommended ingestion method for static firewall logs.
• Where exactly these log files should be placed on the Security Onion system.
• How to ensure logs are parsed correctly (custom pipelines, modules, ingest configs, etc.).
• Common pitfalls to avoid (timestamp handling, duplication, index mapping, ECS compatibility, alert limitations).

If anyone has handled this type of static / historical log ingestion before, I’d really appreciate:

• Your recommended approach
• High-level steps or sample configs
• Things to avoid

Thanks in advance.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[reyesj2]

Are the logs in syslog format? Have you had those firewall logs ingested into Security Onion before (real time logs)? This helps know they are in a format already understood by Security Onion.

Right now, syslogs are setup to be received over 514 tcp/udp and that comes from these integrations

You might have success with adding a new custom 'filestream' integration to your policy and send the logs to the same ingest pipeline that the 'syslog over tcp' integration is using.

[Divyanshu0313]

Yes, the logs are in standard syslog format from a FortiGate firewall.

However, in this case, the logs are historical/static files, not live syslog streams. My concern with using the filestream integration and sending them through the existing syslog ingest pipeline is related to timestamp handling.

Since these logs contain older timestamps, ingesting them as-is may result in:

• Alert storms (e.g., brute-force rules triggering due to burst ingestion)
• Timeline distortion in dashboards
• Misleading alert context, because events are indexed in bulk but reflect older activity

Given this, I’m trying to determine whether it would be possible (and supported) to ingest these static logs using a custom Logstash configuration instead — for example:

• A dedicated file input
• Controlled parsing
• Optional timestamp normalization or controlled reindexing

Is customizing the Logstash pipeline for this type of historical ingestion supported or recommended in Security Onion 2.4? If so, could you provide guidance on the appropriate approach?

My goal is to safely ingest historical firewall logs for forensic analysis.

Thanks again for your help.