crowdsec unifi collection breaks with newest release 10.1.78

[blassley]

UniFi Collection: Incompatibility with UDM-SE/UDM-Pro (OpenSSH + CEF Format)

Description

The crowdsecurity/unifi collection parsers fail to parse logs from UniFi Dream Machine SE (UDM-SE) and likely UDM-Pro devices due to differences in SSH daemon and CEF field ordering compared to older UniFi hardware.

Environment

• UniFi Device: Dream Machine SE (UDM-SE)
• Firmware Version: 10.1.78 (UniFi Network)
• CrowdSec Version: Latest (tested January 2025)
• Collection Version: crowdsecurity/unifi v0.2

Issues Identified

1. SSH Logs Not Parsed (OpenSSH vs Dropbear)

Problem: UDM-SE uses OpenSSH (sshd), but the collection only includes parsers for Dropbear logs.

Example Log Line:

Jan 25 01:35:01 UDM-SE UDM-SE[-]:  sshd[2559218]: Invalid user fakeuser from 192.168.1.50 port 56926

Current Behavior:

• crowdsecurity/unifi-logs parses the raw syslog format ✅
• crowdsecurity/dropbear-logs fails (device uses sshd, not dropbear) ❌
• crowdsecurity/sshd-logs fails (doesn't match UniFi syslog format) ❌

Result: SSH brute force attempts are not detected.

2. CEF Format Field Order Mismatch

Problem: The crowdsecurity/unifi-cef parser expects CEF fields in a specific order with UNIFIsubCategory present, but UDM-SE sends fields in a different order and omits UNIFIsubCategory.

Example CEF Log:

0|Ubiquiti|UniFi Network|10.1.78|544|Admin Accessed UniFi Network|1|src=192.168.1.229 UNIFIcategory=Audit UNIFIhost=UDM-SE UNIFIaccessMethod=web UNIFIadmin=Branden UNIFIutcTime=2026-01-25T00:48:36.337Z msg=Branden accessed UniFi Network using the web. Source IP: 192.168.1.229

Expected by Parser (UNIFI_ADMIN_PATTERN):

UNIFIcategory=X UNIFIsubCategory=Y UNIFIhost=Z UNIFIaccessMethod=W UNIFIadmin=V src=IP ...

Actual from UDM-SE:

src=IP UNIFIcategory=X UNIFIhost=Z UNIFIaccessMethod=W UNIFIadmin=V UNIFIutcTime=... (no UNIFIsubCategory)

Current Behavior:

• crowdsecurity/cef-logs parses CEF structure ✅
• crowdsecurity/unifi-cef fails to extract fields ❌

Result: Admin access events and security alerts are not parsed for scenarios.

Working Custom Parsers

I've created custom parsers that work with UDM-SE. These could be integrated into the collection:

Custom UniFi SSH Parser

File: parsers/s01-parse/unifi-sshd.yaml

onsuccess: next_stage
filter: "evt.Parsed.message startsWith 'sshd'"
name: crowdsecurity/unifi-sshd
description: "Parse UniFi OpenSSH (sshd) logs for UDM devices"
grok:
  pattern: 'sshd\[%{INT:pid}\]: Invalid user %{USER:sshd_invalid_user} from %{IPORHOST:source_ip}( port %{INT:source_port})?'
  apply_on: message
statics:
  - meta: service
    value: ssh
  - meta: source_ip
    expression: evt.Parsed.source_ip
  - meta: log_type
    value: ssh_failed-auth
  - target: evt.StrTime
    expression: evt.Parsed.timestamp

Custom UniFi CEF Parser

File: parsers/s01-parse/unifi-cef-udm.yaml

onsuccess: next_stage
filter: "evt.Parsed.cef_device_vendor == 'Ubiquiti' && evt.Parsed.cef_device_product == 'UniFi Network'"
name: crowdsecurity/unifi-cef-udm
description: "Parse UniFi CEF logs for UDM devices (alternate field order)"
pattern_syntax:
  UNIFI_ADMIN_PATTERN: 'src=(%{IP:src_ip}) UNIFIcategory=(%{DATA:unifi_category}) UNIFIhost=(%{DATA:unifi_host}) UNIFIaccessMethod=(%{DATA:unifi_access_method}) UNIFIadmin=(%{DATA:unifi_admin}) UNIFIutcTime=(%{DATA:unifi_utc_time}) msg=(%{GREEDYDATA:msg})'
nodes:
  - grok:
      pattern: '%{UNIFI_ADMIN_PATTERN}'
      apply_on: message
    statics:
      - meta: service
        value: unifi
      - meta: source_ip
        expression: evt.Parsed.src_ip
      - meta: admin_user
        expression: evt.Parsed.unifi_admin
      - meta: category
        expression: evt.Parsed.unifi_category
      - meta: access_method
        expression: evt.Parsed.unifi_access_method
      - meta: host
        expression: evt.Parsed.unifi_host
      - meta: message
        expression: evt.Parsed.msg
      - target: evt.StrTime
        expression: evt.Parsed.unifi_utc_time

Proposed Solutions

Option 1: Update Existing Parsers

• Make unifi-cef parser more flexible with multiple pattern variations
• Add OpenSSH support to the UniFi collection

Option 2: Add Device-Specific Parsers

• Keep existing parsers for legacy devices (USG, CloudKey with Dropbear)
• Add new parsers specifically for UDM/UDM-Pro devices (OpenSSH)

Option 3: Auto-Detection

• Create a generic parser that auto-detects CEF field order
• Add logic to detect SSH daemon type (dropbear vs sshd)

Testing

I've verified these custom parsers work correctly with:

• SSH brute force detection (crowdsecurity/ssh-bf scenario)
• CEF admin access logging
• Real-time log parsing from rsyslog

Additional Information

• UDM-SE logs are received via rsyslog on UDP port 4242
• Logs are split into CEF and syslog files based on the CEF program name
• The crowdsecurity/unifi-logs s00-raw parser works correctly; only s01-parse stage parsers need updates

Impact

This affects anyone using:

• UniFi Dream Machine (UDM)
• UniFi Dream Machine Pro (UDM-Pro)
• UniFi Dream Machine SE (UDM-SE)
• Possibly newer UniFi Express devices

[LaurenceJJones]

Hey! already had #1663 in works cause unifi is rapidily changing the CEF format and we cant keep up with every changing update and having to handle multiple versions, so we added a helper which can handle this more gracefully.

With the sshd log lines, are you using the templates we provided in the unifi collection EG:

module(load="imudp")

# Only allow your senders (legacy-style; applies to all UDP inputs)
$AllowedSender UDP, 192.168.1.0/24, 192.168.11.1/32

# Templates
template(name="CEF"    type="string" string="%msg%\n")
template(name="Syslog" type="string" string="%timegenerated% %hostname% %programname%[%procid%]: %msg%\n")

# Bind the UDP/4242 input to a ruleset so only those messages hit the UniFi actions
input(
  type="imudp"
  name="unifi_in"
  port="4242"
  ruleset="unifi"
)

ruleset(name="unifi") {
    if $rawmsg startswith "CEF:" then {
        action(type="omfile" file="/var/log/unifi-cef.log" template="CEF")
    } else {
        action(type="omfile" file="/var/log/unifi-syslog.log" template="Syslog")
    }
    stop
}

as im surprised that it logging the sshd within the message and not as the program name property which syslog knows to parse from the incoming rsyslog.