native Bruteforce prevention in V3

[OmarMcAdam]

with V2 we could implement fail2ban via the log files to mitigate bruteforce attacks on SMTP

the logfiles were removed with logging refactor as per: #2864

we would like to see a native way for Postal to handle bruteforce attacks, both on the SMTP server and potentially also the web server

[pjv]

We also use fail2ban on host-based log files with Postal V2. Migrating to V3 currently means running with no protection from bad actors which target SMTP servers mercilessly.

[OmarMcAdam]

yup, its a bit of a shame v3 breaks this

[adamcooke]

The method that was used in v2 was not a good practice for containerised workloads. Those log files could end up fairly large without any option to easily rotate them.

However, there are ways to have your Docker logs piped to your local syslog which you can then monitor with fail2ban.

I haven't tried it but perhaps this article might help?

[pjv]

@adamcooke You’re completely right it’s a bad idea to log into a container. Redirecting STDIO to syslog on the host via docker’s config is probably the best stopgap in the interim but having brute-force protection in the SMTP container would be brilliant long-term.

[adamcooke]

I've added some docs about redirecting this log output to syslog so it can be used with fail2ban. https://docs.postalserver.io/features/logging#redirecting-logs-to-the-host-syslog

[rakurtz]

if anyone like me stumbles upon this thread and wants to implement fail2ban for smtp. this is a working implementation:

let docker write to syslog

add this file: /opt/postal/install/docker-compose.override.yml with the following content

version: "3.9"
services:
  smtp:
    logging:
      driver: syslog
      options:
        tag: postal-smtp

install fail2ban

apt update && apt install fail2ban

configure fail2ban

add this file: /etc/fail2ban/filter.d/postal-auth.conf
with this content:

[Definition]
failregex = ^.* postal postal-smtp\[\d+\]: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} \+0000 WARN\s+Authentication failure for <HOST> trace_id=\S+ component=smtp-server$
ignoreregex =

and this file /etc/fail2ban/jail.d/postal-auth.conf

[postal-auth]
enabled = true
filter = postal-auth
logpath = /var/log/syslog
maxretry = 3
bantime = 3600
findtime = 600
backend = auto
action = iptables-allports[name=postal]

check and restart

check the regex: fail2ban-regex /var/log/syslog /etc/fail2ban/filter.d/postal-auth.conf

start fail2ban:

systemctl restart fail2ban

check the status:

fail2ban-client status postal-auth

[bitbay]

In my case on Debian 12 syslog is deprecated in favor of journald.
fail2ban 1.0.2 from debian repository.
(version 1.1.0.1 is latest at the moment but has to be installed via dpkg, configuring that version is somewhat different)
This is how i am trying to make got it working:

docker-compose

  smtp:
    logging:
      driver: journald
      options:
        tag: postal-smtp

journalctl for checking logs

# cli retrieving logs
$ journalctl CONTAINER_NAME=postal-smtp-1

Configure fail2ban for postal

/etc/fail2ban/filter.d/postal-auth.conf

[INCLUDES]
# Read common prefixes.
before = common.conf

[Definition]
# TODO: should define a preregex (common for all lines) to make failregex clearer!
failregex = ^.*postal-smtp\[\d+\]: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} [\+\-]\d+\s+WARN\s+Authentication failure for <HOST> trace_id=\S+ component=smtp-server$
ignoreregex =

Adding systemd as backend in /etc/fail2ban/jail.d/postal-auth.conf

[postal-auth]
enabled = true
filter = postal-auth
maxretry = 3
bantime = 3600
findtime = 600
backend = systemd
action = iptables-allports[name=postal]
# CONTAINER_TAG is set in docker-compose under logging.options.tag
journalmatch = _SYSTEMD_UNIT=docker.service  CONTAINER_TAG="postal-smtp-1"

Adding systemd as backend in /etc/fail2ban/jail.d/defaults-debian.conf

[sshd]
enabled = true
backend = systemd
# only had to use this with version 1.0.2
sshd_backend = systemd

Check regex against journald entries

$ fail2ban-regex --journalmatch='CONTAINER_TAG="postal-smtp"' systemd-journal[journalflags=1] postal-auth

---

With this i have the same rule (hopefully) configured, but after looking at the previous command's output i see what i think is a brute force attack, but fail2ban-regex gives 0 matches.

The log from journalctl

Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  Client identified as XX.XXX.XXX.XX trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  <= EHLO User trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  => 250-My capabilities are trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  => 250 AUTH CRAM-MD5 PLAIN LOGIN trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  <= AUTH LOGIN trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  => 334 VXNlcm5hbWU6 trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  <= QUIT trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  => 334 UGFzc3dvcmQ6 trace_id=ESMDIEF7 component=smtp-server
Sep 03 11:33:41 localhost postal-smtp[575]: 2024-09-03 11:33:41 +0000 DEBUG  Connection closed trace_id=ESMDIEF7 component=smtp-server

The same IP with different intervals tries the same actions:

Client identified as XX.XXX.XXX.XX 
<= EHLO User 
=> 250-My capabilities are 
=> 250 AUTH CRAM-MD5 PLAIN LOGIN 
<= AUTH LOGIN 
=> 334 VXNlcm5hbWU6 
<= QUIT 
=> 334 UGFzc3dvcmQ6 
Connection closed 

As far i understand the user after requesting a LOGIN when confronted with a username (VXNlcm5hbWU6) challenge QUITs immediately - not sure about why the server asks for password (UGFzc3dvcmQ6) if the client left without providing a username 🤷

[willpower232]

Good to know there is another option, I've usually just apt install rsyslog 😅

Postals SMTP client is quite basic so I imagine it just sees the word QUIT as the username, it might be that this is the one place the username does not matter.

[FlorinBuffet]

@bitbay Do you have an idea why the first step, logging to journald, does not work?

[bitbay]

@bitbay Do you have an idea why the first step, logging to journald, does not work?

Would need more context to try to give You an answer, i posted all the needed details in my previous response.

[nneedd]

Used the above steps, but to use fail2ban with nftables

Configure fail2ban for postal
/etc/fail2ban/filter.d/postal-auth.conf

[INCLUDES]
before = common.conf

[Definition]
failregex = postal-smtp\[\d+\]: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} [\+\-]\d+ WARN\s+Authentication failure for <HOST>( \(.*\))? trace_id=\S+ component=smtp-server
ignoreregex =

And /etc/fail2ban/jail.d/postal-auth.conf

[postal-auth]
enabled = true
filter = postal-auth

action   = nftables[name=postal-auth, port=25, protocol=tcp]

backend = systemd
port = smtp,2525
         
maxretry = 3
bantime = 3600
findtime = 60000

# CONTAINER_TAG is set in docker-compose under logging.options.tag
journalmatch = _SYSTEMD_UNIT=docker.service  CONTAINER_TAG="postal-smtp"

also added a few lines to the nftables settings so that fail2ban can add rules to the firewall operation

table inet filter {
...
    chain f2b-postal {

    }
...

...
    chain input {
        type filter hook input priority 0; policy drop;


        # --- Fail2Ban --- 
        jump f2b-postal
...

log from fail2ban

root@postal:~# fail2ban-client status postal-auth
Status for the jail: postal-auth
|- Filter
|  |- Currently failed: 4
|  |- Total failed:     15
|  `- Journal matches:  _SYSTEMD_UNIT=docker.service CONTAINER_TAG=postal-smtp
`- Actions
   |- Currently banned: 2
   |- Total banned:     7
   `- Banned IP list:   some ip addresses

[rakurtz]

i am also redirecting the logs of postal-web to syslog via this entry in the /opt/postal/install/docker-compose.override.yml:

version: "3.9"
services:
  smtp:
    logging:
      driver: syslog
      options:
        tag: postal-smtp
  web:
    logging:
      driver: syslog
      options:
        tag: postal-web

but the logs unfortunately don't show any failed authentication attempts to the web interface. Any ideas on that anyone?

[willpower232]

I presume that the auth failure will be logged by rails and not by the webserver, if they're even logged which may not be the case.

[rakurtz]

i will hide the web interface behind a firewall anyways so i guess i don't necessarily need to secure this with fail2ban. Thanks for you quick reply!

[olegbliaher]

I don't know if it's still relevant for you but I've created a PR that does just that (except all the fail2ban jazz): #3221

[arthurzenika]

We might be contributing some documentation and configuration to get https://www.crowdsec.net/ to ingest the postal logs in a similar fashion that has been done here with failban. Crowdsec has a similar approach to fail2ban with an added community block list and other nice features. (fail2ban is awesome, if it fits your needs, stick with that solution).