Dockerized sftpgo with external auth hook and s3 compatible storage backend

[premax]

Hello there,

I am new to this great piece of software and I'm trying to set up the following configuration:

1. Ideally, all configuration should be done using Docker Compose environment variables or returned by the auth hook JSON
2. I'd like to avoid using admin panel entirely
3. I already have my user account stored in the database backend and my hook is handling authentication.
4. I'd like to store user files in S3-compatible object storage backend (Backblaze) and preferably avoid storing nything locally.

I've been studying the documentation and examples for many hours now but I still can't get the last part (S3 storage) working.

Here is my Docker Compose definition - I assume I'm probably using the wrong variables?

  sftpserver:
    image: drakkan/sftpgo:latest
    volumes:
      - certbot_data:/etc/letsencrypt
      - sftp_config:/var/lib/sftpgo
      - sftp_data:/srv/sftpgo
    ports:
      - "2022:2022"       # SFTP
      - "21:21"           # FTP(S) explicit
      - "990:990"         # FTPS implicit
      - "50000-50010:50000-50010"  # FTP passive ports
    environment:
      # ustawienia backendu B2 (object storage)
      - SFTPGO_OBJECTSTORAGE__PROVIDER=s3
      - SFTPGO_OBJECTSTORAGE__B2__KEYID=${B2_ARCHIVE_APPLICATION_KEY_ID}
      - SFTPGO_OBJECTSTORAGE__B2__APPLICATIONKEY=${B2_ARCHIVE_APPLICATION_KEY}
      - SFTPGO_OBJECTSTORAGE__B2__BUCKET=${B2_ARCHIVE_BUCKET}

      - SFTPGO_OBJECTSTORAGE__PROVIDER=S3   # also tried s3
      - SFTPGO_OBJECTSTORAGE__S3__NAME=s3
      - SFTPGO_OBJECTSTORAGE__S3__ENDPOINT=https://s3.us-west-001.backblazeb2.com
      - SFTPGO_OBJECTSTORAGE__S3__REGION=us-west-001
      - SFTPGO_OBJECTSTORAGE__S3__ACCESSKEYID=${B2_ARCHIVE_APPLICATION_KEY_ID}
      - SFTPGO_OBJECTSTORAGE__S3__SECRETACCESSKEY=${B2_ARCHIVE_APPLICATION_KEY}
      - SFTPGO_OBJECTSTORAGE__S3__BUCKET=*********
      - SFTPGO_OBJECTSTORAGE__S3__FORCEPATHSTYLE=true

#Also tried this:
#      - AWS_REGION=us-west-001
#      - AWS_ENDPOINT=https://s3.us-west-001.backblazeb2.com
#      - AWS_ACCESS_KEY_ID=${B2_ARCHIVE_APPLICATION_KEY_ID}
#      - AWS_SECRET_ACCESS_KEY=${B2_ARCHIVE_APPLICATION_KEY}

      - SFTPGO_DATA_PROVIDER__EXTERNAL_AUTH_HOOK=https://my-https-hook.com/hook.php
      - SFTPGO_DATA_PROVIDER__EXTERNAL_AUTH_SCOPE=1

      - SFTPGO_FTPD__BINDINGS__0__PORT=21
      - SFTPGO_FTPD__BINDINGS__0__PASSIVE_HOST=${SFTP_HOST}
      - SFTPGO_FTPD__BINDINGS__0__TLS_MODE=0 # explicit TLS

      - SFTPGO_FTPD__BINDINGS__1__PORT=990
      - SFTPGO_FTPD__BINDINGS__1__PASSIVE_HOST=${SFTP_HOST}
      - SFTPGO_FTPD__BINDINGS__1__TLS_MODE=2 # implicit TLS

      - SFTPGO_FTPD__CERTIFICATE_FILE=/etc/letsencrypt/live/${APPDOMAIN}/fullchain.pem
      - SFTPGO_FTPD__CERTIFICATE_KEY_FILE=/etc/letsencrypt/live/${APPDOMAIN}/privkey.pem

And here is the JSON that my hook is returning along with 200 code after successfull authentication:

{
  "status": 1,
  "username": "hup",
  "home_dir": "/tmp",
  "uid": 1000,
  "gid": 1000,
  "permissions": {
    "/": [
      "*"
    ]
  },
  "filesystem": {
    "provider": "s3",
    "s3config": {
      "region": "us-west-001",
      "bucket": "*********",
      "key_prefix": "hup/",
      "endpoint": "https://s3.us-west-001.backblazeb2.com",
      "force_path_style":true
    }
  }
}

If you have a similar object storage setup, please share your configuration — it would be very helpful.

[fostermi]

You can only configure s3 backends in users, groups or virtual folders configurations. AFAIK, there is no configuration settings via the config file or ENV vars like you are attempting. sftpgo still requires local storage to create a placeholder directory for each user which by default is /srv/sftpg/data. This is used as temporary storage for file uploads before the file gets moved to s3 so it'll have to be large enough to accommodate potential large files from multiple users at once. You should leave the user's home_dir to the system default and configure the user's storage at the user or group level (or group + virtual folder). This page might help. https://docs.sftpgo.com/2.7/tutorials/postgresql-s3/#add-virtual-users