alist使用nginx反代后公网上传大文件导致本地磁盘爆满

[gyfy98]

环境说明

我在电视盒子（arm64的ubuntu，内存1+8）上配置了alist，usb插了一块移动硬盘，挂载到了alist（配置如下）。

接着利用nginx反代到 内网ip:80/alist 上，接着路由器将80端口转到8000（自家的公网ipv4，80无法访问），并在路由器上配置了ddns方便记录动态ip。

问题出现步骤

1. 用 内网ip/alist/hdd/dav wevdav协议上传文件（windows系统，软件raildrive，下同），上传前利用MobaXterm观察本地磁盘占用43%，上传期间占用率不会提高。跑满了机顶盒的百兆带宽。
2. 在用ddns域名:8000/alist/hdd/dav上传文件时，上传速率减半（设备仍处在同一局域网下）。磁盘占用会随着文件的大小上升而上升，上传完毕或失败后会回落到43%（<2g文件上传时，MobaXterm观察发现/dev/root占用会逐渐提高5%左右。当上传8G时，观察到本地磁盘的/dev/root逐渐飙升到100%随后上传失败。）该种方法上传文件偶尔发现alist会异常崩溃退出（小文件且磁盘没满）。
3. 在上传8g文件、磁盘块占满时用指令du -lh --max-depth=1 和不上传文件时的对比，没发现明显的占用磁盘的文件夹。

相关配置如下：

nginx：

# ~/nginx/nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	client_max_body_size 500m;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log on;
	error_log /etc/nginx/logs/log.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}

# ~/nginx/locations.d/alist
location ^~/alist {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Range $http_range;
	  proxy_set_header If-Range $http_if_range;
    proxy_redirect off;
    proxy_pass http://127.0.0.1:5244/alist;
    
    # the max size of file to upload
    client_max_body_size 20000m;
}

alist配置如下：

{
  "force": false,
  "site_url": "/alist",
  "cdn": "",
  "jwt_secret": "ZIA2Bm3uFmgiRngh",
  "token_expires_in": 48,
  "database": {
    "type": "sqlite3",
    "host": "",
    "port": 0,
    "user": "",
    "password": "",
    "name": "",
    "db_file": "data/data.db",
    "table_prefix": "x_",
    "ssl_mode": "",
    "dsn": ""
  },
  "meilisearch": {
    "host": "http://localhost:7700",
    "api_key": "",
    "index_prefix": ""
  },
  "scheme": {
    "address": "0.0.0.0",
    "http_port": 5244,
    "https_port": -1,
    "force_https": false,
    "cert_file": "",
    "key_file": "",
    "unix_file": "",
    "unix_file_perm": ""
  },
  "temp_dir": "data/temp",
  "bleve_dir": "data/bleve",
  "dist_dir": "",
  "log": {
    "enable": true,
    "name": "data/log/log.log",
    "max_size": 10,
    "max_backups": 30,
    "max_age": 28,
    "compress": false
  },
  "delayed_start": 0,
  "max_connections": 10,
  "tls_insecure_skip_verify": true,
  "tasks": {
    "download": {
      "workers": 5,
      "max_retry": 1
    },
    "transfer": {
      "workers": 5,
      "max_retry": 2
    },
    "upload": {
      "workers": 5,
      "max_retry": 0
    },
    "copy": {
      "workers": 5,
      "max_retry": 2
    }
  },
  "cors": {
    "allow_origins": [
      "*"
    ],
    "allow_methods": [
      "*"
    ],
    "allow_headers": [
      "*"
    ]
  },
  "s3": {
    "enable": false,
    "port": 5246,
    "ssl": false
  }
}

[gyfy98]

补充下：上传失败的时候客户端报错401，我在alist的日志里截取了有401部分的内容：

	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_arm64.s:1222 
[GIN] 2024/08/25 - 11:35:32 | 404 |    2.872328ms |     192.168.0.1 | PROPFIND  "/alist/dav/hdd/n12t/archives/packages/com.baidu.homework/1724556605133/apk.tar.zst"
[GIN] 2024/08/25 - 11:35:32 | 405 |     560.083µs |     192.168.0.1 | MKCOL    "/alist/dav/hdd/n12t/archives/packages/com.baidu.homework/1724556605133/"
[GIN] 2024/08/25 - 11:36:22 | 201 |  4.276445987s |     192.168.0.1 | PUT      "/alist/dav/hdd/n12t/archives/packages/com.baidu.homework/1724556605133/apk.tar.zst"
[GIN] 2024/08/25 - 11:36:22 | 207 |     775.623µs |     192.168.0.1 | PROPFIND  "/alist/dav/hdd/n12t/archives/packages/com.baidu.homework/1724556605133/apk.tar.zst"
[GIN] 2024/08/25 - 11:36:47 | 401 |     147.208µs |     192.168.0.1 | PROPFIND  "/alist/dav/"
[GIN] 2024/08/25 - 11:36:47 | 207 |    1.135248ms |     192.168.0.1 | PROPFIND  "/alist/dav/"
�[33mWARN�[0m[2024-08-25 11:37:17] failed get /hdd/n12t/DataBackupCloudTmpTest: object not found 
�[31mERRO�[0m[2024-08-25 11:37:17] PROPFIND /alist/dav/hdd/n12t/DataBackupCloudTmpTest object not found

[univerioiln233]

你这个用法有意思啊。
我也苦恼于用webdav磁盘爆满。你可以试试在usb盘的空间上运行alist。