ProcessPartial mode does not function as expected

[spectrumjade]

I'm currently rolling out ModSecurity, and I've found that ProcessPartial mode does not function as one would expect.

As I understand it, the expected behavior is that ModSec will buffer as much of the request body as it is configured to (based on SecRequestBodyLimit) and use that as a basis for rule processing while allowing the request to pass through (in accordance with the rules).

What actually seems to happen is that the entire request body will be buffered, regardless of the limits set (prior to my earlier patch, #597, it would reject the request based on the Content-Length header in phase 1). This is not ideal for me, as I do not want data to be buffered to disk at all, and buffering entire requests (which may be as large as 5 GB) in memory has a number of disadvantages.

I had attempted to make some small changes to the input filter, but ultimately these ended poorly (requests being truncated under certain circumstances, not unlike #406). I've had to employ a really unfortunate workaround so that ModSec is usable (basically just disabling buffering for larger requests).

Does anyone else see this behavior? Perhaps I'm missing something?

[btlogy]

I was playing around thos SecRequestBody parameters and some possible config parsing issues (see #728) and I do see the same behavior as you describe.
We have one web service uploading big file to a file server and we have to disable the SecRequestBodyAccess for this location in order to not blowup the apache temp folder.
I was indeed expecting the SecRequestBodyLimit to avoid unlimited size POST to be stored on disk. This looks like a DoS to us as we can not provide as much temp space than permanent one (which is managed by user quotas for instance).
Another thought is the number of parallel request might be problematic as well. Let say we got 1 Gb dedicated tmp disk for Apache (export TMPDIR=/var/tmp/apache). As I see from our tests, if SecRequestBodyLimit is set to 536870912 (512 MB), mod_security and mod_proxy will dechunk and cache the file using twice its size on disk (+ some overhead ?) and will fill up the tmp disk with only one request.
Therefore, I wish we could define a sort of pool size (and maybe a timeout or threshold sending a 420 or 507) to be in control of the total size of those RequestBody on disk. Those two features (stop buffering and pool size) might prevent easy DoS on the tmp folder.

[zimmerle]

The fix is now available on our mainline. The pull request #709 was just merged.