Skip to content

Use io.Reader to avoid file size limits#95

Open
mporsch wants to merge 10 commits intoflexera-public:masterfrom
mporsch:io-reader
Open

Use io.Reader to avoid file size limits#95
mporsch wants to merge 10 commits intoflexera-public:masterfrom
mporsch:io-reader

Conversation

@mporsch
Copy link
Contributor

@mporsch mporsch commented Aug 10, 2023

Currently the (download) file size is limited at 100MB.
If only this constant is increased, the server has to buffer all content in memory before passing it on.

In this PR we use an io.Reader to process everything as a stream.
Also we remove the file size limits and expose some timeouts to the command line timeout values of server and client.

Marco Porsch and others added 7 commits August 16, 2023 12:51
this reverts "use command-line-exposed timeout for client write deadline" and "use command-line-exposed timeout for tunnel to allow writing large requests"
This fixes a race condition caused by the invalid use of sync.WaitGroup which
caused "panic: sync: WaitGroup is reused before previous Wait has returned".
encountered when the chan in the server HTTP payloadHandler sends the signal
before the tunnel reader started waiting

fixed by obtaining the lock before sending the signal which is only available
if the tunnel reader is in wait or has closed already
@mporsch
Copy link
Contributor Author

mporsch commented Aug 22, 2024

Adding some more commits that fix an issue intruduced by the bad use of sync.WaitGroup earlier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant