-
-
Notifications
You must be signed in to change notification settings - Fork 6.2k
fix(graceful): prevent WaitGroup panic on Ctrl+C shutdown #35578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…repeated shutdowns Fixes go-gitea#35559 Signed-off-by: ita004 <[email protected]>
|
I don't think I did twice Ctrl+C in #35559 |
yes, I also reproduced the panic with a single Ctrl+C.. the shutdown flow triggers multiple paths that wait on the same WaitGroup, which causes the reuse panic.... this fix ensures wg.Wait() is only called once, so shutdown completes cleanly without the panic. |
|
This PR's change is just one more patch to the existing messy code (from Restore Graceful Restarting & Socket Activation (#7274)). The root problem is that the "WaitGroup" is abused. Although @zeripath ever complained that I rewrote his code, but for this case, I think we still need to rewrite the code with a correct design to fix the root problem, just like other complete fixes: logger (#24726), queue (#24505), diff (#19958), etc. It shouldn't add more hacky and fragile patches like #11219. |
thanks for the detailed feedback, that makes sense... |
|
FYI: there are some related fixes like "Refactor graceful manager, fix misused WaitGroup #29738". |
… reject new work during shutdown
| // Shutdown the waitgroup to prevent new connections | ||
| // and wait for existing connections to finish | ||
| srv.wg.Shutdown() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the current design would work or it can be improved.
doHammer means that it will stop the server immediately without waiting for any running tasks (for example: existing user request connection).
The graceful package's design overall looks problematic and it seems unfixable to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for clarifying, i understand what you mean now... that makes sense: doHammer should remain immediate and not wait for any running connections, just forcefully stop the server....
i'll keep this note in mind, and if there’s a future plan to refactor or redesign the graceful manager to address the deeper issues, i'd be happy to participate or help with that whenever it’s appropriate...
|
The rewritten: |
This PR fixes a panic issue in the WaitGroup that occurs when Gitea is shut down using Ctrl+C. It ensures that all active connection pointers in the server are properly tracked and forcibly closed when the hammer shutdown is invoked. The process remains graceful — the normal shutdown sequence runs before the hammer is triggered, and existing connections are given a timeout period to complete gracefully. This PR also fixes `no logger writer` problem. Now the log close will only be invoked when the command exit. - Fixes #35468 - Fixes #35551 - Fixes #35559 - Replace #35578 --------- Co-authored-by: wxiaoguang <[email protected]>
What
Guard the
wg.Wait()call usingsync.Onceto avoid"sync: WaitGroup is reused before previous Wait has returned"panic during shutdown.Why
Multiple shutdown paths could cause
Wait()to be invoked more than once,which led to a negative WaitGroup counter and panic when pressing Ctrl+C.
How tested
make backend)./gitea weband triggered Ctrl+C — server shuts down cleanly (no panic)Fixes #35559