-
Notifications
You must be signed in to change notification settings - Fork 32
🐛⚗️ Remove cancel_on_disconnect decorator from certain api-server endpoints
#7986
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛⚗️ Remove cancel_on_disconnect decorator from certain api-server endpoints
#7986
Conversation
cancle_on_disconnect decorator from certain api-server endpointscancle_on_disconnect decorator from certain api-server endpoints
cancle_on_disconnect decorator from certain api-server endpointscancel_on_disconnect decorator from certain api-server endpoints
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #7986 +/- ##
===========================================
- Coverage 87.88% 69.69% -18.20%
===========================================
Files 1841 796 -1045
Lines 71081 36610 -34471
Branches 1231 176 -1055
===========================================
- Hits 62473 25516 -36957
- Misses 8252 11036 +2784
+ Partials 356 58 -298
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
|
pcrespov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the result of your tests is that we should now add those, please make sure that it is commented in the endpoint (or even better add a failing test) so we do not "undo" it



What do these changes do?
disable_on_disconnectdecorator from 2 endpoints in the api-server: The one for getting upload links for a file multipart upload and the one for completing the multipart uploadMotivation for this PR
Joel (from the sim4life desktop team) has experienced that the api-server can sometimes hang in requests. I recently introduced timeouts in the api-server clients "speaking" with storage. So that is no longer the issue. I.e. the issue must be in the api-server itself. In order to reproduce I wrote a small script which uploads 100 different files concurrently via the api-server and observed that on my local deployment I could indeed make some of the requests hang. The api-server was still responsive, but some requests never received a response. I put a breakpoint in the api-server and investigated the tasks scheduled in the event loop and I could see that some of the "poller tasks" from the
disable_on_disconnectdecorator were missing their "twin" (these tasks are supposed to come in pairs). This suggests that, after the api-server had processed the request, it didn't manage to kill the "poller task" and the response is not being sent back to the client until that task is gone.Follow up
cancle_on_disconnectis indeed the issue then it is important that we find a proper solution because that guy is used everywhere.Related issue/s
POST /files/contentdoesn't hang in api-server osparc-issues#1922How to test
Dev-ops