Procedure for what to do if there's an incident with the Release Worker.
-
If the incident was caused by a recent change, try rollbacking the release.
-
If the incident affects traffic towards the Release Worker, update the Node.js status page (https://status.nodejs.org). If it is a ongoing security incident that we cannot disclose publicly yet, do not includes the details of the incident in the status page.
-
Optional, but preferably updates will be echoed on social media.
-
For any prolonged incidents, please consider pinning an issue tracking the incident so as to avoid spam.
-
Please also monitor any issues in repositories such as this one, nodejs/node, and nodejs/nodejs.org for users asking about the incident and link them to the status page.
-
-
If there is an ongoing security incident requiring code changes, a force push to the
mainbranch can be performed by a Collaborator if there is reasonable risk that opening a PR with the change would allow more bad actors to exploit the vulnerability. The code changes must still be approved by another Collaborator before the force push is performed, however. -
If the issue requires support from Cloudflare, try reaching out through the
ext-nodejs-cloudflarechannel in the OpenJS Slack. -
If needed, create an issue on this repository to serve as a discussion board for any changes that need to be made to avoid the same incident from happening again.
There is no exact criteria, however, these cases will most likely call for an incident to be declared:
-
The production deployment of the Release Worker is unavailable to the public or is otherwise operating in a way that impacts users' abilities to interact with it en masse. This includes behaviors that we are responsible for and those that Cloudflare is responsible for.
-
There is a ongoing security issue that involves the production deployment of the Release Worker.
Note the Node.js Web Infrastructure, Build, and TSC teams can declare an incident wherever they see fit, however.