Cleaning up when project creation fails

[dave-shawley]

One of the issues that I created when I implemented project creation automations is that if one of them fails, it leaves anything that has already been created dangling which is a bad user experience IMO. The current implementation is a list of "things to do" in React along with state that describes the outcome of each step. This is currently about 150 lines of JS for the action-related logic, another 50 or so for the state configuration, and another 50 or so for the Modal dialog. That isn't so bad; however, the "action-related logic" portion is a little hairy since it has to be careful about the order and has dependencies woven throughout.

Cleaning up already executed actions could be implemented by running backwards through the already executed steps and adding knowledge about how to "undo" each of them. We could codify this somewhat by having a pattern to the undo URLs -- e.g, POST /ui/automations/gitlab/create would be paired with POST /ui/automations/gitlab/destroy passing the body from the create response. I would like to investigate an API structural change to make this a little less opinionated and require less work on the side of the client.

The option that I like the most is a variant on the standard two-phase commit.

1. React client creates a new project in "creating" mode
2. React client adds automations with the ability to retrieve a specific automation steps status or retrieve the status of all running automations for a project
3. React client waits for the automations to complete by polling the API
4. React client "commits" the project when the automations complete

• If the React client doesn't commit within some time period, then the API can roll back the automations.
• We would need a "state" for projects to filter out things that are in-progress and prevent updates to them from outside actors

The sequence would look something like the following sequence. This looks quite a bit more complex than it actually is. The automations are hypermedia driven so we can actually implement this off to the side of the Imbi API server if we want.

https://www.websequencediagrams.com/files/render?link=fQvw6rDp9r2ViO3F0Q77V3d3R4dL0fQX9DHZehnFOXACVWJ1XPvOWYoUCP3pGs3O

Notes

I kept the automation results in a separate database table and explicitly do not delete them when the automation completes. We can use these to clean up when we delete an Imbi project in the future should we care to.

Don't read too much into the "async process" part. I'm not sure how durable these have to be. What will it look like when we have multiple Imbi API services running? The automations are only accessible to the FE as links so we can target specific instances and spawn a Tornado callback or run a sub-process and keep its PID. In any case, I can see good ways to make this "safe" even in the face of scaling outward.