Perform solve request in the main gateway call

[jedevc]

🛠️ Replaces #1750. This is a lot neater than the original implementation and doesn't require that we go and change the entire API.

Now, we always perform the full solve request in the main gateway call. This ensures that progress works properly (and doesn't stagger), and makes the lifetime semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to always return a single ResultContext, even though the details of how a gateway is created is different:

• For a failed build, we can just keep the gateway open.
• For a successful build, we immediately open another gateway and re-evaluate the build definition in that gateway. This should give an instant cache hit (since the build was just successful).

[ktock]

Thanks!

[jedevc]

A bit more tidying, I've removed one of the nested goroutines which wasn't necessary, and tidied up the calls to cancel and close(baseCh) by using a defer.

I think it might be possible to try and merge the done and solve status baseCh, but I've held off doing that, I think that might get a little confusing.

[tonistiigi]

Let's get this in for the release 🤞 , but this area needs a follow-up refactoring. We need to take a step back, create a design issue that describes all the cases this code needs to solve and current code paths. Then put together a graph of components and function signatures with their scope description. If needed, we need to wrap or refactor the buildkit client.Build() signature if it is too complicated to use in these new cases.