Use native binaries rather than docker

[joe-p]

This has been discussed on a couple of times, but I want to raise the topic again because I think there's an approach we can take that dramatically reduces the friction for the "first-touch" dev experience.

The main rationale for using docker is because of the postgres DB indexer needs. Having users install Postgres on bare-metal is unreasonable. The indexer, however, is not needed for smart contract development (only full Dapp development). As such, I think it would be reasonable for the default installation of sandnet to not include indexer. Without needing indexer we can then use native binaries.

The main sticking point with trying to use native binaries is not having Windows binaries. To alleviate this, we could have the Windows version of algokit use WSL when ran on Windows. The requirement for WSL is still not ideal, but it's better than WSL + Docker.

If a dev wants to eventually use Indexer, there can be an optional flag that creates a docker container on the host network that can communicate with the local algod. Eventually we can hopefully write a Conduit plugin that uses SQLite instead of Postgres and have everything native.

[robdmoore]

I agree that indexer isn't always needed, but it's handy for more than just dApp development. It's also incredibly handy for automated testing and also for powering things like dAppFlow so users can visualise what's happening on their sandbox.

The new Docker setup we have implemented spins up in seconds, and while it requires docker that's a straightforward install on Mac, Linux and Windows (Docker itself is straightforward - it's WSL that's the problem which in the above example is still an issue). It's worth calling out, your spiked out approach with msys2 is interesting as a potentially more lightweight Windows option... But a 25 min build time isn't the right experience of course. In my opinion, having native Windows binaries is essential to get the experience we want and a prerequisite to seriously explore this.

We could deploy the native binary approach for non-Windows of course, but then the experience differs across operating systems making support (and the CLI itself) more complex and goes against the AlgoKit principle of meeting devs where they are.

As I've said previously, I want us to support native binary versions of Algod as well as Docker because I think as well as not requiring Docker it opens up interesting possibilities around things like high scale/speed parallel testing etc. Right now though I think the Docker experience we have implemented provides the best cross-platform experience (noting there will be a small proportion of users who may/will struggle with WSL installation). Long term if we get feature parity with the Docker setup then maybe we can remove the Docker stuff altogether at that point? That would require a sqlite experience for indexer though as you mention.

[joe-p]

I agree that indexer isn't always needed, but it's handy for more than just dApp development. It's also incredibly handy for automated testing and also for powering things like dAppFlow so users can visualise what's happening on their sandbox.

What scenarios would indexer be used for testing over algod? Beaker, which is what I assume most people would be writing their automated tests with, only uses algod. Point taken for explorers though, forgot about that.

How does the following sound?

goal, algod, kmd - All ran as native binaries (Windows uses WSL until we get binaries)
indexer, postgres - Ran in docker (uses host networking so it can communicate with native binaries)

By default, algokit sandbox start will download the binaries, a pre-created network, and the indexer docker images. If docker is not installed, a warning message saying something along the lines of docker not installed. Network will be created without indexer. Proceed with network creation? (Y/n). There could also be an option algokit sandbox start --no-indexer which will not even attempt to download the docker images.

This gives us the best of both worlds. The "first-touch" experience can be incredibly low-friction as it doesn't depend on docker, while also providing the users with indexer if needed. The former is crucial for things like workshops.

Another major benefit to using native binaries is interaction with local filesystems when using goal. With docker you need to either manually copy in files or use mount points. Both of which have some serious drawbacks.

Edit: Also worth mentioning Docker itself isn't without issue either. On my M1 Pro MBP Docker Desktop would often completely hang when restoring from sleep. GUI would be completely unresponsive and the Unix sock would be unavailable. This requires force quitting to fix. Even using colima, which is another runtime for managing the QEMU VMs used for docker, ran into the same issue (albeit at a lower frequency). I suspect it has something to do with Apple's VM powerstate management. In general, requiring users to use a VM for something is not ideal, which is exactly what Docker on Windows and Mac does.

[Loedn]

As most of the problems we encounter with windows users are WSL related and there is no official support for windows native binaries, the only viable solution in this project initial timeline is keeping sandbox in a container. For future iterations of AlgoKit we will evaluate other options, community feedback will be crucial in making this decision moving forward.

[joe-p]

As most of the problems we encounter with windows users are WSL related and there is no official support for windows native binaries

But we can use native binaries in WSL. Yes we still have the dependency for WSL, but it reduces overall complexity and it dramatically simplifies the experience for Mac/Linux users. We're also strongly pushing to get something in terms of binaries for Windows, whether they are official from Inc or not.

the only viable solution in this project initial timeline is keeping sandbox in a container.

I question whether pivoting to native binaries dramatically impacts timelines given the current state of development. With containers there is also the extra dev effort of managing local files with goal. A bit of a tangent, but I really don't think we should be supporting goal commands if we don't have a really good solution for interacting with the host file system (and I don't think there really is a good solution unless we spend additional dev hours supporting custom GID/UIDs during container runtime). copyTo/copyFrom is confusing for developers, especially since all of the documentation on goal is written with the assumption it has access to the filesystem.

community feedback will be crucial in making this decision moving forward.

The community feedback is already here. A lot of devs have problems related to docker. Granted algokit and pre-made images will solve some of these issues, but the hard install dependency is still there.

Update 1/10/23: For the greenhouse bootcamp and likely other workshops our current solution is to use Gitpod. Currently our gitpod uses algodeploy under the hood, but once we are able to use algokit we will migrate to using Gitpod + Algokit

The Issue is far(!) too dense (I did not read through).

Just giving a 2nd opinion re

Use native binaries rather than docker

Quick: do not!

@robdmoore , I find the choice to use docker very positive. Docker is a quasi-standard, essentially a must-have in todays IT landscape. And it saves much trouble.

The sandboxing thing (if it operates as I think it does): excellent design.

(I would suggest to close this issue, and if anyone feels the need to redesign the whole thing, new separate issues should be filed)

[joe-p]

And it saves much trouble.

While this is true in some contexts, it's not always true. The necessary binaries run on bare metal without any dependencies needed. In the case of Algokit, docker adds more complexity than it removes. If we used native binaries, there wouldn't need to be wrappers around docker compose commands and we would not have the UX pains of copying files to/from containers or some of the ioctl errors that users run into when working with passwords. Then of course there's the pain of installing docker in the first place (particularly on Windows) and the performance sacrifices (primarily IO on Windows/Mac).

And it saves much trouble.

While this is true in some contexts, it's not always true.
[...] - (not reading through)

I've extracted the essence, and gave my feedback.

It ends here for me.

Seeing the setup effort #232 (comment), and remembering my personal struggle with docker when I first used it (it can take days or weeks to get fluent with docker), then "going full native binary without docker" could(!) be a valid goal (valid as in "worth the effort").