Docker Integration Test Strategy

Using Docker containers for integration testing is great, but there are two logical ways to approach it:

• Using docker compose
• Using a Docker API wrapper such as testcontainers or docker-java (other languages have their own wrappers too)

You can also use both together. The testcontainers library has limited support for launching from a compose file. Similarly, if you manually launch via docker compose you could then use the docker-java library to control the containers from within your tests.

When troubleshooting tests, I've found the docker compose route to be nicer because you can launch the containers you need and leave them running while you run your tests over and over. With the testcontainers approach you're constantly stopping and starting containers each time you run tests, which is slow.

Another advantage of the docker compose route is you can define a dedicated tester container to run the tests in. I don't believe this is a readily available mode of operation with testcontainers. The library supports running inside a container, but I believe that means you are still responsible for bootstrapping it, which means you're likely using docker compose and it isn't clear this is the cleanest way to have tests run in a sibling container. To clarify, this is a somewhat more complicated use-case, but important when the test runner must run in the same network as the containers the test depend on. For example, EPICS CA uses UDP broadcasts to communicate by default, which only reliably works if on the same network subnet. Note: there are workarounds like using host network mode when launching containers, or using a CA Nameserver or CA Gateway, or avoiding the CA protocol and using execInContainer instead, but these workarounds seem worse than simply running tests in a new test runner container in the same network.

Another advantage of docker compose is I'm already using it for development and demo purposes. Using docker compose for integration testing means I don't need to learn a wrapper API and all it's idiosyncrasies. This is even more valuable if you're using multiple programming languages and you're trying to use a common strategy for integration testing. For example, I have Java, Python, and NodeJS apps. It would be nice to have a common way to run integration tests. The testcontainers API for each language is slightly different (and feature support varies too). The compose file format is the same regardless of project language.

Note: This has been discussed before here.

Note: integration tests should be able to run easily on a CI server such as GitHub Actions, plus on a local workstation during development.