Commit ceac900
authored
Replace Docker with Docker Compose for running the DB to be targeted by integration tests (#9)
* Prepare the grounds for using Docker Compose when running integration tests
* Improve script used for checking container health state
* Add support for enabling printing info for debugging purposes
* Integration tests are now targeting a database run via Docker Compose
* Fix incorrect YAML property
* Remove comments from YAML fragment as Azure DevOps complains about them
* Fix path pointing to Docker Compose file
* Add pipe symbol as requested by Docker Compose task
* Enable debugging for Docker Compose services
* Use hard-coded sleep to ensure compose services are up when running IT
* Fix YAML indentation
* Give enough time to PostgreSQL DB to start before running IT
* Tweak Docker Compose task parameters
* Ensure compose services are healthy before running IT
* Fix Docker Compose command used for checking service status
* Use PowerShell Core to interact with Docker Compose
* Fix passing environment variables to PowerShell Core script
* Fix PowerShell Core script syntax
* Fix Azure DevOps logging command syntax used for displaying info messages
* Temporarily hard-code host port for the db compose service
* Provide a default database port for the template job file
* Log database host port for debugging purposes
* Identify the Docker host port associated with the 'db' compose service
* Publish test log file for debugging purposes
* Publish test log file for debugging purposes
* Temporarily log IT connection string for debugging purposes
* Fix typo in parameter reference
* Remove logging connection string as CI pipeline has been fixed
* Fix typo
* Use PowerShell script to start compose services used by IT
* Remove no longer needed build step
* Wrap environment variable values with single quotes
* Remove > symbol since PowerShell was misinterpreting it
* Split PowerShell long line via tick symbol
* Un-ingore .env file to be used by Docker Compose
* Avoid breaking build due to Docker Compose writing to standard error
* Fix referencing variables storing connection strings to be used when running IT
* Fix variable name by adding missing dot symbol
* Fix referencing variables in steps related to running IT
* Move PowerShell script used for starting compose service to a better location
* Move PowerShell script used for starting compose service to a better location
* Move PowerShell script used for starting compose service to a better location
* Move PowerShell script used for starting compose service to a better location
* Remove invalid string
* Add comments and validation rules to PowerShell script parameters
* Fix displaying message about mapping container port to host one
* Fix displaying variable value after mapping container port to host one
* Made Docker Compose service names more explicit
* Add macOS agent
* Fail script if Docker Compose writes anything to standard error stream
* Fail build in case compose services fail to start
* Fix handling Docker Compose command standard streams
* Fix redirecting all streams to null
* Fix redirecting success,warning and error streams to null
* Temporarily disable build on macOS agents till the pipeline is working as expected
* Replace redirecting Docker Compose streams with --log-level flag
* Replace redirecting Docker Compose streams with --log-level flag, take 2
* Replace redirecting Docker Compose streams with --log-level flag, take 3
* Replace redirecting Docker Compose streams with --log-level flag, take 4
* Detect errors while running Docker Compose via $LASTEXITCODE
* Fix exit condition when everything it's OK
* Re-enable macOS agent
* Install Docker Compose via brew on macOS agent
* Revert commit since compose issue occurring on macOS agent is caused by older Docker version
* Use compose file v3.7 since macOS agent cannot run a newer version of Docker compatible with v3.8+
* Fix incrementing the counter used when checking whether Docker service has started
* Enable Windows-based agent too
* Fix validating input parameter
* Parameterize the name of the Docker image used for running the DB to be targeted by IT
* Fix passing arguments to PowerShell script
* Explicitly set string properties to avoid issues due to invalid types
* Fix reference to compose variable
* Log more info for debugging purposes
* Log more info for debugging purposes, take 2
* Wrap parameter values with single quotes
* Explicitly set job parameters to avoid pipeline considering them not present
* Remove no longer needed logging messages
* Fix parameter name in XML doc
* Disable ReSharper warning about unused variable since it will be used at some point
* Add new word to project dictionary
* Avoid possible NullReferenceException when running integration tests
* Disable ReSharper warning since I'm testing whether the constructor will throw an exception
* Since compose services are run as part of CI pipeline, no need for attaching a console
* Add extra link to article describing the meaning of 'xmin' property in an EF Core entity
* Remove redundant environment variables
* Cache NuGet packages when requested only
* Remove unused 'databasePort' job parameter
* Remove extra whitespaces
* Restoring NuGet packages happens only in case there is no cache for them
* Fix condition by adding missing round bracket
* Fix condition used for evaluating whether restoring NuGet packages is performed or not
* Remove whitespaces since they break the commands
* Remove job parameters default values since they bring no real value right now
* Update .NET Core SDK to v3.1.302
* Update ReportGenerator to v4.6.4
* Rollback updating ReportGenerator to v4.6.4 since Azure DevOps feed doesn't have it yet
* Try installing latest available version of Docker via homebrew
* Rollback installing latest version of Docker since macOS based agent does not support it
* Replace hard-coded value with variable
* Set default value for job parameter to avoid code duplication
* Log connection string pointing to test database for debugging purposes
* The databaseHost job parameter must be explicitly set
* Experiment with specifying volume for the DB to be targeted by integration tests
* Experiment with specifying volume for the DB to be targeted by integration tests, take 2
See more about this approach here: https://stackoverflow.com/a/55366707
* Experiment with specifying volume for the DB to be targeted by integration tests, take 3
* Fix Docker Compose volume related error occurring on Windows based agents
See more here: docker/compose#5842 (comment).
* Add missing key-value pair separator
* Wrap environment variable in single quotes hoping
things will work out :)
* Experiment with specifying volume for the DB to be targeted by integration tests, take 4
* Set volume path according to the Docker container (Linux or Windows)
* Make script starting compose services more robust
* Fix splitting long line in PowerShell script
* Make script starting compose services more robust, take 2
* Fix creating a custom object in PowerShell
* Fix Docker Compose command broken due to formatting code
* Fix referencing true and false boolean values
* Fix referencing container id when fetching port mappings
* Wrap container port with double quotes hoping port mappings will be fetched on Windows
* Specify host port to avoid Windows-based Azure DevOps agents not fetching any port mappings
* Let again Docker assign a dynamic host port
* Replace identifying ports via 'docker port' with 'docker inspect'
* Fix variable name
* Fix fetching all JSON properties
* Create Docker volume before starting compose services
AFAIK, creating the volume may be needed before starting compose services on Windows-based Azure DevOps agents since PostgreSQL complains that its folder has different permissions than the expected ones
* Fix declaring named volume
* Explicitly set durations used by Docker Compose
* Wrap strings in double quotes
* Rename job parameter to increase code readability
* Display message about creating Docker volume to make the build step more transparent
* Fix passing job parameter as PowerShell script parameter value
* Ignore comments and empty lines found in Docker Compose environment file
* Expose Docker Compose logs as build artifact for debugging purposes
* Let the PowerShell script handle errors
* Publish Docker Compose logs as build artifact
* Publish Docker Compose logs as build artifact, take 2
* Display Docker Compose logs
* Publish Docker Compose logs via build step
* Fix splitting long line into multiple shorter ones
* Fix publishing Docker Compose logs
* Create missing folders if needed when writing Docker Compose log file to disk
* Create missing folders if needed when writing Docker Compose log file to disk, take 2
* Ensure Docker Compose writing to standard error stream does not break script
* Remove no longer needed Azure DevOps command
* Add compose project name to the name of the compose service log file being published
* Add missing parameter when creating Docker volumes
* Print Azure DevOps-style errors when Docker commands fail
* Add timestamps to the published Docker Compose logs
* Avoid checking whether commands have ended with success since they write to stderr
* Fix container path passed as job parameter value
* Publish compose service log files in a folder per Azure DevOps agent OS
* Fix container path
* Fixcommand used for creating compose service volume
* Make use of PGDATA env var in order to fix Windows-based agent error
More specifically, this is the output of the db4it compose service:
2020-08-08T18:00:53.256999700Z initdb: error: could not access directory "C:/pgsql/data": Permission denied
2020-08-08T18:00:53.292312700Z Access is denied.
2020-08-08T18:00:53.292312700Z Access is denied.
2020-08-08T18:00:53.292312700Z The system cannot find the drive specified.
2020-08-08T18:00:55.477969200Z psql: error: could not connect to server: could not connect to server: Connection
refused (0x0000274D/10061)
2020-08-08T18:00:55.477969200Z Is the server running on host "localhost" (::1) and accepting
2020-08-08T18:00:55.477969200Z TCP/IP connections on port 5432?
2020-08-08T18:00:55.477969200Z could not connect to server: Connection refused (0x0000274D/10061)
2020-08-08T18:00:55.477969200Z Is the server running on host "localhost" (127.0.0.1) and accepting
2020-08-08T18:00:55.477969200Z TCP/IP connections on port 5432?
* Fix container path
* Fix path pointing to test related artifacts to be published
* Fix container path, take 2
* MOve .artifactignore file to repo root, as recommended by official doc
See more here: https://docs.microsoft.com/en-us/azure/devops/artifacts/reference/artifactignore?view=azure-devops
* Fix container path, take 3
Fix error:
ERROR: for db4it Cannot create container for service db4it: invalid volume specification: 'db4it_data:C:\\db4it:rw'
* Simplify folder path storing published compose service log file
* Tune a little bit the health check durations
* Print message for debugging purposes
* Rephrase message to include compose project name
* Tune a little bit printing message for debugging purposes
* Fix printing message
* Fix printing another message
* Ensure no PowerShell errors goe unnoticed
* Need to set errorActionPreference to Continue and failOnStderr to False to overcome compose issue
See more about this issue here: docker/compose#5590.
* Remove db4it volume - let's see what happens
* Remove any trace of Docker volume since PostgreSQL compose service does not start on Windows
* Polish messages printed for debugging purposes
* Refine publish artifact exclusion and inclusion rules
* Reduce number of if-statements
* Move .artifactignore file under Tests folder to limit what gets published
* Change build step id and name to better reflect its purpose
* Ensure compose project name is unique across each builds
* Use minimum amount of info to ensure unique compose project names across all builds
* Remove no longer used variable
* Rename compose project name to follow the Azure DevOps user-defined variable naming convention
See this convention here: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#variable-characters.
* Make compose project name even shorter
* Ensure compose project name contains letters only, as using numbers (e.g. build ID) does not work
* Remove job parameter duplication
* Ensure job parameter is set outside job template file to avoid considering it empty string
* Revert compose project name to last working value
* Specify job parameter value outside job template to ensure it will not be seen as empty string
* Fix variable name used as example
* Change variable name to correctly state its purpose
* Temporarily expose more ports for testing purposes
* Revert multiple ports to just one
* Make use of 'docker container <VERB>' commands
* Declare external volume for the compose service
* Remove volume since build fails on Windows1 parent b45bbdc commit ceac900
File tree
14 files changed
+563
-395
lines changed- Build
- db4it-compose
- Sources/Todo.Persistence/Entities
- Tests
- IntegrationTests/Todo.WebApi.IntegrationTests/Infrastructure
14 files changed
+563
-395
lines changedThis file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
0 commit comments