Build: Decouple setup and exec in integration tests for better flexibility (#1985)

<!--
Thanks for opening a pull request!
-->

<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->

# Rationale for this change

Decouple setup and exec in integration tests for better flexibility and
isolation.

For example, currently, `make test-integration` must be executed in a
physical server.
It is because `docker xxx` cannot be executed within a container (docker
in docker is out of scope of this PR).
Sometimes, it is useful for local debugging, as poetry cannot provide
system level isolation, includes environment variables and shared libs.

This PR allows us to run `make test-integration-exec` within a container
environment, after we have run `make test-integration-setup` on this
server.

# Are these changes tested?
Existing tests.

# Are there any user-facing changes?

No.

Author: frankliee

Makefile

@@ -52,13 +52,17 @@ test-s3: # Run tests marked with s3, can add arguments with PYTEST_ARGS="-vv"
 	sh ./dev/run-minio.sh
 	poetry run pytest tests/ -m s3 ${PYTEST_ARGS}
 
-test-integration: ## Run all integration tests, can add arguments with PYTEST_ARGS="-vv"
+test-integration: | test-integration-setup test-integration-exec ## Run all integration tests, can add arguments with PYTEST_ARGS="-vv"
+
+test-integration-setup: # Prepare the environment for integration
 	docker compose -f dev/docker-compose-integration.yml kill
 	docker compose -f dev/docker-compose-integration.yml rm -f
 	docker compose -f dev/docker-compose-integration.yml up -d
 	sleep 10
 	docker compose -f dev/docker-compose-integration.yml cp ./dev/provision.py spark-iceberg:/opt/spark/provision.py
 	docker compose -f dev/docker-compose-integration.yml exec -T spark-iceberg ipython ./provision.py
+
+test-integration-exec:  # Execute integration tests, can add arguments with PYTEST_ARGS="-vv"
 	poetry run pytest tests/ -v -m integration ${PYTEST_ARGS}
 
 test-integration-rebuild: