Skip to content

Conversation

@HeartLinked
Copy link
Contributor

@HeartLinked HeartLinked commented Nov 28, 2025

Add integration testing infrastructure for the REST Catalog:

  • Docker Compose setup (docker-compose.yml) for running a real REST catalog server
  • Test utilities for process management (cmd_util.cc/h) - run cli commands
  • Docker Compose control utilities (docker_compose_util.cc/h) - managing containerized services
  • Created src/iceberg/test/util/ directory.
  • Modified GitHub CI to support integration tests.
  • These test cases(cmd / docker compose related)only work on Linux/MacOS, added conditional compilation flags for Linux-specific test utilities.

close #333 .

@HeartLinked HeartLinked changed the title test: Implement REST Catalog Integration Tests with Docker in CI Pipe… test: add REST Catalog Integration Tests with Docker Nov 28, 2025
@HeartLinked HeartLinked force-pushed the test/integration-rest branch 3 times, most recently from fb22743 to 19b9859 Compare November 28, 2025 10:31
@HeartLinked HeartLinked marked this pull request as ready for review November 28, 2025 10:55
Copy link
Member

@wgtmac wgtmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran it on my MacOS and fall test cases failed.

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from RestCatalogIntegrationTest
[INFO] Starting to docker compose up, command: docker compose -p iceberg-rest-catalog-service up -d --wait --timeout 60
 Network iceberg-rest-catalog-service_rest_bridge  Creating
 Network iceberg-rest-catalog-service_rest_bridge  Created
 Container iceberg-rest-catalog-service-rest-1  Creating
 Container iceberg-rest-catalog-service-rest-1  Created
 Container iceberg-rest-catalog-service-rest-1  Starting
 Container iceberg-rest-catalog-service-rest-1  Started
 Container iceberg-rest-catalog-service-rest-1  Waiting
 Container iceberg-rest-catalog-service-rest-1  Healthy
[INFO] docker compose up succeed!
[INFO] Waiting for REST catalog to be ready at localhost:8181...
[INFO] REST catalog is ready!
[ RUN      ] RestCatalogIntegrationTest.MakeCatalogSuccess
/Users/gangwu/Projects/iceberg-cpp/src/iceberg/test/rest_catalog_test.cc:136: Failure
Value of: catalog_result
Expected: is an Ok result
  Actual: 40-byte object <0F-00 00-00 00-00 00-00 20-C0 74-03 00-60 00-00 1D-00 00-00 00-00 00-00 20-00 00-00 00-00 00-80 00-00 00-00 00-00 00-00>, which contains error: Failed to parse JSON string:

[  FAILED  ] RestCatalogIntegrationTest.MakeCatalogSuccess (53 ms)
[ RUN      ] RestCatalogIntegrationTest.ListNamespaces
/Users/gangwu/Projects/iceberg-cpp/src/iceberg/test/rest_catalog_test.cc:144: Failure
Value of: catalog_result
Expected: is an Ok result
  Actual: 40-byte object <0F-00 00-00 01-00 00-00 00-80 74-03 00-60 00-00 1D-00 00-00 00-00 00-00 20-00 00-00 00-00 00-80 00-C0 74-03 00-60 00-00>, which contains error: Failed to parse JSON string:

[  FAILED  ] RestCatalogIntegrationTest.ListNamespaces (34 ms)
[ DISABLED ] RestCatalogIntegrationTest.DISABLED_GetNonExistentNamespace
[ DISABLED ] RestCatalogIntegrationTest.DISABLED_CreateAndDropNamespace
[ DISABLED ] RestCatalogIntegrationTest.DISABLED_UpdateNamespaceProperties
[ DISABLED ] RestCatalogIntegrationTest.DISABLED_FullNamespaceWorkflow
[INFO] Starting to docker compose down, command: docker compose -p iceberg-rest-catalog-service down -v --remove-orphans
 Container iceberg-rest-catalog-service-rest-1  Stopping
 Container iceberg-rest-catalog-service-rest-1  Stopped
 Container iceberg-rest-catalog-service-rest-1  Removing
 Container iceberg-rest-catalog-service-rest-1  Removed
 Network iceberg-rest-catalog-service_rest_bridge  Removing
 Network iceberg-rest-catalog-service_rest_bridge  Removed
[INFO] docker compose down succeed!
[----------] 2 tests from RestCatalogIntegrationTest (87 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (4975 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] RestCatalogIntegrationTest.MakeCatalogSuccess
[  FAILED  ] RestCatalogIntegrationTest.ListNamespaces

@HeartLinked HeartLinked force-pushed the test/integration-rest branch from e82c13a to 962c65b Compare December 3, 2025 04:02
@wgtmac wgtmac changed the title test: add REST Catalog Integration Tests with Docker test: add rest catalog integration test Dec 3, 2025
@wgtmac wgtmac merged commit b1901e7 into apache:main Dec 3, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement REST Catalog Integration Tests with Docker in CI Pipeline

3 participants