-
Notifications
You must be signed in to change notification settings - Fork 83
Description
Problem
The Presto Docker Compose project and the CLP Package Docker Compose project use different conventions for file mounting, which causes Presto queries to return zero rows.
Current Behavior
- Presto project (PR feat(presto-clp): Add Docker compose setup for Presto cluster that can connect to clp-json. #1132): Mounts the host's
/path/to/var/data/archivesdirectory (or/path/to/var/data/staged-archiveswhen using S3 output) into the container at the same absolute path as on the host. - CLP Package project (PR feat(deployment)!: Migrate package orchestration to Docker Compose (resolves #1177); Temporarily remove support for multi-node deployments. #1178): Always mounts the host's
/path/to/var/data/archivesdirectory to/var/data/archivesin the container, and/path/to/var/data/staged-archivesto/var/data/staged-archives.
Root Cause
With the clp-s storage engine, when archives are compressed through the Package project, the metadata DB's clp-datasets table stores the archive_storage_directory field using container paths (e.g., /var/data/archives) instead of the corresponding host paths (e.g., /path/to/var/data/archives). This mismatch prevents the Presto coordinator from locating the archives, as they are not mounted under /var/data/archives in the Presto container.
Impact
Presto queries return zero rows because the coordinator cannot locate the archives due to path mismatches.
Historical Context
Before PR #1178, when the components were orchestrated via Python subprocess calls to the docker CLI, they followed the same mapping convention as the current Presto project.
References
- PR feat(presto-clp): Add Docker compose setup for Presto cluster that can connect to clp-json. #1132 (Presto Docker Compose setup)
- PR feat(deployment)!: Migrate package orchestration to Docker Compose (resolves #1177); Temporarily remove support for multi-node deployments. #1178 (CLP Package Docker Compose project)
- Original comment: feat(presto-clp): Add Docker compose setup for Presto cluster that can connect to clp-json. #1132 (comment)