After upgrade of OSS install on Docker in EC2, permission errors on the /tmp/workspace directory

[techxorcist]

Good day.

I have upgraded an OSS instance of Airbyte from v0.50.35 to v0.57.2. All pods referred to in the docker-compose start successfully but any source or destination fails on execution of the check() method. In the airbyte-worker logs, there is a file permissions error:

ERROR StatusConsoleListener FileManager (/tmp/workspace/4de9764c-5e55-4c00-9cf6-f23c54d6db2d/0/logs.log) java.io.IOException: Could not create directory /tmp/workspace/4de9764c-5e55-4c00-9cf6-f23c54d6db2d/0

The /tmp folder is world-writeable in the instance so a file permissions error is puzzling.

Other points from my investigation:

• There is discussion of the same issue in Slack.

• The pull request for the draft March 2024 release notes mentions breaking changes to Docker and Helm deployments though the details in the linked-to information seem only concerned with the Helm deployment.

• The v0.57.2 docker-compose.yaml has some changes in the volume section for airbyte-worker (and -server and -proxy) since v0.50.35 but not to the WORKSPACE_ROOT:
  

• The .env the script loads and calls has similar change to a directory location reference but, again, hard to see the relation:
  

Any insight would be very helpful. Thank you.

[techxorcist]

Resolved with the explanation on the March 2024 release notes PR and clarification from @nataliekwong. As mentioned in this issue, deleting and recreating the airbyte_workspace volume can resolve the issue. It is also possible to resolve the issue by changing the ownership on /tmp with the following commands, run on the host machine:

docker exec -u root -it airbyte-worker /bin/bash
chown -R airbyte:airbyte /tmp

[pradosh-suar]

This issue is happening because of line 121 in DockerProcessFactory.java file which is trying to create a multilevel directory “/tmp/workspace/f54cc18e-ce16-406d-9c5b-b1fe2493cf73/0” using Files.createDirectory() method. This method is only used to create a subdirectory inside an existing directory not a chain.

Can you please change the code to use Files.createDirectories() method, this will solve the issue. Will be grateful if it get addressed little early, we are blocked.