Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/workflows/data-loader-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: CI for CLI

on:
pull_request:
workflow_dispatch:

jobs:
data_loader_ci:
name: Build Data Loader CLI
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Set version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be missing something, but why is this needed?

Also, there are some differences between the original ci.yaml's check job and this new workflow. Can you update this workflow based on the original ci.yaml? I guess it might fix the weird write permission issue.

id: version
run: |
VERSION=$(echo "${GITHUB_REF}" | sed -e "s#refs/tags/v##g")
echo "version=${VERSION}" >> $GITHUB_OUTPUT
- name: Set up JDK 8 and 11 (default 8)
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: |
11
8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it necessary to set up both Java 8 and 11 ?

- name: Cache Gradle packages
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may miss something, but if you don't require a custom caching logic, using the setup-gradle action is usually preferred for a grade build compared to using the action/cache action.

Suggested change
- name: Cache Gradle packages
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

- name: Gradle Data Loader CLI build
run: ./gradlew :data-loader:cli:build --no-daemon --stacktrace

- name: Cleanup Gradle cache
run: |
rm -f ~/.gradle/caches/modules-2/modules-2.lock
rm -f ~/.gradle/caches/modules-2/gc.properties
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a newline?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you follow my suggestion of using gradle/actions/setup-gradle@v4, this is not necessary.

Suggested change
- name: Cleanup Gradle cache
run: |
rm -f ~/.gradle/caches/modules-2/modules-2.lock
rm -f ~/.gradle/caches/modules-2/gc.properties

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

1 change: 1 addition & 0 deletions data-loader/cli/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ javadoc {

// Build a fat jar
shadowJar {
mustRunAfter distZip, distTar
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is added via other PR so can be ignored here.

archiveClassifier.set("")
manifest {
attributes 'Main-Class': 'com.scalar.db.dataloader.DataLoaderCli'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,12 @@
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.PosixFilePermission;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.Stream;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.slf4j.Logger;
Expand Down Expand Up @@ -44,13 +48,25 @@ void validateOrCreateTargetDirectory_DirectoryDoesNotExist_CreatesDirectory()
@Test
void validateOrCreateTargetDirectory_DirectoryNotWritable_ThrowsException() throws IOException {
Path readOnlyDirectory = Files.createDirectory(Paths.get(tempDir.toString(), "readOnlyDir"));
readOnlyDirectory.toFile().setWritable(false);

// Try to make it read-only using POSIX if supported
Copy link
Contributor

@komamitsu komamitsu Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the cause of the test failure is still unclear, so I think we shouldn't merge this kind of ad-hoc workaround.

I ran ./gradlew :data-loader:cli:build --no-daemon --stacktrace without this workaround with the latest data-loader-ci.yaml, but it didn't fail
https://github.com/scalar-labs/scalardb/actions/runs/15702570827/job/44240661813. I guess the updated workflow file may have resolved the issue. Can you try without this workaround again just in case?

Even if it still fails, the unit tests should be already executed in ci.yaml's check job, so I think we can skip the unit tests in data-loader-ci.yaml by using ./gradlew -x test like ./gradlew -x test :data-loader:cli:build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inv-jishnu (If you missed this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@komamitsu san (cc @ypeckstadt ),
Sorry I missed this. You are right about the tests already executing in check job. And as I mentioned, since the test failure only occurs on my local environment with act, it might be some issue with my environment and we can ignore that.
Thank you for pointing this out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change above was reverted by @ypeckstadt san in this commit 6def419.

try {
Set<PosixFilePermission> perms = new HashSet<>();
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_EXECUTE);
Files.setPosixFilePermissions(readOnlyDirectory, perms);
} catch (UnsupportedOperationException | IOException e) {
// Fall back for systems without POSIX support
readOnlyDirectory.toFile().setWritable(false);
}
// Verify it is actually non-writable
boolean isWritable = Files.isWritable(readOnlyDirectory);
Assumptions.assumeFalse(isWritable, "Directory is still writable; skipping test.");

// Test
assertThrows(
DirectoryValidationException.class,
() -> {
DirectoryUtils.validateOrCreateTargetDirectory(readOnlyDirectory.toString());
});
() -> DirectoryUtils.validateOrCreateTargetDirectory(readOnlyDirectory.toString()));
}

@Test
Expand Down