Add permission list management #2819

KodaiD · 2025-06-25T06:58:19Z

Description

This PR adds integration tests for the list of permissions required by ScalarDB per underlying storage. The list is helpful for users to know what permissions they need where they have restricted permissions rather than full admin access.

Related issues and/or PRs

N/A

Changes made

Added new integration tests for DistributedStorage and DistributedStorageAdmin implementations:
- Cassandra
- DynamoDB
- JDBC (MySQL)
Created GitHub Actions workflow to run permission tests

Checklist

I have commented my code, particularly in hard-to-understand areas.
I have updated the documentation to reflect the changes.
I have considered whether similar issues could occur in other products, components, or modules if this PR is for bug fixes.
Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
Tests (unit, integration, etc.) have been added for the changes.
My changes generate no new warnings.
Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

N/A

Release notes

N/A

Copilot

Pull Request Overview

This PR adds permission-based integration tests for Cassandra, DynamoDB, and JDBC storage backends in ScalarDB. It introduces a PermissionTestUtils interface, two abstract test bases for storage and admin operations, concrete implementations for each backend, and updates build scripts and GitHub workflows to include these new tests.

New PermissionTestUtils interface and implementations for JDBC, DynamoDB, and Cassandra
Abstract test bases (DistributedStoragePermissionIntegrationTestBase and DistributedStorageAdminPermissionIntegrationTestBase) with concrete integration tests
Updated build.gradle and GitHub Actions workflow to run permission integration tests

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
integration-test/src/main/java/com/scalar/db/util/PermissionTestUtils.java	Defines common interface for creating/granting users
integration-test/src/main/java/com/scalar/db/api/DistributedStoragePermissionIntegrationTestBase.java	Adds base tests for storage operations with normal user
integration-test/src/main/java/com/scalar/db/api/DistributedStorageAdminPermissionIntegrationTestBase.java	Adds base tests for admin operations with normal user
core/src/integration-test/java/com/scalar/db/storage/jdbc/*	JDBC-specific `PermissionTestUtils`, tests, and env
core/src/integration-test/java/com/scalar/db/storage/dynamo/*	Dynamo-specific `PermissionTestUtils`, tests, and env
core/src/integration-test/java/com/scalar/db/storage/cassandra/*	Cassandra-specific `PermissionTestUtils`, tests, and env
core/build.gradle	Adds new source sets and tasks for permission tests
.github/workflows/test-permission.yaml	Workflow to run permission tests on GitHub Actions

Comments suppressed due to low confidence (2)

core/src/integration-test/java/com/scalar/db/storage/jdbc/JdbcPermissionTestUtils.java:72

[nitpick] There's an unintended trailing backslash in this comment. Please remove '\' for clarity.

integration-test/src/main/java/com/scalar/db/util/PermissionTestUtils.java:3

[nitpick] To support try-with-resources and clearer lifecycle management, consider having this interface extend AutoCloseable.

public interface PermissionTestUtils {

...on-test/src/main/java/com/scalar/db/api/DistributedStoragePermissionIntegrationTestBase.java

core/build.gradle

core/src/integration-test/java/com/scalar/db/storage/dynamo/DynamoEnv.java

KodaiD · 2025-06-27T05:35:50Z

.github/workflows/test-permission.yaml

@@ -0,0 +1,199 @@
+name: Test Permissions


For JDBC databases, only MySQL is tested for now.

KodaiD · 2025-06-27T05:37:11Z

.github/workflows/test-permission.yaml

+on:
+  workflow_dispatch:


Currently, the job is assumed to be run manually, for example, when a new release is made.

KodaiD · 2025-06-27T05:39:07Z

core/build.gradle

+            exclude '**/com/scalar/db/storage/cassandra/CassandraPermissionTestUtils.java'
+            exclude '**/com/scalar/db/storage/dynamo/DynamoPermissionTestUtils.java'
+            exclude '**/com/scalar/db/storage/jdbc/JdbcPermissionTestUtils.java'
+            exclude '**/com/scalar/db/storage/cassandra/CassandraPermissionIntegrationTest.java'
+            exclude '**/com/scalar/db/storage/dynamo/DynamoPermissionIntegrationTest.java'
+            exclude '**/com/scalar/db/storage/jdbc/JdbcPermissionIntegrationTest.java'
+            exclude '**/com/scalar/db/storage/cassandra/CassandraAdminPermissionIntegrationTest.java'
+            exclude '**/com/scalar/db/storage/dynamo/DynamoAdminPermissionIntegrationTest.java'
+            exclude '**/com/scalar/db/storage/jdbc/JdbcAdminPermissionIntegrationTest.java'


To avoid the permission tests are run in the CI, the test files are excluded.

KodaiD · 2025-06-27T05:59:35Z

...ation-test/java/com/scalar/db/storage/cassandra/CassandraAdminPermissionIntegrationTest.java

+  @Override
+  protected void waitForNamespaceCreation() {


Because CassandraAdmin does not have the wait for table creation mechanism, we need to prepare it in the test.

KodaiD · 2025-06-27T05:59:59Z

core/src/integration-test/java/com/scalar/db/storage/cassandra/CassandraEnv.java

    return properties;
  }
+
+  public static Properties getPropertiesForNormalUser(String testName) {


To test the permission list, a normal user is created and used for the test.

KodaiD · 2025-06-27T06:04:16Z

core/src/integration-test/java/com/scalar/db/storage/dynamo/DynamoEnv.java

To handle non-emulator DynamoDB, the option to distinguish between emulator and non-emulator is added.

KodaiD · 2025-06-27T06:09:00Z

.../src/integration-test/java/com/scalar/db/storage/dynamo/DynamoPermissionIntegrationTest.java

+
+  @Override
+  protected Map<String, String> getCreationOptions() {
+    return ImmutableMap.of(DynamoAdmin.NO_SCALING, "false", DynamoAdmin.NO_BACKUP, "false");


To test under the environment assuming the production environment, the scaling and the backup are enabled.

KodaiD · 2025-06-27T06:10:50Z

core/src/integration-test/java/com/scalar/db/storage/dynamo/DynamoPermissionTestUtils.java

+  @Override
+  public void createNormalUser(String userName, String password) {
+    // Do nothing for DynamoDB.
+  }
+
+  @Override
+  public void dropNormalUser(String userName) {
+    // Do nothing for DynamoDB.
+  }


The current implementation assumes that the same IAM user is used for tests. Only the IAM policy is updated for each test.

KodaiD · 2025-07-01T00:27:50Z

.github/workflows/test-permission.yaml

+
+on:
+  workflow_dispatch:
+  pull_request:


To ensure the workflow runs correctly, a pull request trigger is added. It will be removed before merging.

KodaiD · 2025-07-01T08:57:09Z

Since it is going to be a big PR, I will split it into smaller PRs for each backend.

KodaiD added 8 commits June 23, 2025 15:56

Add integration test for permission list

60b00ac

Merge branch 'master' into permission-list

88235e0

Add test config

22842fa

Add close() to scan call

45385d6

Fix DynamoEnv to provide a single getProperties method

61dd00f

Create user and grant permissions in tests

b89e66a

Divide the integration test into parts per storage

c4b7ff4

Add GitHub Actions workflow

f49de8f

KodaiD requested a review from Copilot June 25, 2025 06:58

KodaiD self-assigned this Jun 25, 2025

KodaiD added the improvement label Jun 25, 2025

Copilot AI reviewed Jun 25, 2025

View reviewed changes

...on-test/src/main/java/com/scalar/db/api/DistributedStoragePermissionIntegrationTestBase.java Outdated Show resolved Hide resolved

core/build.gradle Show resolved Hide resolved

core/src/integration-test/java/com/scalar/db/storage/dynamo/DynamoEnv.java Outdated Show resolved Hide resolved

KodaiD changed the title ~~Permission list~~ Add permission list management Jun 25, 2025

KodaiD added 5 commits June 26, 2025 16:13

Update grant method in Dynamo

52db096

Reformat

2d7cad5

Merge branch 'master' into permission-list

3daa2b6

Exclude permission test files from the existing test settings

4b8afae

Update tests

12f6928

KodaiD commented Jun 27, 2025

View reviewed changes

KodaiD added 3 commits June 27, 2025 16:23

Remove the unnecessary dependency

f42a6fe

Remove the unnecessary synchronization method

2cf06d5

Add pull_request trigger to test workflow

401730b

KodaiD commented Jul 1, 2025

View reviewed changes

KodaiD added 4 commits July 1, 2025 10:00

Fix to use conditional usrer-related queries

fe9d220

Fix to use custom yaml file for Cassandra to enable auth

7b4961b

Fix to sleep fixed time

1922d9c

Fix to use bitnami/cassandra image

0688b38

KodaiD closed this Jul 1, 2025

brfrn169 deleted the permission-list branch July 8, 2025 11:50

		on:
		workflow_dispatch:

Add permission list management #2819

Add permission list management #2819

Uh oh!

Conversation

KodaiD commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues and/or PRs

Changes made

Checklist

Additional notes (optional)

Release notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KodaiD Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KodaiD commented Jul 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

KodaiD commented Jun 25, 2025 •

edited

Loading

KodaiD Jun 27, 2025 •

edited

Loading