Skip to content

Enhancement/upload multiple files #393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,22 @@ In this case, `myfolder/folder2/file2.txt` is the only matched file and will be

If `parent` is set to `false`, it wil be uploaded to `gs://bucket-name/folder2/file2.txt`.

### Upload Multiple Files

To upload multiple specific files, use the YAML pipe (`|`) syntax to specify multiple paths:

```yaml
- name: Upload specific files to GCS
uses: google-github-actions/upload-cloud-storage@v2
with:
path: |
file1.json
file2.json
path/to/another-file.txt
destination: 'bucket-name'
parent: false
```

## Inputs

<!-- BEGIN_AUTOGEN_INPUTS -->
Expand Down
12 changes: 11 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ inputs:
path:
description: |-
The path to a file or folder inside the action's filesystem that should be
uploaded to the bucket.
uploaded to the bucket. You could also specify multiple paths by separating them
with newlines.

You can specify either the absolute path or the relative path from the
action:
Expand All @@ -54,6 +55,15 @@ inputs:
```yaml
path: '../path/to/file'
```

To upload multiple specific files:

```yaml
path: |
file1.json
file2.json
path/to/file3.txt
```
required: true

destination:
Expand Down
2 changes: 1 addition & 1 deletion dist/index.js

Large diffs are not rendered by default.

25 changes: 10 additions & 15 deletions src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,7 @@ import * as path from 'path';

import { Client } from './client';
import { parseHeadersInput } from './headers';
import {
absoluteRootAndComputedGlob,
deepClone,
parseBucketNameAndPrefix,
expandGlob,
} from './util';
import { deepClone, parseBucketNameAndPrefix, processMultiplePaths } from './util';

const NO_FILES_WARNING =
`There are no files to upload! Make sure the workflow uses the "checkout"` +
Expand All @@ -60,7 +55,7 @@ export async function run(): Promise<void> {
const universe = core.getInput('universe') || 'googleapis.com';

// GCS inputs
const root = core.getInput('path', { required: true });
const pathInput = core.getInput('path', { required: true });
const destination = core.getInput('destination', { required: true });
const gzip = parseBoolean(core.getInput('gzip'));
const resumable = parseBoolean(core.getInput('resumable'));
Expand All @@ -75,13 +70,13 @@ export async function run(): Promise<void> {
const processGcloudIgnore = parseBoolean(core.getInput('process_gcloudignore'));
const metadata = headersInput === '' ? {} : parseHeadersInput(headersInput);

// Compute the absolute root and compute the glob.
const [absoluteRoot, computedGlob, rootIsDir] = await absoluteRootAndComputedGlob(root, glob);
core.debug(`Computed absoluteRoot from "${root}" to "${absoluteRoot}" (isDir: ${rootIsDir})`);
core.debug(`Computed computedGlob from "${glob}" to "${computedGlob}"`);
// Process path input (supports multiple paths separated by newlines)
const { files, absoluteRoot, givenRoot, rootIsDir } = await processMultiplePaths(
pathInput,
glob,
);

// Build complete file list.
const files = await expandGlob(absoluteRoot, computedGlob);
core.debug(`Computed absoluteRoot to "${absoluteRoot}" (isDir: ${rootIsDir})`);
core.debug(`Found ${files.length} files: ${JSON.stringify(files)}`);

// Process ignores:
Expand Down Expand Up @@ -114,7 +109,7 @@ export async function run(): Promise<void> {
}

for (let i = 0; i < files.length; i++) {
const name = path.join(root, files[i]);
const name = path.join(givenRoot, files[i]);
try {
if (ignores.ignores(name)) {
core.debug(`Ignoring ${name} because of ignore file`);
Expand Down Expand Up @@ -151,7 +146,7 @@ export async function run(): Promise<void> {
// Compute the list of file destinations in the bucket based on given
// parameters.
const destinations = Client.computeDestinations({
givenRoot: root,
givenRoot: givenRoot,
absoluteRoot: absoluteRoot,
files: files,
prefix: prefix,
Expand Down
70 changes: 70 additions & 0 deletions src/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,76 @@ export async function expandGlob(directoryPath: string, glob: string): Promise<s
return filesList.sort();
}

/**
* processMultiplePaths handles multiple paths specified in the path input by
* splitting on newlines and processing each path individually.
*
* @param pathInput The path input string (may contain multiple paths separated by newlines)
* @param glob The glob pattern to apply to each path
* @return Object containing files array, absolute root, given root, and whether any path is a directory
*/
export async function processMultiplePaths(
pathInput: string,
glob: string,
): Promise<{
files: string[];
absoluteRoot: string;
givenRoot: string;
rootIsDir: boolean;
}> {
// Split path input by newlines and filter out empty lines
const paths = pathInput
.split('\n')
.map((p: string) => p.trim())
.filter((p: string) => p.length > 0);

if (paths.length === 1) {
// Single path - use existing logic
const [absoluteRoot, computedGlob, rootIsDir] = await absoluteRootAndComputedGlob(
paths[0],
glob,
);
const files = await expandGlob(absoluteRoot, computedGlob);

return {
files,
absoluteRoot,
givenRoot: paths[0],
rootIsDir,
};
}

// Multiple paths - collect files from all paths
const githubWorkspace = process.env.GITHUB_WORKSPACE;
if (!githubWorkspace) {
throw new Error(`$GITHUB_WORKSPACE is not set`);
}

const allFiles: string[] = [];

for (const singlePath of paths) {
const [pathAbsoluteRoot, computedGlob] = await absoluteRootAndComputedGlob(singlePath, glob);
const pathFiles = await expandGlob(pathAbsoluteRoot, computedGlob);

// For multiple paths, we add the relative path from workspace to the file
for (const file of pathFiles) {
const fullFilePath = path.join(pathAbsoluteRoot, file);
const relativeToWorkspace = path.posix.relative(githubWorkspace, fullFilePath);
allFiles.push(relativeToWorkspace);
}
}

// Remove duplicates while preserving order
const uniqueFiles = [...new Set(allFiles)];

return {
files: uniqueFiles,
absoluteRoot: githubWorkspace,
givenRoot: githubWorkspace,
rootIsDir: true,
};
}

/**
* deepClone makes a deep clone of the given object.
*
Expand Down
171 changes: 171 additions & 0 deletions tests/multiple-paths.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
/*
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { test } from 'node:test';
import assert from 'node:assert';

import * as path from 'path';
import * as os from 'os';
import { promises as fs } from 'fs';

import * as core from '@actions/core';
import { clearEnv, forceRemove, setInputs } from '@google-github-actions/actions-utils';
import { Bucket } from '@google-cloud/storage';
import { GoogleAuth } from 'google-auth-library';

import { mockUpload } from './helpers.test';
import { run } from '../src/main';

/**
* Test multiple paths functionality
*/
test('#run multiple paths', { concurrency: true }, async (suite) => {
let githubWorkspace: string;

suite.before(() => {
suite.mock.method(core, 'debug', () => {});
suite.mock.method(core, 'info', () => {});
suite.mock.method(core, 'warning', () => {});
suite.mock.method(core, 'setOutput', () => {});
suite.mock.method(core, 'setSecret', () => {});
suite.mock.method(core, 'group', () => {});
suite.mock.method(core, 'startGroup', () => {});
suite.mock.method(core, 'endGroup', () => {});
suite.mock.method(core, 'addPath', () => {});
suite.mock.method(core, 'exportVariable', () => {});

// We do not care about authentication in the unit tests
suite.mock.method(GoogleAuth.prototype, 'getClient', () => {});
});

suite.beforeEach(async () => {
// Create a temporary directory to serve as the actions workspace
githubWorkspace = await fs.mkdtemp(path.join(os.tmpdir(), 'gha-'));

// Create test files directly in the workspace
await fs.writeFile(path.join(githubWorkspace, 'file1.json'), '{"test": 1}');
await fs.writeFile(path.join(githubWorkspace, 'file2.json'), '{"test": 2}');
await fs.writeFile(path.join(githubWorkspace, 'other.txt'), 'not a json file');

process.env.GITHUB_WORKSPACE = githubWorkspace;
});

suite.afterEach(async () => {
await forceRemove(githubWorkspace);

clearEnv((key) => {
return key.startsWith(`INPUT_`) || key.startsWith(`GITHUB_`);
});
});

await suite.test('uploads multiple specific files', async (t) => {
const uploadMock = t.mock.method(Bucket.prototype, 'upload', mockUpload);

setInputs({
path: `file1.json
file2.json`,
destination: 'my-bucket',
parent: 'false',
process_gcloudignore: 'false',
});

await run();

// Check that both files were uploaded
const uploadedFiles = uploadMock.mock.calls
.map((call) => call?.arguments?.at(0) as string)
.sort();
assert.strictEqual(uploadedFiles.length, 2);
assert.deepStrictEqual(uploadedFiles, [
path.join(githubWorkspace, 'file1.json'),
path.join(githubWorkspace, 'file2.json'),
]);

// Check call sites - should be called twice
assert.strictEqual(uploadMock.mock.calls.length, 2);
});

await suite.test('uploads multiple specific files with pipe syntax', async (t) => {
const uploadMock = t.mock.method(Bucket.prototype, 'upload', mockUpload);

setInputs({
path: `file1.json
file2.json`,
destination: 'my-bucket',
parent: 'false',
process_gcloudignore: 'false',
});

await run();

// Check that both files were uploaded and not the .txt file
const uploadedFiles = uploadMock.mock.calls
.map((call) => call?.arguments?.at(0) as string)
.sort();
assert.strictEqual(uploadedFiles.length, 2);

// Verify that only JSON files were uploaded, not the .txt file
assert.deepStrictEqual(uploadedFiles, [
path.join(githubWorkspace, 'file1.json'),
path.join(githubWorkspace, 'file2.json'),
]);
});

await suite.test('handles single path normally', async (t) => {
const uploadMock = t.mock.method(Bucket.prototype, 'upload', mockUpload);

setInputs({
path: 'file1.json',
destination: 'my-bucket',
parent: 'false',
process_gcloudignore: 'false',
});

await run();

// Check that only one file was uploaded
const uploadedFiles = uploadMock.mock.calls.map((call) => call?.arguments?.at(0) as string);
assert.strictEqual(uploadedFiles.length, 1);
assert.deepStrictEqual(uploadedFiles, [path.join(githubWorkspace, 'file1.json')]);
});

await suite.test('ignores empty lines in multiple paths', async (t) => {
const uploadMock = t.mock.method(Bucket.prototype, 'upload', mockUpload);

setInputs({
path: `file1.json

file2.json

`,
destination: 'my-bucket',
parent: 'false',
process_gcloudignore: 'false',
});

await run();

// Check that both files were uploaded despite empty lines
const uploadedFiles = uploadMock.mock.calls
.map((call) => call?.arguments?.at(0) as string)
.sort();
assert.strictEqual(uploadedFiles.length, 2);
assert.deepStrictEqual(uploadedFiles, [
path.join(githubWorkspace, 'file1.json'),
path.join(githubWorkspace, 'file2.json'),
]);
});
});