Skip to content

feat: create re-usable github action #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
+18,850 −1
Open

feat: create re-usable github action #60

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f51eec0
feat: create re-usable github action
SgtPooki Sep 30, 2025
9505425
chore: fix lint
SgtPooki Sep 30, 2025
caf1514
fix: use local source
SgtPooki Sep 30, 2025
eba600f
chore: cp code from sgtpooki/filecoin-upload-action
SgtPooki Oct 3, 2025
f513fc9
docs: add docs and examples
SgtPooki Oct 3, 2025
f5feac2
fix: lint failures on extension names (#70)
SgtPooki Oct 6, 2025
46f8eaf
fix: usage by filecoin-project action test
SgtPooki Oct 6, 2025
5ff99e9
refactor: rename action
SgtPooki Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions .github/actions/filecoin-pin-upload-action/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Filecoin Pin Upload Action (Local Copy)

This is a local copy of the composite action that packs a file/directory into a UnixFS CAR, uploads via `filecoin-pin` to Filecoin (Synapse), and publishes useful artifacts.

Use it from this repo via:

```yaml
uses: ./.github/actions/filecoin-pin-upload-action
with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you also need to specify the triggering conditions?

privateKey: ${{ secrets.FILECOIN_WALLET_KEY }}
path: dist
minDays: 10
minBalance: "5" # USDFC
maxTopUp: "50" # USDFC
providerAddress: "0xa3971A7234a3379A1813d9867B531e7EeB20ae07"
```

Notes:
- This local copy depends on the published `filecoin-pin` npm package (imports `filecoin-pin/dist/...`).
- For PR events, the action posts a comment with the IPFS Root CID.

Inputs
- `privateKey` (required): Wallet private key.
- `path` (default: `dist`): Build output path.
- `minDays` (default: `10`): Minimum runway in days.
- `minBalance` (optional): Minimum deposit (USDFC).
- `maxTopUp` (optional): Maximum additional deposit (USDFC).
- `token` (default: `USDFC`): Supported token.
- `withCDN` (default: `false`): Request CDN if available.
- `providerAddress` (default shown above): Override storage provider address (Calibration/Mainnet). Leave empty to allow auto-selection.

Security notes for PR workflows
- If you use `pull_request`, the workflow and action come from the PR branch. PR authors can modify inputs (e.g., `minDays`, `minBalance`). Set a conservative `maxTopUp` to cap spending.
- If you need PRs to always run the workflow definition from `main`, consider `pull_request_target`. WARNING: it runs with base repo permissions and may have access to secrets. Do not run untrusted PR code with those secrets. Prefer a two-workflow model (`pull_request` build → `workflow_run` deploy) when in doubt.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to use pull_request_target to prevent a workflow definition from being manipulated.

I guess then the issue is if someone usses this action with other stuff in their workflow, where the workflow uses client-supplied code. This is where can get into trouble because client could provide code that allows reading of secrets.

Maybe have some note like:

Never use this action in a workflow with untrusted code directly (like npm install or building PR code) as they will have access to full access to the repsository's secrets.


Security considerations (PRs)
- Running uploads on pull_request means PR authors can change inputs (e.g., `minDays`, `minBalance`) within the PR, which can influence deposits/top-ups.
- Always set a conservative `maxTopUp` to cap the maximum funds added in a single run.
- Protect your main branch and review workflow changes. Require approval for workflows from outside collaborators.
- Forked PRs don’t receive secrets by default, so funding won’t run there; same-repo PRs do have access to secrets.

Caching details
- Cache key: `filecoin-pin-v1-${root_cid}` ensures uploads are skipped for identical content.
- You can invalidate all caches by changing the version prefix (e.g., `v2`).
- Retention is managed by GitHub Actions and organization settings; it’s not configurable per cache entry in actions/cache v4. Each restore updates last-access time.

## Setup Checklist (Security + Reliability)

- Pin the action when used from another repo: `uses: filecoin-project/filecoin-pin/.github/actions/filecoin-pin-upload-action@<commit-sha>`
- Restrict allowed actions in repo/org settings (Actions → General → Allow select actions) to:
- GitHub official (e.g., `actions/*`)
- Your org (e.g., `filecoin-project/*`)
- Grant the workflow/job `actions: read` if you want artifact reuse to work across runs.
- Cap spend with `maxTopUp` (pushes) and a lower cap (or zero) on PRs.
- Consider Environments with required reviewers for any deposit/top-up steps.
- Keep workflow files protected with CODEOWNERS + branch protection.
- Never run untrusted PR code with secrets under `pull_request_target`. Prefer a two‑step model if you need main‑defined workflows.

## PR Safety Options

- Low/zero PR top‑ups (simple)
- In your workflow, set a small cap for PRs. Uploads still work if already funded.
- Example:
```yaml
with:
maxTopUp: ${{ github.event_name == 'pull_request' && '0' || '50' }}
```

- Label‑gated PR spending (reviewer control)
- Default PR cap is 0; maintainers add `allow-upload` label to raise the cap.
- Example:
```yaml
- name: Decide PR cap
id: caps
if: ${{ github.event_name == 'pull_request' }}
uses: actions/github-script@v7
with:
script: |
const labels = (context.payload.pull_request.labels||[]).map(l=>l.name)
core.setOutput('PR_CAP', labels.includes('allow-upload') ? '5' : '0')

- name: Upload
uses: ./.github/actions/filecoin-pin-upload-action
with:
maxTopUp: "${{ steps.caps.outputs.PR_CAP || '50' }}"
```

- Two‑step (safest) with artifacts
- PR workflow (no secrets): `with: mode: prepare` → uploads CAR + metadata as artifact
- workflow_run on main: download artifact and `with: mode: upload` → validates and uploads with secrets

## Two‑Step Usage

Prepare (PR, no secrets):
```yaml
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20.x }
- run: npm ci && npm run build
- name: Prepare CAR (no secrets)
uses: ./.github/actions/filecoin-pin-upload-action
with:
mode: prepare
path: dist
artifactName: filecoin-pin-${{ github.run_id }}-${{ github.sha }}
```

Upload (workflow_run on main, with secrets):
```yaml
steps:
- uses: actions/checkout@v4
- name: Download artifact
uses: actions/download-artifact@v4
with:
name: filecoin-pin-${{ github.event.workflow_run.run_id }}-${{ github.event.workflow_run.head_sha }}
path: filecoin-pin-artifacts
- name: Upload to Filecoin
uses: ./.github/actions/filecoin-pin-upload-action
with:
mode: upload
prebuiltCarPath: filecoin-pin-artifacts/upload.car
privateKey: ${{ secrets.FILECOIN_WALLET_KEY }}
minDays: 10
minBalance: "5"
maxTopUp: "50"
providerAddress: "0xa3971A7234a3379A1813d9867B531e7EeB20ae07"
```
251 changes: 251 additions & 0 deletions .github/actions/filecoin-pin-upload-action/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
name: "Filecoin Pin Upload"
description: "Pack site into a CAR, upload via filecoin-pin to Filecoin, and publish build artifacts."
branding:
icon: upload-cloud
color: blue

inputs:
github_token:
description: GitHub token for commenting and artifacts. Defaults to workflow token.
required: false
privateKey:
description: Required. Wallet private key used to fund uploads (USDFC on Calibration/Mainnet).
required: true
path:
description: Path to content to upload (file or directory). Typically your build output directory.
required: false
default: dist
minDays:
description: "Minimum runway (days) to keep current spend alive. Security note: on PRs, authors can change this."
required: false
default: "10"
minBalance:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? Remove?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, I think minBalance could be removed.. with minDays and maxTopUp, I think that's the ideal UX consumers would think about.. i.e. "How long do i want to persist this data, and whats the max i'm willing to top up each time I need to deposit more funds"

we may want to add maxBalance as a runaway topUp from PR spamming stop-gap.

description: "Minimum deposit balance to maintain in Filecoin Pay (USDFC). Security note: on PRs, authors can change this."
required: false
maxTopUp:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As currently stated, this is still somewhat hackable in that I can trigger lots of workflows to increase your deposits.

As a protection, how about something like:

doNotDepositIfFilecoinWarmStorageHasBalanceGreaterThan

I know this is verbose. We can come up with a better name.

The key idea here is that we don't consider any depositing if the balance is alraedy above a certain number.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is referencing the maxTopUp. I see your point, but that mental model assumes repo owners want to deposit a bunch into warmstorage, and then need to manage that separately.

I feel like the approach where a repo owner can keep their warmstorage deposit low, and limit deposit sizes to only whats necessary for new runs, keeps maintainence burden low while also protecting users. For my projects, this is how I would want to think about it.

If we do doNotDepositIfFilecoinWarmStorageHasBalanceGreaterThan (or maxBalance):

  1. we might not be able to increase balance a little even though we actually want to
    • User must go top up manually, update gh action balance amount, then kick off workflow again
  2. we do not guard against PRs that increase storage runway requirements a lot
    • if balance is under maxBalance, we still need to implicitly do "maxTopUp" by calculating deficit and comparing against maxBalance

essentially, my mental model when approaching this github action was:

  1. As a repo owner, I want to be able to persist my content on filecoin with minimal maintenance
  2. As a repo owner, I want to see PR previews, but only want to support X deposit amount for each preview
  3. As a repo owner, I don't want to allow my wallet to be impacted in unexpected ways, and want easy guardrails for limiting fund impacts from PRs and main branch uploads.

for 2 above, there's still work needed to accomplish this, and also questionable UX/DX around whether previews need to be persisted on filecoin at all? (maybe pin to IPFS provider first, and then only upload to filecoin on merge?) Also, we can't currently delete data-sets/pieces AFAIK.

Your comment seems to be in the realm of item 3 above. With that said, I do think a maxBalance would be appropriate (prevents a flood of PRs increasing by maxTopUp quickly growing the balance unexpectedly).

I need to align my mental model for how we expect the action to work, so I can do some proper threat modeling.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chatting with @BigLep:

PR previews could specify a particular data-set to use, so wallet owners can control PR persistance.
action could take in metadata_extension json field to pass in any metadata (remove withCDN)
action could take in explicit data_set_filter metadata tag so they can target different datasets for PRs, prod, staging, etc..

maxBalance will limit total loss .. we should add a disclaimer that we can't really map maxBalance to datasets.

users can manage separate wallets or funding and should do so if they are concerned beyond the maxBalance

for PRs from forks, lets disable that for now to prevent non-maintainers from generating Previews and draining funds from repo owners.

description: Maximum allowed additional deposit during this run (USDFC). Strongly recommended to cap PR spend.
required: false
token:
description: Payment token identifier. Currently only "USDFC" is supported; address override reserved for future.
required: false
default: "USDFC"
withCDN:
description: If true, request CDN in the storage context (depends on provider capabilities).
required: false
default: "false"
providerAddress:
description: Optional override for storage provider address (on Calibration/Mainnet). Defaults to a known good provider on Calibration.
required: false
default: "0xa3971A7234a3379A1813d9867B531e7EeB20ae07"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about default to using Synapse's default rather than encoding something here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is primarily for testing: ezpzdz was claimed by someone in slack to be "the most reliable" and I've also noticed the same since I started using it... This should be removed from any prod version

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Maybe then to make it clear, we just prefix it with some lime "internal" or "temp"?


outputs:
root_cid:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ipfs_root_cid to be consistent with our naming other places?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see SgtPooki/filecoin-upload-action@77386ae

description: IPFS Root CID
value: ${{ steps.run.outputs.root_cid }}
data_set_id:
description: Synapse Data Set ID
value: ${{ steps.run.outputs.data_set_id }}
piece_cid:
description: Filecoin Piece CID
value: ${{ steps.run.outputs.piece_cid }}
provider_id:
description: Storage Provider ID
value: ${{ steps.run.outputs.provider_id }}
provider_name:
description: Storage Provider Name
value: ${{ steps.run.outputs.provider_name }}
car_path:
description: Path to the created CAR file
value: ${{ steps.run.outputs.car_path }}
metadata_path:
description: Path to JSON with upload metadata
value: ${{ steps.run.outputs.metadata_path }}

runs:
using: "composite"
steps:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'

- name: Install action deps
shell: bash
working-directory: .github/actions/filecoin-pin-upload-action
run: |
npm install --no-audit --no-fund

- name: Compute root + CAR
id: compute
shell: bash
working-directory: .github/actions/filecoin-pin-upload-action
env:
INPUT_GITHUB_TOKEN: ${{ inputs.github_token }}
INPUT_PRIVATEKEY: ${{ inputs.privateKey }}
INPUT_PATH: ${{ inputs.path }}
INPUT_MINDAYS: ${{ inputs.minDays }}
INPUT_MINBALANCE: ${{ inputs.minBalance }}
INPUT_MAXTOPUP: ${{ inputs.maxTopUp }}
INPUT_WITHCDN: ${{ inputs.withCDN }}
INPUT_TOKEN: ${{ inputs.token }}
INPUT_PROVIDERADDRESS: ${{ inputs.providerAddress }}
ACTION_PHASE: compute
run: |
node run.mjs

- name: Restore upload cache
id: cache-restore
uses: actions/cache/restore@v4
with:
key: filecoin-pin-v1-${{ steps.compute.outputs.root_cid }}
path: .filecoin-pin-cache/${{ steps.compute.outputs.root_cid }}

- name: Use cached metadata
if: ${{ steps.cache-restore.outputs.cache-hit == 'true' }}
id: from-cache
shell: bash
working-directory: .github/actions/filecoin-pin-upload-action
env:
CACHE_DIR: ${{ github.workspace }}/.filecoin-pin-cache/${{ steps.compute.outputs.root_cid }}
INPUT_GITHUB_TOKEN: ${{ inputs.github_token }}
INPUT_PRIVATEKEY: ${{ inputs.privateKey }}
INPUT_MINDAYS: ${{ inputs.minDays }}
INPUT_MINBALANCE: ${{ inputs.minBalance }}
INPUT_MAXTOPUP: ${{ inputs.maxTopUp }}
INPUT_WITHCDN: ${{ inputs.withCDN }}
INPUT_TOKEN: ${{ inputs.token }}
INPUT_PROVIDERADDRESS: ${{ inputs.providerAddress }}
PREPARED_CAR_PATH: ${{ steps.compute.outputs.car_path }}
PREPARED_ROOT_CID: ${{ steps.compute.outputs.root_cid }}
ACTION_PHASE: from-cache
run: |
node run.mjs

- name: Find previous artifact by Root CID
if: ${{ steps.cache-restore.outputs.cache-hit != 'true' }}
id: find-artifact
uses: actions/github-script@v7
env:
ROOT_CID: ${{ steps.compute.outputs.root_cid }}
with:
github-token: ${{ inputs.github_token || github.token }}
script: |
const { owner, repo } = context.repo
const ROOT_CID = process.env.ROOT_CID
const targetName = `filecoin-pin-${ROOT_CID}`
const items = await github.paginate(github.rest.actions.listArtifactsForRepo, { owner, repo, per_page: 100 })
const found = items.find(a => a.name === targetName && !a.expired)
if (found) {
core.setOutput('artifact_id', String(found.id))
core.setOutput('run_id', String(found.workflow_run?.id || ''))
} else {
core.setOutput('artifact_id', '')
core.setOutput('run_id', '')
}

- name: Download previous artifact
if: ${{ steps.cache-restore.outputs.cache-hit != 'true' && steps.find-artifact.outputs.artifact_id != '' }}
uses: actions/download-artifact@v4
with:
name: filecoin-pin-${{ steps.compute.outputs.root_cid }}
run-id: ${{ steps.find-artifact.outputs.run_id }}
path: filecoin-pin-artifacts-restore

- name: Use artifact metadata
if: ${{ steps.cache-restore.outputs.cache-hit != 'true' && steps.find-artifact.outputs.artifact_id != '' }}
id: from-artifact
shell: bash
working-directory: .github/actions/filecoin-pin-upload-action
env:
CACHE_DIR: ${{ github.workspace }}/filecoin-pin-artifacts-restore
INPUT_GITHUB_TOKEN: ${{ inputs.github_token }}
INPUT_PRIVATEKEY: ${{ inputs.privateKey }}
INPUT_MINDAYS: ${{ inputs.minDays }}
INPUT_MINBALANCE: ${{ inputs.minBalance }}
INPUT_MAXTOPUP: ${{ inputs.maxTopUp }}
INPUT_WITHCDN: ${{ inputs.withCDN }}
INPUT_TOKEN: ${{ inputs.token }}
INPUT_PROVIDERADDRESS: ${{ inputs.providerAddress }}
FROM_ARTIFACT: "true"
ACTION_PHASE: from-cache
run: |
node run.mjs

- name: Upload via filecoin-pin
if: ${{ steps.cache-restore.outputs.cache-hit != 'true' && steps.find-artifact.outputs.artifact_id == '' }}
id: run
shell: bash
working-directory: .github/actions/filecoin-pin-upload-action
env:
INPUT_GITHUB_TOKEN: ${{ inputs.github_token }}
INPUT_PRIVATEKEY: ${{ inputs.privateKey }}
INPUT_PATH: ${{ inputs.path }}
INPUT_MINDAYS: ${{ inputs.minDays }}
INPUT_MINBALANCE: ${{ inputs.minBalance }}
INPUT_MAXTOPUP: ${{ inputs.maxTopUp }}
INPUT_WITHCDN: ${{ inputs.withCDN }}
INPUT_TOKEN: ${{ inputs.token }}
INPUT_PROVIDERADDRESS: ${{ inputs.providerAddress }}
ACTION_PHASE: upload
PREPARED_CAR_PATH: ${{ steps.compute.outputs.car_path }}
PREPARED_ROOT_CID: ${{ steps.compute.outputs.root_cid }}
run: |
node run.mjs

- name: Save upload cache
if: ${{ steps.cache-restore.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v4
with:
key: filecoin-pin-v1-${{ steps.compute.outputs.root_cid }}
path: .filecoin-pin-cache/${{ steps.compute.outputs.root_cid }}

- name: Upload CAR + metadata artifacts
if: ${{ (steps.run.outputs.car_path || steps.from-cache.outputs.car_path || steps.from-artifact.outputs.car_path) != '' }}
uses: actions/upload-artifact@v4
with:
name: filecoin-pin-${{ steps.run.outputs.root_cid || steps.from-cache.outputs.root_cid || steps.from-artifact.outputs.root_cid || steps.compute.outputs.root_cid }}
path: |
${{ steps.run.outputs.car_path || steps.from-cache.outputs.car_path || steps.from-artifact.outputs.car_path }}
${{ steps.run.outputs.metadata_path || steps.from-cache.outputs.metadata_path || steps.from-artifact.outputs.metadata_path }}

- name: Comment on PR with IPFS Root CID
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this in a script file rather than inlining the script?

if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_target' }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this safe to run on pull_request_target? I assume we don't want this to run on forks.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command in particular is safe, because it doesn't have access to secrets, but the action does need to be split up into a two workflow process. I am working on that now

uses: actions/github-script@v7
env:
IPFS_ROOT_CID: ${{ steps.run.outputs.root_cid || steps.from-cache.outputs.root_cid || steps.from-artifact.outputs.root_cid }}
DATA_SET_ID: ${{ steps.run.outputs.data_set_id || steps.from-cache.outputs.data_set_id || steps.from-artifact.outputs.data_set_id }}
PIECE_CID: ${{ steps.run.outputs.piece_cid || steps.from-cache.outputs.piece_cid || steps.from-artifact.outputs.piece_cid }}
UPLOAD_STATUS: ${{ steps.run.outputs.upload_status || steps.from-cache.outputs.upload_status || steps.from-artifact.outputs.upload_status }}
with:
github-token: ${{ inputs.github_token || github.token }}
script: |
const { IPFS_ROOT_CID, DATA_SET_ID, PIECE_CID, UPLOAD_STATUS } = process.env
const preview = 'https://ipfs.io/ipfs/' + IPFS_ROOT_CID
let statusLine = '- Status: '
if (UPLOAD_STATUS === 'uploaded') statusLine += 'Uploaded new content'
else if (UPLOAD_STATUS === 'reused-cache') statusLine += 'Reused cached content'
else if (UPLOAD_STATUS === 'reused-artifact') statusLine += 'Reused artifact content'
else statusLine += 'Unknown (see job logs)'
const body = [
'<!-- filecoin-pin-upload-action -->',
'Filecoin Pin Upload ✅',
'',
'- IPFS Root CID: `' + IPFS_ROOT_CID + '`',
'- Data Set ID: `' + DATA_SET_ID + '`',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be nice to link to the PDP Explorer. I believe the URL is https://pdp.vxb.ai/ but it's not working for me currrently.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea its at https://pdp.vxb.ai/calibration. I can add it.

'- Piece CID: `' + PIECE_CID + '`',
'',
statusLine,
'',
'- Preview (temporary centralized gateway):',
' - ' + preview,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to use dweb.link?

].join('\n')

const { owner, repo } = context.repo
const issue_number = context.issue.number
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number })
const existing = comments.find(c => c.user?.type === 'Bot' && (c.body || '').includes('filecoin-pin-upload-action'))
if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body })
} else {
await github.rest.issues.createComment({ owner, repo, issue_number, body })
}
Loading