-
Notifications
You must be signed in to change notification settings - Fork 0
feat: create re-usable github action #60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SgtPooki
wants to merge
8
commits into
master
Choose a base branch
from
49-feat-github-action-for-filecoin-pin-uploads
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+18,850
−1
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
f51eec0
feat: create re-usable github action
SgtPooki 9505425
chore: fix lint
SgtPooki caf1514
fix: use local source
SgtPooki eba600f
chore: cp code from sgtpooki/filecoin-upload-action
SgtPooki f513fc9
docs: add docs and examples
SgtPooki f5feac2
fix: lint failures on extension names (#70)
SgtPooki 46f8eaf
fix: usage by filecoin-project action test
SgtPooki 5ff99e9
refactor: rename action
SgtPooki File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# Internal Flow | ||
|
||
This document explains how the action works internally and why each step exists. | ||
|
||
## High-Level Execution | ||
|
||
1. **run.mjs** is the entry point. It: | ||
- Persists basic GitHub run metadata in the shared context. | ||
- Calls `runBuild()` to create the CAR file. | ||
- Calls `runUpload()` to upload to Filecoin. | ||
- Ensures `cleanupSynapse()` runs on success or failure. | ||
|
||
2. **Build phase (`src/build.js`)** | ||
- Parses inputs via `parseInputs('compute')`. This validates `path` and `network` but does not require the wallet key. | ||
- Detects fork PRs (by comparing head/base repo names). When detected, it records `uploadStatus=fork-pr-blocked` in the context and emits a notice that upload will be blocked. | ||
- Resolves `path` against the workspace and generates a CAR using `createCarFile()`. | ||
- Stores the CAR file path, size, and IPFS root CID in the in-memory context (see `src/context.js`). | ||
- Merges additional metadata (run id, PR details) through `mergeAndSaveContext()`. | ||
|
||
3. **Upload phase (`src/upload.js`)** | ||
- Parses inputs via `parseInputs('upload')`. This enforces presence of `walletPrivateKey` and confirms `network`, `minStorageDays`, and `filecoinPayBalanceLimit` rules. | ||
- If the build context marked the run as `fork-pr-blocked`, the upload phase writes outputs, posts the explanatory PR comment, and exits without touching Filecoin. | ||
- Validates that the CAR file still exists on disk. | ||
- Calls `initializeSynapse({ walletPrivateKey, network })`, which selects the correct RPC endpoint (`RPC_URLS[network].websocket`) and bootstraps filecoin-pin. | ||
- Fetches current payment status, then hands control to `handlePayments()` for deposit logic. | ||
- Uploads the CAR to Filecoin via `uploadCarToFilecoin()`; this returns piece CID, dataset id, provider info, preview URL, and canonical network name from filecoin-pin. | ||
- Updates the context, writes GitHub Action outputs, appends a step summary, and posts/updates the PR comment via `commentOnPR()`. | ||
|
||
## Input Parsing (`src/inputs.js`) | ||
|
||
`parseInputs()` uses a single schema for both phases: | ||
- `path`: required for both phases. | ||
- `walletPrivateKey`: required when `phase !== 'compute'`. | ||
- `network`: required; must be `mainnet` or `calibration`. | ||
- `minStorageDays`: optional number (defaults to `0` when unset). | ||
- `filecoinPayBalanceLimit`: bigint parsed from USDFC string; required when `minStorageDays > 0`. | ||
- `providerAddress`, `withCDN`: optional advanced settings with defaults. | ||
|
||
The helper supports both environment-variable fallback (`INPUT_<NAME>`) and the `INPUTS_JSON` bundle populated by `action.yml`. | ||
|
||
## Payment Handling (`src/filecoin.js` – `handlePayments`) | ||
|
||
- Ensures Synapse allowances are configured via `checkAndSetAllowances()`. | ||
- Pulls current balance with `getPaymentStatus()`. | ||
- If `minStorageDays > 0`, computes the top-up required using `computeTopUpForDuration()`. | ||
- Enforces the hard ceiling defined by `filecoinPayBalanceLimit`. If the current balance already meets or exceeds the limit, no deposit happens. If the computed top-up would exceed the limit, it is reduced to the largest permissible amount. | ||
- Executes a deposit through `depositUSDFC()` when the final top-up is positive and refreshes payment status for downstream reporting. | ||
|
||
## Context & Outputs | ||
|
||
- Context lives in-memory inside `src/context.js`. Build and upload occur in the same job, so filesystem artifacts are not required for hand-off. | ||
- `writeOutputs()` exposes CID, dataset, provider, CAR path, and status. Fork-blocked runs still surface the CAR information to aid reviewers. | ||
- `writeSummary()` appends a markdown summary detailing payment status, provider links (via `pdp.vxb.ai/<network>`), and CAR size. | ||
- `commentOnPR()` reuses existing bot comments when possible and uses the default workflow token. | ||
|
||
## Error Handling | ||
|
||
- Domain-specific failures throw `FilecoinPinError` with codes for insufficient funds, invalid private keys, and balance-limit violations. | ||
- `handleError()` surfaces guidance tailored to the inputs (e.g., advising updates to `filecoinPayBalanceLimit`). | ||
- `run.mjs` guarantees Synapse cleanup even when build or upload throws. | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
# Filecoin Pin Upload Action | ||
|
||
Composite GitHub Action that packs a file or directory into a UnixFS CAR, uploads it to Filecoin, and publishes artifacts and context for easy reuse. | ||
|
||
## Quick Start | ||
|
||
Run your build in an untrusted workflow, publish the build output as an artifact, then run this action in a trusted workflow to create the CAR and upload to Filecoin. Fork PR support is currently disabled, so workflows must run within the same repository. | ||
|
||
**Step 1: Build workflow** (no secrets): | ||
```yaml | ||
# .github/workflows/build-pr.yml | ||
name: Build PR Content | ||
on: pull_request | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- run: npm ci && npm run build | ||
- name: Upload build artifacts | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: site-dist | ||
path: dist | ||
``` | ||
|
||
**Step 2: Upload workflow** (runs after build, uses secrets): | ||
```yaml | ||
# .github/workflows/upload-to-filecoin.yml | ||
name: Upload to Filecoin | ||
on: | ||
workflow_run: | ||
workflows: ["Build PR Content"] | ||
types: [completed] | ||
|
||
jobs: | ||
upload: | ||
if: github.event.workflow_run.conclusion == 'success' | ||
runs-on: ubuntu-latest | ||
permissions: | ||
actions: read | ||
pull-requests: write | ||
steps: | ||
- name: Download build artifacts | ||
uses: actions/download-artifact@v4 | ||
with: | ||
name: site-dist | ||
path: dist | ||
github-token: ${{ github.token }} | ||
repository: ${{ github.event.workflow_run.repository.full_name }} | ||
run-id: ${{ github.event.workflow_run.id }} | ||
|
||
- name: Upload to Filecoin | ||
uses: sgtpooki/filecoin-upload-action@v1 | ||
with: | ||
path: dist | ||
walletPrivateKey: ${{ secrets.FILECOIN_WALLET_KEY }} | ||
network: calibration | ||
minStorageDays: "30" | ||
filecoinPayBalanceLimit: "0.25" | ||
``` | ||
|
||
**Versioning**: This action uses [Semantic Release](https://semantic-release.gitbook.io/) for automated versioning. Use version tags like `@v1`, `@v1.0.0`, or commit SHAs for supply-chain safety. | ||
|
||
## Inputs | ||
|
||
See [action.yml](./action.yml) the input parameters and their descriptions. | ||
|
||
## Security & Permissions Checklist | ||
|
||
- ✅ Pin the action by version tag or commit SHA | ||
- ✅ Grant `actions: read` if you want artifact reuse (cache fallback) to work | ||
- ✅ Protect workflow files with CODEOWNERS/branch protection | ||
- ✅ **Always** hardcode `minStorageDays` and `filecoinPayBalanceLimit` in trusted workflows | ||
- ✅ **Never** use `pull_request_target` - use the two-workflow pattern instead | ||
- ✅ Enable **branch protection** on main to require reviews for workflow changes | ||
- ✅ Use **CODEOWNERS** to require security team approval for workflow modifications | ||
- ⚠️ Consider gating deposits with Environments that require approval | ||
|
||
## Usage | ||
|
||
The action uses a secure two-workflow pattern by default. This currently works for **same-repo PRs only** (fork PR support temporarily disabled). | ||
|
||
Split your CI into untrusted build + trusted upload workflows. | ||
|
||
**Security Note**: The `workflow_run` trigger always executes the workflow file from your main branch, not from the PR. Even if a PR modifies the upload workflow to change hardcoded limits, those changes won't apply until the PR is merged. | ||
|
||
## Current Limitations | ||
|
||
**⚠️ Fork PR Support Disabled** | ||
|
||
- Only same-repo PRs and direct pushes to main are supported | ||
- PR commenting works, but shows different message for fork PRs | ||
- This limits non-maintainer PR actors from draining funds from unaware repo owners | ||
|
||
**See [examples/two-workflow-pattern/](./examples/two-workflow-pattern/)** for complete, ready-to-use workflow files. | ||
|
||
## Releases & Versioning | ||
|
||
This action uses [Semantic Release](https://semantic-release.gitbook.io/) for automated versioning based on [Conventional Commits](https://www.conventionalcommits.org/). | ||
|
||
### Available Versions | ||
|
||
- **`@v1`** - Latest v1.x.x release (recommended for most users) | ||
- **`@v1.0.0`** - Specific version (recommended for production) | ||
- **`@<commit-sha>`** - Specific commit (maximum security) | ||
|
||
### Version Bumps | ||
|
||
- **Patch** (`1.0.0` → `1.0.1`): Bug fixes, docs, refactoring | ||
- **Minor** (`1.0.0` → `1.1.0`): New features | ||
- **Major** (`1.0.0` → `2.0.0`): Breaking changes | ||
|
||
### Release Process | ||
|
||
Releases are automatically created when changes are pushed to `main` with conventional commit messages. See [CONTRIBUTING.md](./CONTRIBUTING.md) for commit message guidelines. | ||
|
||
## Documentation | ||
|
||
- **[examples/two-workflow-pattern/](./examples/two-workflow-pattern/)** - Ready-to-use workflow files (recommended) | ||
- **[USAGE.md](./USAGE.md)** - Complete usage guide with all patterns | ||
- **[FLOW.md](./FLOW.md)** - Internal architecture & how the action works under the hood | ||
- **[examples/README.md](./examples/README.md)** - Detailed setup instructions | ||
|
||
## Caching & Artifacts | ||
|
||
- Cache key: `filecoin-pin-v1-${ipfsRootCid}` enables reuse for identical content. | ||
- Artifacts: `filecoin-pin-artifacts/upload.car` and `filecoin-pin-artifacts/context.json` are published for each run. | ||
- PR comments include the IPFS root CID, dataset ID, piece CID, and preview link. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,163 @@ | ||
# Usage Guide | ||
|
||
This action builds a UnixFS CAR from your site or files and uploads it to Filecoin in a single invocation. For security, separate untrusted build steps from the trusted upload step. | ||
|
||
## Recommended Pattern: Build + Upload Workflows | ||
|
||
1. **Build workflow** (no secrets) compiles your project and uploads the build output as an artifact. | ||
2. **Upload workflow** (trusted) downloads the artifact, runs this action, and provides wallet secrets. | ||
|
||
### Workflow 1: Build (Untrusted) | ||
|
||
```yaml | ||
# .github/workflows/build-pr.yml | ||
name: Build PR Content | ||
|
||
on: | ||
pull_request: | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.event.pull_request.head.sha }} | ||
|
||
- name: Build your site | ||
run: npm run build | ||
|
||
- name: Upload build artifacts | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: site-dist | ||
path: dist | ||
``` | ||
|
||
### Workflow 2: Upload (Trusted) | ||
|
||
```yaml | ||
# .github/workflows/upload-to-filecoin.yml | ||
name: Upload to Filecoin | ||
|
||
on: | ||
workflow_run: | ||
workflows: ["Build PR Content"] | ||
types: [completed] | ||
|
||
jobs: | ||
upload: | ||
if: ${{ github.event.workflow_run.conclusion == 'success' }} | ||
runs-on: ubuntu-latest | ||
permissions: | ||
actions: read | ||
pull-requests: write | ||
steps: | ||
- name: Download build artifacts | ||
uses: actions/download-artifact@v4 | ||
with: | ||
name: site-dist | ||
path: dist | ||
github-token: ${{ github.token }} | ||
repository: ${{ github.event.workflow_run.repository.full_name }} | ||
run-id: ${{ github.event.workflow_run.id }} | ||
|
||
- name: Upload to Filecoin | ||
uses: sgtpooki/filecoin-upload-action@v1 | ||
with: | ||
path: dist | ||
walletPrivateKey: ${{ secrets.WALLET_PRIVATE_KEY }} | ||
network: calibration | ||
minStorageDays: "30" | ||
filecoinPayBalanceLimit: "0.25" | ||
``` | ||
|
||
**Security hints**: | ||
- Build workflow never sees wallet secrets. | ||
- Upload workflow runs from the main branch version of the file when triggered via `workflow_run`, so PRs cannot change hardcoded values until merged. | ||
- Hardcode financial parameters in trusted workflows and review changes carefully. | ||
|
||
--- | ||
|
||
## Alternative: Single Workflow (Trusted Repos Only) | ||
|
||
If every contributor is trusted and you do not accept fork PRs, you can run build and upload in the same job: | ||
|
||
```yaml | ||
name: Upload to Filecoin | ||
|
||
on: | ||
pull_request: | ||
push: | ||
branches: [main] | ||
|
||
jobs: | ||
upload: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- run: npm run build | ||
- name: Upload to Filecoin | ||
uses: sgtpooki/filecoin-upload-action@v1 | ||
with: | ||
path: dist | ||
walletPrivateKey: ${{ secrets.WALLET_PRIVATE_KEY }} | ||
network: mainnet | ||
minStorageDays: "7" | ||
filecoinPayBalanceLimit: "1.00" | ||
``` | ||
|
||
Use this approach only when you fully trust everyone who can open PRs. | ||
|
||
--- | ||
|
||
## Input Reference | ||
|
||
### `path` | ||
- **Type**: `string` | ||
- **Required**: Yes | ||
- **Description**: File or directory to package into a CAR and upload. | ||
|
||
### `walletPrivateKey` | ||
- **Type**: `string` | ||
- **Required**: Yes when uploading | ||
- **Description**: EVM-compatible private key for the Filecoin wallet. | ||
|
||
### `network` | ||
- **Type**: `string` | ||
- **Required**: Yes | ||
- **Options**: `mainnet`, `calibration` | ||
- **Description**: Selects the Filecoin network; controls the RPC endpoint used by filecoin-pin. | ||
|
||
### `minStorageDays` | ||
- **Type**: `string` | ||
- **Required**: No | ||
- **Description**: Desired storage runway in days. When provided, the action calculates the deposit needed to reach this runway. | ||
|
||
### `filecoinPayBalanceLimit` | ||
- **Type**: `string` | ||
- **Required**: Yes if `minStorageDays` is provided | ||
- **Description**: Maximum Filecoin Pay balance (USDFC) allowed after deposits. | ||
|
||
### `providerAddress` | ||
- **Type**: `string` | ||
- **Default**: `0xa3971A7234a3379A1813d9867B531e7EeB20ae07` | ||
- **Description**: Optional override for the storage provider. | ||
|
||
### `withCDN` | ||
- **Type**: `boolean` | ||
- **Default**: `false` | ||
- **Description**: Request CDN support when available. Warning: filecoin-pin does not yet adjust deposit calculations for CDN usage. | ||
|
||
--- | ||
|
||
## Outputs | ||
|
||
- `ipfsRootCid`: IPFS Root CID | ||
- `dataSetId`: Synapse Data Set ID | ||
- `pieceCid`: Filecoin Piece CID | ||
- `providerId`: Storage Provider ID | ||
- `providerName`: Storage Provider Name | ||
- `carPath`: Path to the generated CAR file | ||
- `uploadStatus`: Status of the run (e.g., `uploaded`, `fork-pr-blocked`) | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe discuss the failure cases that can occur that will fail the action?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should be covered by FLOW and USAGE now, but we can add into README if we want. there is also src/errors.js that has some common errors we can call out.