Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/actions/upload/FLOW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Internal Flow

This document explains how the action works internally and why each step exists.

## High-Level Execution

1. **run.mjs** is the entry point. It:
- Persists basic GitHub run metadata in the shared context.
- Calls `runBuild()` to create the CAR file.
- Calls `runUpload()` to upload to Filecoin.
- Ensures `cleanupSynapse()` runs on success or failure.

2. **Build phase (`src/build.js`)**
- Parses inputs via `parseInputs('compute')`. This validates `path` and `network` but does not require the wallet key.
- Detects fork PRs (by comparing head/base repo names). When detected, it records `uploadStatus=fork-pr-blocked` in the context and emits a notice that upload will be blocked.
- Resolves `path` against the workspace and generates a CAR using `createCarFile()`.
- Stores the CAR file path, size, and IPFS root CID in the in-memory context (see `src/context.js`).
- Merges additional metadata (run id, PR details) through `mergeAndSaveContext()`.

3. **Upload phase (`src/upload.js`)**
- Parses inputs via `parseInputs('upload')`. This enforces presence of `walletPrivateKey` and confirms `network`, `minStorageDays`, and `filecoinPayBalanceLimit` rules.
- If the build context marked the run as `fork-pr-blocked`, the upload phase writes outputs, posts the explanatory PR comment, and exits without touching Filecoin.
- Validates that the CAR file still exists on disk.
- Calls `initializeSynapse({ walletPrivateKey, network })`, which selects the correct RPC endpoint (`RPC_URLS[network].websocket`) and bootstraps filecoin-pin.
- Fetches current payment status, then hands control to `handlePayments()` for deposit logic.
- Uploads the CAR to Filecoin via `uploadCarToFilecoin()`; this returns piece CID, dataset id, provider info, preview URL, and canonical network name from filecoin-pin.
- Updates the context, writes GitHub Action outputs, appends a step summary, and posts/updates the PR comment via `commentOnPR()`.

## Input Parsing (`src/inputs.js`)

`parseInputs()` uses a single schema for both phases:
- `path`: required for both phases.
- `walletPrivateKey`: required when `phase !== 'compute'`.
- `network`: required; must be `mainnet` or `calibration`.
- `minStorageDays`: optional number (defaults to `0` when unset).
- `filecoinPayBalanceLimit`: bigint parsed from USDFC string; required when `minStorageDays > 0`.
- `providerAddress`, `withCDN`: optional advanced settings with defaults.

The helper supports both environment-variable fallback (`INPUT_<NAME>`) and the `INPUTS_JSON` bundle populated by `action.yml`.

## Payment Handling (`src/filecoin.js` – `handlePayments`)

- Ensures Synapse allowances are configured via `checkAndSetAllowances()`.
- Pulls current balance with `getPaymentStatus()`.
- If `minStorageDays > 0`, computes the top-up required using `computeTopUpForDuration()`.
- Enforces the hard ceiling defined by `filecoinPayBalanceLimit`. If the current balance already meets or exceeds the limit, no deposit happens. If the computed top-up would exceed the limit, it is reduced to the largest permissible amount.
- Executes a deposit through `depositUSDFC()` when the final top-up is positive and refreshes payment status for downstream reporting.

## Context & Outputs

- Context lives in-memory inside `src/context.js`. Build and upload occur in the same job, so filesystem artifacts are not required for hand-off.
- `writeOutputs()` exposes CID, dataset, provider, CAR path, and status. Fork-blocked runs still surface the CAR information to aid reviewers.
- `writeSummary()` appends a markdown summary detailing payment status, provider links (via `pdp.vxb.ai/<network>`), and CAR size.
- `commentOnPR()` reuses existing bot comments when possible and uses the default workflow token.

## Error Handling

- Domain-specific failures throw `FilecoinPinError` with codes for insufficient funds, invalid private keys, and balance-limit violations.
- `handleError()` surfaces guidance tailored to the inputs (e.g., advising updates to `filecoinPayBalanceLimit`).
- `run.mjs` guarantees Synapse cleanup even when build or upload throws.

130 changes: 130 additions & 0 deletions .github/actions/upload/README.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe discuss the failure cases that can occur that will fail the action?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be covered by FLOW and USAGE now, but we can add into README if we want. there is also src/errors.js that has some common errors we can call out.

Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Filecoin Pin Upload Action

Composite GitHub Action that packs a file or directory into a UnixFS CAR, uploads it to Filecoin, and publishes artifacts and context for easy reuse.

## Quick Start

Run your build in an untrusted workflow, publish the build output as an artifact, then run this action in a trusted workflow to create the CAR and upload to Filecoin. Fork PR support is currently disabled, so workflows must run within the same repository.

**Step 1: Build workflow** (no secrets):
```yaml
# .github/workflows/build-pr.yml
name: Build PR Content
on: pull_request

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run build
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: site-dist
path: dist
```

**Step 2: Upload workflow** (runs after build, uses secrets):
```yaml
# .github/workflows/upload-to-filecoin.yml
name: Upload to Filecoin
on:
workflow_run:
workflows: ["Build PR Content"]
types: [completed]

jobs:
upload:
if: github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
permissions:
actions: read
pull-requests: write
steps:
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: site-dist
path: dist
github-token: ${{ github.token }}
repository: ${{ github.event.workflow_run.repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}

- name: Upload to Filecoin
uses: sgtpooki/filecoin-upload-action@v1
with:
path: dist
walletPrivateKey: ${{ secrets.FILECOIN_WALLET_KEY }}
network: calibration
minStorageDays: "30"
filecoinPayBalanceLimit: "0.25"
```

**Versioning**: This action uses [Semantic Release](https://semantic-release.gitbook.io/) for automated versioning. Use version tags like `@v1`, `@v1.0.0`, or commit SHAs for supply-chain safety.

## Inputs

See [action.yml](./action.yml) the input parameters and their descriptions.

## Security & Permissions Checklist

- ✅ Pin the action by version tag or commit SHA
- ✅ Grant `actions: read` if you want artifact reuse (cache fallback) to work
- ✅ Protect workflow files with CODEOWNERS/branch protection
- ✅ **Always** hardcode `minStorageDays` and `filecoinPayBalanceLimit` in trusted workflows
- ✅ **Never** use `pull_request_target` - use the two-workflow pattern instead
- ✅ Enable **branch protection** on main to require reviews for workflow changes
- ✅ Use **CODEOWNERS** to require security team approval for workflow modifications
- ⚠️ Consider gating deposits with Environments that require approval

## Usage

The action uses a secure two-workflow pattern by default. This currently works for **same-repo PRs only** (fork PR support temporarily disabled).

Split your CI into untrusted build + trusted upload workflows.

**Security Note**: The `workflow_run` trigger always executes the workflow file from your main branch, not from the PR. Even if a PR modifies the upload workflow to change hardcoded limits, those changes won't apply until the PR is merged.

## Current Limitations

**⚠️ Fork PR Support Disabled**

- Only same-repo PRs and direct pushes to main are supported
- PR commenting works, but shows different message for fork PRs
- This limits non-maintainer PR actors from draining funds from unaware repo owners

**See [examples/two-workflow-pattern/](./examples/two-workflow-pattern/)** for complete, ready-to-use workflow files.

## Releases & Versioning

This action uses [Semantic Release](https://semantic-release.gitbook.io/) for automated versioning based on [Conventional Commits](https://www.conventionalcommits.org/).

### Available Versions

- **`@v1`** - Latest v1.x.x release (recommended for most users)
- **`@v1.0.0`** - Specific version (recommended for production)
- **`@<commit-sha>`** - Specific commit (maximum security)

### Version Bumps

- **Patch** (`1.0.0` → `1.0.1`): Bug fixes, docs, refactoring
- **Minor** (`1.0.0` → `1.1.0`): New features
- **Major** (`1.0.0` → `2.0.0`): Breaking changes

### Release Process

Releases are automatically created when changes are pushed to `main` with conventional commit messages. See [CONTRIBUTING.md](./CONTRIBUTING.md) for commit message guidelines.

## Documentation

- **[examples/two-workflow-pattern/](./examples/two-workflow-pattern/)** - Ready-to-use workflow files (recommended)
- **[USAGE.md](./USAGE.md)** - Complete usage guide with all patterns
- **[FLOW.md](./FLOW.md)** - Internal architecture & how the action works under the hood
- **[examples/README.md](./examples/README.md)** - Detailed setup instructions

## Caching & Artifacts

- Cache key: `filecoin-pin-v1-${ipfsRootCid}` enables reuse for identical content.
- Artifacts: `filecoin-pin-artifacts/upload.car` and `filecoin-pin-artifacts/context.json` are published for each run.
- PR comments include the IPFS root CID, dataset ID, piece CID, and preview link.
163 changes: 163 additions & 0 deletions .github/actions/upload/USAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Usage Guide

This action builds a UnixFS CAR from your site or files and uploads it to Filecoin in a single invocation. For security, separate untrusted build steps from the trusted upload step.

## Recommended Pattern: Build + Upload Workflows

1. **Build workflow** (no secrets) compiles your project and uploads the build output as an artifact.
2. **Upload workflow** (trusted) downloads the artifact, runs this action, and provides wallet secrets.

### Workflow 1: Build (Untrusted)

```yaml
# .github/workflows/build-pr.yml
name: Build PR Content

on:
pull_request:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Build your site
run: npm run build

- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: site-dist
path: dist
```

### Workflow 2: Upload (Trusted)

```yaml
# .github/workflows/upload-to-filecoin.yml
name: Upload to Filecoin

on:
workflow_run:
workflows: ["Build PR Content"]
types: [completed]

jobs:
upload:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
permissions:
actions: read
pull-requests: write
steps:
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: site-dist
path: dist
github-token: ${{ github.token }}
repository: ${{ github.event.workflow_run.repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}

- name: Upload to Filecoin
uses: sgtpooki/filecoin-upload-action@v1
with:
path: dist
walletPrivateKey: ${{ secrets.WALLET_PRIVATE_KEY }}
network: calibration
minStorageDays: "30"
filecoinPayBalanceLimit: "0.25"
```

**Security hints**:
- Build workflow never sees wallet secrets.
- Upload workflow runs from the main branch version of the file when triggered via `workflow_run`, so PRs cannot change hardcoded values until merged.
- Hardcode financial parameters in trusted workflows and review changes carefully.

---

## Alternative: Single Workflow (Trusted Repos Only)

If every contributor is trusted and you do not accept fork PRs, you can run build and upload in the same job:

```yaml
name: Upload to Filecoin

on:
pull_request:
push:
branches: [main]

jobs:
upload:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm run build
- name: Upload to Filecoin
uses: sgtpooki/filecoin-upload-action@v1
with:
path: dist
walletPrivateKey: ${{ secrets.WALLET_PRIVATE_KEY }}
network: mainnet
minStorageDays: "7"
filecoinPayBalanceLimit: "1.00"
```

Use this approach only when you fully trust everyone who can open PRs.

---

## Input Reference

### `path`
- **Type**: `string`
- **Required**: Yes
- **Description**: File or directory to package into a CAR and upload.

### `walletPrivateKey`
- **Type**: `string`
- **Required**: Yes when uploading
- **Description**: EVM-compatible private key for the Filecoin wallet.

### `network`
- **Type**: `string`
- **Required**: Yes
- **Options**: `mainnet`, `calibration`
- **Description**: Selects the Filecoin network; controls the RPC endpoint used by filecoin-pin.

### `minStorageDays`
- **Type**: `string`
- **Required**: No
- **Description**: Desired storage runway in days. When provided, the action calculates the deposit needed to reach this runway.

### `filecoinPayBalanceLimit`
- **Type**: `string`
- **Required**: Yes if `minStorageDays` is provided
- **Description**: Maximum Filecoin Pay balance (USDFC) allowed after deposits.

### `providerAddress`
- **Type**: `string`
- **Default**: `0xa3971A7234a3379A1813d9867B531e7EeB20ae07`
- **Description**: Optional override for the storage provider.

### `withCDN`
- **Type**: `boolean`
- **Default**: `false`
- **Description**: Request CDN support when available. Warning: filecoin-pin does not yet adjust deposit calculations for CDN usage.

---

## Outputs

- `ipfsRootCid`: IPFS Root CID
- `dataSetId`: Synapse Data Set ID
- `pieceCid`: Filecoin Piece CID
- `providerId`: Storage Provider ID
- `providerName`: Storage Provider Name
- `carPath`: Path to the generated CAR file
- `uploadStatus`: Status of the run (e.g., `uploaded`, `fork-pr-blocked`)

Loading