Skip to content

Proposal: monorepo & no git submodules #229

@rvagg

Description

@rvagg

Problem

  • We're struggling with our organisation and tooling, for dev and releases.
  • We have inconsistencies in layout and tooling across the various projects.
  • We have challenges with versioning and keeping everything in sync, and submodules just plain suck.
  • We're pretending that these contracts are independent, but they're not, really, they're all part of the same thing, even if filecoin-pay is intended to serve a broader ecosystem of services.

Proposal

  • Pull in https://github.com/FilOzone/filecoin-pay, https://github.com/FilOzone/pdp and https://github.com/FilOzone/SessionKeyRegistry into this repository
  • Use npm workspaces to organise them and manage our dependencies
    • OpenZeppelin publish their contracts to npm so it's just a versioned dependency
    • forge-std isn't published (authoritatively) to npm but we can use a git dependency and pin to a specific commit
  • Use https://github.com/googleapis/release-please or https://github.com/changesets/changesets to manage versioning and creating formal releases along with their changelogs. We're going release-please with the JS repos so we could just be consistent - we could roll up the manual changelog editing thing into the release-please flow by editing/suggesting directly on the PR that release-please opens and maintains.
  • Use npm scripts to either manage or replace our existing tools/ scripts (we can just run them from there, or write them in JS, or whatever other combo we want)
  • Rely on conventional-commits for versioning - we can do some light enforcement of PR title like we do in Lotus and just use squash merges when there's commit sprawl to deal with (we could even enforce always-squash-merge, even using a merge queue to do it).

Our npm monorepo would be defined as this in package.json:

    "workspaces": [
      "packages/pay",
      "packages/pdp",
      "packages/service-provider-registry",
      "packages/session-key-registry",
      "packages/warm-storage"
    ]

Dependencies would look something like:

    "dependencies": {
      "@openzeppelin/contracts": "5.0.2",
      "@pythnetwork/pyth-sdk-solidity": "3.0.0",
      "forge-std": "github:foundry-rs/forge-std#v1.8.2"
      "@filoz/pdp": "1.0.0",
      "@filoz/pay": "1.0.0"
    }

We'd use a remappings to link them, probably foundry.toml but could also be remappings.txt I think:

      '@openzeppelin/=node_modules/@openzeppelin/',
      'forge-std/=node_modules/forge-std/src/',
      '@filoz/pay/=packages/pay/src/',
      '@filoz/pdp/=packages/pdp/src/',
      '@filoz/warm-storage/=packages/warm-storage/src/',

We'd end up with a structure something like this:

filecoin-onchain/
  ├── packages/
  │   ├── pay/
  │   │   ├── src/
  │   │   │   └── Payments.sol
  │   │   ├── test/
  │   │   └── package.json
  │   ├── pdp/
  │   │   ├── src/
  │   │   │   ├── PDPVerifier.sol
  │   │   │   └── interfaces/
  │   │   │       └── IPDPVerifier.sol
  │   │   ├── test/
  │   │   └── package.json
  │   └── warm-storage/
  │       ├── src/
  │       │   ├── WarmStorage.sol
  │       │   └── WarmStorageView.sol
  │       ├── internal/
  │       │   └── WarmStorageStateLibrary.sol
  │       ├── test/
  │       └── package.json
  ├── scripts/
  │   └── deploy-all.sh
  ├── node_modules/
  ├── foundry.toml
  └── package.json

release-please (and changesets I think) works well with monorepos, each package can have its own changelog and package and gets its own release PR.

https://github.com/mario-eth/soldeer is an alternative to using npm for managing dependencies, it's specifically for Solidity packages. But I think we might be better off using a more holistic solution, particularly one we are already using elsewhere.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

🐱 Todo

Relationships

None yet

Development

No branches or pull requests

Issue actions