Skip to content

Conversation

@Chaitu-Tatipamula
Copy link
Contributor

@Chaitu-Tatipamula Chaitu-Tatipamula commented Oct 17, 2025

Migrate from Git Submodules to NPM Workspaces Monorepo

Summary

Migrates the Filecoin Services project from git submodules to a unified npm workspaces monorepo structure to improve developer experience and simplify dependency management.

Changes

  • Remove git submodules and service_contracts/ directory
  • Consolidate packages into unified monorepo with npm workspaces
  • Centralize dependencies using npm for all packages except fvm-solidity
  • Single Foundry project with unified build/test system
  • Consolidated Makefile with 50+ targets for all development needs
  • Updated GitHub workflows to work with new monorepo structure
  • ABI extraction system compatible with new structure

Resolves #229

- Remove git submodules and service_contracts directory
- Consolidate packages into unified monorepo with npm workspaces
- Centralize dependency management using npm for all packages
- Create single root Foundry project with unified build/test system
- Consolidate individual Makefiles into comprehensive root Makefile
- Update all GitHub workflows to work with new monorepo structure
- Implement ABI extraction system compatible with new structure
- Add comprehensive help system and development workflow targets
- Preserve all package tools and functionality in new structure
@FilOzzy FilOzzy added this to FS Oct 17, 2025
@github-project-automation github-project-automation bot moved this to 📌 Triage in FS Oct 17, 2025
@Chaitu-Tatipamula Chaitu-Tatipamula marked this pull request as draft October 17, 2025 20:08
- Add fvm-solidity as git submodule from filecoin-project/fvm-solidity
- Update .gitignore to allow lib directory for submodules
- Add missing warm-storage lib files (SignatureVerificationLib, StateLibrary, etc.)
- Resolves build failures due to missing FVM Solidity dependencies

This change enables the monorepo structure to build successfully with all
necessary dependencies
@wjmelements
Copy link
Contributor

We have a nice modular codebase. Submodules are great. Monorepos are a burden in minor ways.

  1. The tests are all run for everything every time. This wastes money and developer time. Developers may workaround by running fewer tests, but CI will not.
  2. Reusing our modular contracts such as FilecoinPayV1 for other services is more difficult with monorepos. Their build and install times are longer, though they can sometimes workaround by undoing this change on their own fork.
  3. Modular code in monorepos degrades into interdependent code, causing import cycles.
  4. Monorepos have unbounded scope and so absorb everything they touch. There is no limiting principle preventing us, for example, from bringing synapse-sdk in here too. Our future services would probably also accumulate here.
  5. Migration eliminates the git blame and git log which are helpful for digging up context and rationale.
  6. The likelihood of merge conflicts and other incompatible changes is higher when the whole team is working at the same time on the same repo. At scale this can create a chokepoint and a merge backlog that can grow faster than it takes to run the tests.

I'm not convinced that the benefits outweigh the costs.

@rvagg
Copy link
Collaborator

rvagg commented Oct 30, 2025

@redpanda-f is tackling this one now, needs greater coordination across our team unfortunately so we'll have to work on it as a short multi-person sprint.

@rvagg rvagg closed this Oct 30, 2025
@github-project-automation github-project-automation bot moved this from 📌 Triage to 🎉 Done in FS Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🎉 Done

Development

Successfully merging this pull request may close these issues.

Proposal: monorepo & no git submodules

3 participants