build: Add a flake.nix dev environment, and switch to bun for package management#4020
Open
matthawkins90 wants to merge 6 commits intodecred:masterfrom
Open
build: Add a flake.nix dev environment, and switch to bun for package management#4020matthawkins90 wants to merge 6 commits intodecred:masterfrom
matthawkins90 wants to merge 6 commits intodecred:masterfrom
Conversation
5ea97dc to
308d9c3
Compare
308d9c3 to
544f82a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate from Yarn to Bun
This PR migrates the project's package manager from Yarn to Bun for faster dependency installation and script execution.
I've been noticing that decrediton's dependencies are starting to look a little long in the tooth, so I want to start bumping old/deprecated dependencies, but I also don't want to spend ages fiddling with the dev environment and waiting for dependencies to reload. This PR should make it a whole lot less frustrating.
Summary
Nix Flake Development Environment
Added a Nix flake for a reproducible env using
direnv:flake.nix: Dev shell withbun,nodejs_20,yarn(fallback for older hardware),python311, andcmakeLD_LIBRARY_PATH: Configures native library paths for Electron (GTK, X11, mesa, etc.).envrc: direnv integrationWhy both Bun and Node.js? I couldn't fully get rid of Node.js. While Bun is the new package manager and script runner,
nodejs_20is still required becauseelectron-buildershells out to Node.js during the build process and doesn't support Bun's runtime.Package.json Script Changes
Updated scripts to use
bun runinstead ofnpm runoryarn:build:bun run build-trezor && bun run build-preload && bun run build-main && bun run build-rendererrebuild-dexc:cd modules/dex && bun run installrebuild-pi-ui: Builds git-based pi-ui dependency and removes duplicate React instancesrebuild-sbffi: Compiles native FFI module via cmake-js (This was required to run the dex and also to start a hot dev build)postinstall: Updated to usebun run rebuild-dexc && bun run rebuild-pi-uidev/dev-nosandbox:bun run hot-server -- --start-hot[-nosandbox]package-*: All packaging scripts now usebun run buildThe
postinstallscript also includesunset npm_execpath npm_config_user_agentto ensureelectron-builder install-app-depsuses the system Node.js rather than attempting to use Bun. I got errors with pi-ui until I did this.Test Setup Changes
Updated
test/setup.jsto polyfillTextEncoderandTextDecoder:Why? The
trezor-connectlibrary (v8.2.12-extended) usesTextEncoder/TextDecoderinternally. While these are available globally in browsers and modern Node.js, the Jest test environment (jsdom) doesn't provide them. I got "TextEncoder is not defined" errors during tests until I added these.Testing
After applying these changes:
Note on flaky tests: Running
bun run testoften results in failing tests due to timing-related flakiness when tests run in parallel. If you encounter failures, running the tests sequentially will typically get them all to pass:bun run test -- --runInBandTesting these changes on various machines:
I tested this on a modern NixOS installation, as well as NixOS on older (2010) hardware which doesn't work with bun. Bun fails with "illegal hardware instruction", so I added yarn again to the flake.nix, got it to work, and then added a section to the README.md.
New Lock Files
Unfortunately, in order to allow building from source on older hardware, we have to keep yarn. This means we have both
yarn.lockandbun.lock. I tried several methods to potentially keep these in sync. Bun hasbun install --yarn, which will generate ayarn.lockfile and keep it in sync withbun.lockbeing the source of truth. However, this won't work, because unfortunately the github commit link to pi-ui that gets generated in theyarn.lockdoesn't actually work.So basically, in an ideal world we wouldn't have two lockfiles that could get out of sync, and we would probably want to ONLY use
bun.lock. But I'm trying to maintain some backwards-compatibility with older hardware that can't use bun, and so this was the best I could come up with.It's possible we could add
bun.lockto the gitignore so that people regenerate a newbun.lockbased on theyarn.lockevery time, but I'm open to suggestions.