chore(nix): add reproducible tarball build with nix #258
Conversation
Add packages.default derivation to flake.nix that builds the npm package tarball using pnpm_10.fetchDeps for reproducible builds. - Use stdenv.mkDerivation with pnpm_10.configHook - Fetch dependencies with fixed hash for reproducibility - Run pnpm build and pnpm pack in buildPhase/installPhase - Output tarball to $out directory This enables `nix build` to generate a reproducible tarball that can be used for npm publishing, ensuring consistent builds across CI and local environments.
Add script and lefthook configuration to automatically update the pnpm deps hash in flake.nix when pnpm-lock.yaml changes. - Add scripts/update-pnpm-hash.sh to calculate and update hash - Configure lefthook to run script on pnpm-lock.yaml changes - Auto-stage updated flake.nix after hash update This ensures the Nix build remains functional after dependency updates without manual hash maintenance.
Update release and dry-publish workflows to use `nix build` instead of `pnpm pack` for generating package tarballs. - release.yaml: Build with nix, publish from result/*.tgz - dry-publish.yaml: Build with nix, pass tarball to pkg-pr-new This ensures CI uses the same reproducible build process as local development, improving build consistency and reliability.
Separate the combined build-and-test job into distinct jobs: - build: Uses nix build for reproducible package generation - test: Runs pnpm test in nix develop shell This allows build and test to run in parallel and makes it clearer which step failed when issues occur.
Remove the preinstall script that enforces pnpm usage. This check is unnecessary as the project uses nix develop which provides the correct pnpm version, and it can cause issues in CI environments.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 8 files
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="scripts/update-pnpm-hash.sh">
<violation number="1" location="scripts/update-pnpm-hash.sh:37">
P1: Script silently masks errors when `nix build` fails for reasons other than hash mismatch. If nix is not installed, flake has syntax errors, or any other failure occurs, the script will misleadingly report "Hash is up to date" and exit successfully. Consider checking the exit status and error type before assuming the hash is up to date.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
scripts/update-pnpm-hash.sh
Outdated
|
|
||
| echo "Updated $FLAKE_FILE with new hash" | ||
| else | ||
| echo "Hash is up to date" |
There was a problem hiding this comment.
P1: Script silently masks errors when nix build fails for reasons other than hash mismatch. If nix is not installed, flake has syntax errors, or any other failure occurs, the script will misleadingly report "Hash is up to date" and exit successfully. Consider checking the exit status and error type before assuming the hash is up to date.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At scripts/update-pnpm-hash.sh, line 37:
<comment>Script silently masks errors when `nix build` fails for reasons other than hash mismatch. If nix is not installed, flake has syntax errors, or any other failure occurs, the script will misleadingly report "Hash is up to date" and exit successfully. Consider checking the exit status and error type before assuming the hash is up to date.</comment>
<file context>
@@ -0,0 +1,38 @@
+
+ echo "Updated $FLAKE_FILE with new hash"
+else
+ echo "Hash is up to date"
+fi
</file context>
Different platforms download different platform-specific npm packages (e.g., darwin-arm64 vs linux-x64 binaries), resulting in different hashes for pnpm deps. - Add pnpmDepsHash map with per-system hashes - Update hash script to detect current system and update only that hash - Add hashes for x86_64-linux (CI) and aarch64-darwin (local dev)
There was a problem hiding this comment.
Pull request overview
This PR adds Nix-based reproducible tarball builds to replace the traditional pnpm pack workflow, ensuring consistent builds across CI and local environments through content-addressable dependency management.
Key Changes:
- Introduces a Nix package derivation in
flake.nixthat usespnpm_10.fetchDepsfor reproducible dependency fetching - Adds automated pnpm dependency hash management via pre-commit hook and update script
- Migrates CI/CD workflows from
pnpm packtonix buildfor all package creation steps
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
flake.nix |
Adds packages.default derivation with nodejs_24, pnpm_10, and build/install phases for creating npm tarball |
scripts/update-pnpm-hash.sh |
Bash script to automatically extract and update pnpm deps hash in flake.nix when lockfile changes |
lefthook.yaml |
Adds pre-commit hook to trigger hash update script when pnpm-lock.yaml is modified |
.github/workflows/release.yaml |
Replaces pnpm pack with nix build and updates tarball path to result/*.tgz |
.github/workflows/dry-publish.yaml |
Adds explicit build step using nix build and updates publish command to use result tarball |
.github/workflows/ci.yaml |
Splits combined build-and-test job into separate parallel build and test jobs |
package.json |
Removes preinstall script that enforced pnpm-only usage |
.gitignore |
Adds result directory (nix build output symlink) to gitignore |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,42 @@ | |||
| #!/usr/bin/env bash | |||
| set -euo pipefail | |||
|
|
|||
There was a problem hiding this comment.
The script lacks a proper shebang validation mechanism and doesn't verify that required commands (nix, sed, grep) are available before use. While the script uses set -euo pipefail for strict error handling, it would be more maintainable to check for command availability upfront and provide clear error messages if dependencies are missing.
| # Ensure the script is running under Bash | |
| if [[ -z "${BASH_VERSION:-}" ]]; then | |
| echo "Error: This script must be run with bash." >&2 | |
| exit 1 | |
| fi | |
| # Verify required commands are available | |
| for cmd in nix sed grep; do | |
| if ! command -v "$cmd" >/dev/null 2>&1; then | |
| echo "Error: Required command '$cmd' not found in PATH." >&2 | |
| exit 1 | |
| fi | |
| done |
| shell: bash | ||
| run: | | ||
| PACKAGE_TGZ=$(ls *.tgz | head -n 1) | ||
| PACKAGE_TGZ=$(ls result/*.tgz | head -n 1) |
There was a problem hiding this comment.
The command ls result/*.tgz | head -n 1 assumes that the nix build command creates exactly one .tgz file in the result directory. If multiple .tgz files exist or if the build produces a different file structure, this will either select an unexpected file or fail. Consider being more explicit about the expected filename pattern or adding error handling to verify exactly one tarball exists before attempting to publish.
| PACKAGE_TGZ=$(ls result/*.tgz | head -n 1) | |
| set -euo pipefail | |
| shopt -s nullglob | |
| files=(result/*.tgz) | |
| if [ "${#files[@]}" -eq 0 ]; then | |
| echo "Error: No .tgz package files found in result/." >&2 | |
| exit 1 | |
| elif [ "${#files[@]}" -gt 1 ]; then | |
| echo "Error: Multiple .tgz package files found in result/: ${files[*]}" >&2 | |
| exit 1 | |
| fi | |
| PACKAGE_TGZ="${files[0]}" |
.github/workflows/dry-publish.yaml
Outdated
|
|
||
| - name: 🚀 Dry Run Publish Package | ||
| run: nix develop --command pnpm dlx pkg-pr-new publish --pnpm | ||
| run: nix develop --command pnpm dlx pkg-pr-new publish result/*.tgz |
There was a problem hiding this comment.
The glob pattern result/*.tgz is passed to pkg-pr-new publish which may not handle glob patterns correctly. Shell glob expansion happens before the command is executed, but if no files match or multiple files match, the behavior might be unexpected. Consider explicitly using shell expansion with quotes or verifying the exact tarball path before passing it to the command.
| run: nix develop --command pnpm dlx pkg-pr-new publish result/*.tgz | |
| run: | | |
| shopt -s nullglob | |
| tgzs=(result/*.tgz) | |
| if [ "${#tgzs[@]}" -ne 1 ]; then | |
| echo "Expected exactly one .tgz file in result/, found ${#tgzs[@]}." >&2 | |
| exit 1 | |
| fi | |
| nix develop --command pnpm dlx pkg-pr-new publish "${tgzs[0]}" |
| packageJson = builtins.fromJSON (builtins.readFile ./package.json); | ||
| pnpmDepsHash = { | ||
| x86_64-linux = "sha256-PrCGXf5r03gfsoGJAzew592Al1G5dx6xa/qFxazuqUo="; | ||
| aarch64-linux = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; |
There was a problem hiding this comment.
The package name "stackone-ai" in the derivation doesn't match the actual npm package name "@stackone/ai" from package.json. While this doesn't break the build since pnpm.fetchDeps doesn't publish to npm, it creates inconsistency and could cause confusion. Consider using the actual package name without the @ symbol (e.g., "stackone-ai-node" to match the GitHub repo name, or extract it from package.json).
| echo "Error: $FLAKE_FILE not found" | ||
| exit 1 | ||
| fi | ||
|
|
There was a problem hiding this comment.
The script assumes the presence of the nix command and doesn't check for its availability. If nix is not installed or not in PATH, the script will fail with an unclear error message. Consider adding a check to verify nix is available before attempting to run it, or handle the error more gracefully with a helpful error message.
| # Check if nix is available | |
| if ! command -v nix >/dev/null 2>&1; then | |
| echo "Error: 'nix' command not found. Please install Nix and ensure it is in your PATH." >&2 | |
| exit 1 | |
| fi |
| stage_fixed: true | ||
| - name: update-pnpm-hash | ||
| glob: 'pnpm-lock.yaml' | ||
| run: ./scripts/update-pnpm-hash.sh |
There was a problem hiding this comment.
The script path ./scripts/update-pnpm-hash.sh uses a relative path and won't work correctly if the git hook is triggered from a subdirectory. Lefthook runs commands from the repository root, but the script should be executable. Consider adding execute permissions to the script file or using bash ./scripts/update-pnpm-hash.sh explicitly to ensure it runs correctly.
| run: ./scripts/update-pnpm-hash.sh | |
| run: bash ./scripts/update-pnpm-hash.sh |
| "lint:oxlint": "oxlint --max-warnings=0 --type-aware --type-check", | ||
| "lint:knip": "knip", | ||
| "preinstall": "npx only-allow pnpm", | ||
| "prepack": "npm pkg delete scripts.preinstall && pnpm run build", |
There was a problem hiding this comment.
The prepack script attempts to delete scripts.preinstall with npm pkg delete scripts.preinstall, but the preinstall script is being removed in this PR. This will cause the prepack script to fail or have unexpected behavior. Since the preinstall script no longer exists, this deletion step should be removed from the prepack script.
| "prepack": "npm pkg delete scripts.preinstall && pnpm run build", | |
| "prepack": "pnpm run build", |
| # Extract the new hash | ||
| NEW_HASH=$(echo "$OUTPUT" | grep "got:" | sed 's/.*got:[[:space:]]*//' | tr -d '[:space:]') | ||
|
|
||
| if [[ -z "$NEW_HASH" ]]; then | ||
| echo "Error: Could not extract new hash" |
There was a problem hiding this comment.
The hash extraction pattern grep "got:" | sed 's/.*got:[[:space:]]*//' assumes a specific nix error message format. The pattern is fragile and may break if nix changes its error message format. Additionally, the sed regex pattern should be more explicit to match the expected format. Consider using a more robust extraction method or adding validation that the extracted hash matches the expected sha256 format (e.g., sha256-[A-Za-z0-9+/=]+).
| # Extract the new hash | |
| NEW_HASH=$(echo "$OUTPUT" | grep "got:" | sed 's/.*got:[[:space:]]*//' | tr -d '[:space:]') | |
| if [[ -z "$NEW_HASH" ]]; then | |
| echo "Error: Could not extract new hash" | |
| # Extract the new hash by matching the expected sha256 format in the output | |
| NEW_HASH=$(printf '%s\n' "$OUTPUT" | grep -oE 'sha256-[A-Za-z0-9+/=]+' | head -n1 || true) | |
| if [[ -z "$NEW_HASH" || ! "$NEW_HASH" =~ ^sha256-[A-Za-z0-9+/=]+$ ]]; then | |
| echo "Error: Could not extract valid sha256 hash from nix output" |
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="flake.nix">
<violation number="1" location="flake.nix:26">
P1: Placeholder hash for `aarch64-linux` will cause build failures. The hash `sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=` is not a valid pnpm deps hash - it needs to be computed by running `nix build` on an aarch64-linux system and using the resulting hash from the error message.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
| packageJson = builtins.fromJSON (builtins.readFile ./package.json); | ||
| pnpmDepsHash = { | ||
| x86_64-linux = "sha256-PrCGXf5r03gfsoGJAzew592Al1G5dx6xa/qFxazuqUo="; | ||
| aarch64-linux = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; |
There was a problem hiding this comment.
P1: Placeholder hash for aarch64-linux will cause build failures. The hash sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= is not a valid pnpm deps hash - it needs to be computed by running nix build on an aarch64-linux system and using the resulting hash from the error message.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At flake.nix, line 26:
<comment>Placeholder hash for `aarch64-linux` will cause build failures. The hash `sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=` is not a valid pnpm deps hash - it needs to be computed by running `nix build` on an aarch64-linux system and using the resulting hash from the error message.</comment>
<file context>
@@ -16,11 +16,16 @@
packageJson = builtins.fromJSON (builtins.readFile ./package.json);
+ pnpmDepsHash = {
+ x86_64-linux = "sha256-PrCGXf5r03gfsoGJAzew592Al1G5dx6xa/qFxazuqUo=";
+ aarch64-linux = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
+ aarch64-darwin = "sha256-GDY7RZUl6A0d3l8Rz6X1sHQfwHgM2GKpcJ65yAKOmrg=";
+ };
</file context>
- Add --print-build-logs to nix build for better CI visibility - Revert dry-publish to use pnpm directly as pkg-pr-new doesn't support pre-built tarballs
commit: |
Summary
nix buildderivation to generate npm package tarball reproduciblyWhat Changed
packages.defaultderivation usingpnpm_10.fetchDepsfor reproducible buildsnix buildinstead ofpnpm packWhy
Using Nix for package building ensures:
Summary by cubic
Adds a reproducible Nix build to generate the npm tarball and updates CI to publish from that output. Also adds an auto-update hook for the pnpm deps hash to keep Nix builds working.
New Features
CI
Written for commit 5878e77. Summary will update automatically on new commits.