Merge branch 'zcash-v4.2.0' into sync-zcash-v4.2.0-merge

Author: dmidem

.codespellrc

@@ -1,4 +1,4 @@
 [codespell]
-ignore-words-list = crate,Sur,inout,Groth,groth,re-use,abl,
+ignore-words-list = crate,Sur,inout,Groth,groth,re-use,abl,zcash,zcashd,zebrad,zebra,utxo,utxos,nullifier,nullifiers,sapling,orchard,sprout,backpressure,proptest,thiserror,rocksdb,libsecp,fullnode,peerset,threadsafe,unrepresentable,getblocktemplate,syncer,Actix,Akka,mermaid,println,eprintln,usize,nocapture,Parallelizable,invis,UTXO,Zcash,Zaino,Zallet,librustzcash,Mainnet,Testnet,mainnet,testnet,idents,reentrancy,serializable,deserializable,deserialization,zkSNARK,zkSNARKs,lightwalletd,statics,ser,endianity,aranges
 exclude-file = book/mermaid.min.js
-skip = ./zebra-rpc/qa/rpc-tests,./supply-chain
+skip = ./zebra-rpc/qa/rpc-tests,./supply-chain,./target

.github/ISSUE_TEMPLATE/release.md

@@ -1,10 +1,9 @@
 ---
 name: "🚀 Zebra Release"
-about: 'Zebra team use only'
-title: 'Publish next Zebra release: (version)'
-labels: 'A-release, C-exclude-from-changelog, P-Medium :zap:'
-assignees: ''
-
+about: "Zebra team use only"
+title: "Publish next Zebra release: (version)"
+labels: "A-release, C-exclude-from-changelog, P-Medium :zap:"
+assignees: ""
 ---
 
 # Prepare for the Release
@@ -16,17 +15,19 @@ They can be skipped for urgent releases.
 
 To check consensus correctness, we want to test that the state format is valid after a full sync. (Format upgrades are tested in CI on each PR.)
 
-- [ ] Make sure there has been [at least one successful full sync test](https://github.com/ZcashFoundation/zebra/actions/workflows/ci-tests.yml?query=event%3Aschedule) since the last state change, or
-- [ ] Start a manual workflow run with a Zebra and `lightwalletd` full sync.
+- [ ] Make sure there has been [at least one successful full sync test](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-ci-integration-tests-gcp.yml?query=event%3Aschedule) since the last state change, or
+- [ ] Start a manual workflow run of [`zfnd-ci-integration-tests-gcp.yml`](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-ci-integration-tests-gcp.yml) with both `run-full-sync: true` and `run-lwd-sync: true`.
 
-State format changes can be made in `zebra-state` or `zebra-chain`. The state format can be changed by data that is sent to the state, data created within the state using `zebra-chain`, or serialization formats in `zebra-state` or `zebra-chain`. 
+State format changes can be made in `zebra-state` or `zebra-chain`. The state format can be changed by data that is sent to the state, data created within the state using `zebra-chain`, or serialization formats in `zebra-state` or `zebra-chain`.
 
 After the test has been started, or if it has finished already:
+
 - [ ] Ask for a state code freeze in Slack. The freeze lasts until the release has been published.
 
 ## Checkpoints
 
 For performance and security, we want to update the Zebra checkpoints in every release.
+
 - [ ] You can copy the latest checkpoints from CI by following [the zebra-checkpoints README](https://github.com/ZcashFoundation/zebra/blob/main/zebra-utils/README.md#zebra-checkpoints).
 
 ## Missed Dependency Updates
@@ -36,7 +37,9 @@ Sometimes `dependabot` misses some dependency updates, or we accidentally turned
 This step can be skipped if there is a large pending dependency upgrade. (For example, shared ECC crates.)
 
 Here's how we make sure we got everything:
+
 - [ ] Run `cargo update` on the latest `main` branch, and keep the output
+- [ ] Until we bump the MSRV to 1.88 or higher, `home` must be downgraded manually: `cargo update home@0.5.12 --precise 0.5.11`
 - [ ] If needed, [add duplicate dependency exceptions to deny.toml](https://github.com/ZcashFoundation/zebra/blob/main/book/src/dev/continuous-integration.md#fixing-duplicate-dependencies-in-check-denytoml-bans)
 - [ ] If needed, remove resolved duplicate dependencies from `deny.toml`
 - [ ] Open a separate PR with the changes
@@ -47,12 +50,15 @@ Here's how we make sure we got everything:
 Follow the steps in the [release checklist](https://github.com/ZcashFoundation/zebra/blob/main/.github/PULL_REQUEST_TEMPLATE/release-checklist.md) to prepare the release:
 
 Release PR:
+
 - [ ] Update Changelog
 - [ ] Update README
 - [ ] Update Zebra Versions
+- [ ] Update Crate Versions and Crate Change Logs
 - [ ] Update End of Support Height
 
 Publish Release:
+
 - [ ] Create & Test GitHub Pre-Release
 - [ ] Publish GitHub Release
 - [ ] Publish Rust Crates

.github/ISSUE_TEMPLATE/usability_testing_plan.md

@@ -1,10 +1,9 @@
 ---
 name: "📋 Usability Testing Plan"
 about: Create a Usability Testing Plan
-title: 'Usability Testing Plan'
+title: "Usability Testing Plan"
 labels: C-research
-assignees: ''
-
+assignees: ""
 ---
 
 # Usability Testing Plan
@@ -53,7 +52,6 @@ assignees: ''
 
 <!-- What needs to happen for the participant to successfully complete the task -->
 
-
 ## Session Outline and timing
 
 <!-- The following sections provide some space to plan out the script and tasks for your participants -->
@@ -81,4 +79,3 @@ assignees: ''
 ## Required documentation
 
 <!-- List the documents you will need to produce and bring to the usability testing sessions, e.g consent forms, usability testing script, questionnaires, etc... -->
-

.github/PULL_REQUEST_TEMPLATE/hotfix-release-checklist.md

@@ -1,10 +1,9 @@
 ---
-name: 'Hotfix Release Checklist Template'
-about: 'Checklist to create and publish a hotfix Zebra release'
-title: 'Release Zebra (version)'
-labels: 'A-release, C-exclude-from-changelog, P-Critical :ambulance:'
-assignees: ''
-
+name: "Hotfix Release Checklist Template"
+about: "Checklist to create and publish a hotfix Zebra release"
+title: "Release Zebra (version)"
+labels: "A-release, C-exclude-from-changelog, P-Critical :ambulance:"
+assignees: ""
 ---
 
 A hotfix release should only be created when a bug or critical issue is discovered in an existing release, and waiting for the next scheduled release is impractical or unacceptable.
@@ -55,7 +54,7 @@ follow semver, depending on the thing being fixed.
 ## Test the Pre-Release (if Zebra hotfix)
 
 - [ ] Wait until the Docker binaries have been built on the hotfix release branch, and the quick tests have passed:
-    - [ ] [ci-tests.yml](https://github.com/ZcashFoundation/zebra/actions/workflows/ci-tests.yml)
+  - [ ] [ci-tests.yml](https://github.com/ZcashFoundation/zebra/actions/workflows/ci-tests.yml)
 - [ ] Wait until the [pre-release deployment machines have successfully launched](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-deploy-nodes-gcp.yml?query=event%3Arelease)
 
 ## Publish Release (if Zebra hotfix)

.github/PULL_REQUEST_TEMPLATE/release-checklist.md

@@ -1,10 +1,9 @@
 ---
-name: 'Release Checklist Template'
-about: 'Checklist to create and publish a Zebra release'
-title: 'Release Zebra (version)'
-labels: 'A-release, C-exclude-from-changelog, P-Critical :ambulance:'
-assignees: ''
-
+name: "Release Checklist Template"
+about: "Checklist to create and publish a Zebra release"
+title: "Release Zebra (version)"
+labels: "A-release, C-exclude-from-changelog, P-Critical :ambulance:"
+assignees: ""
 ---
 
 # Prepare for the Release
@@ -14,6 +13,7 @@ assignees: ''
 # Checkpoints
 
 For performance and security, we want to update the Zebra checkpoints in every release.
+
 - [ ] You can copy the latest checkpoints from CI by following [the zebra-checkpoints README](https://github.com/ZcashFoundation/zebra/blob/main/zebra-utils/README.md#zebra-checkpoints).
 
 # Missed Dependency Updates
@@ -23,7 +23,9 @@ Sometimes `dependabot` misses some dependency updates, or we accidentally turned
 This step can be skipped if there is a large pending dependency upgrade. (For example, shared ECC crates.)
 
 Here's how we make sure we got everything:
+
 - [ ] Run `cargo update` on the latest `main` branch, and keep the output
+- [ ] Until we bump the MSRV to 1.88 or higher, `home` must be downgraded manually: `cargo update home@0.5.12 --precise 0.5.11`
 - [ ] If needed, [add duplicate dependency exceptions to deny.toml](https://github.com/ZcashFoundation/zebra/blob/main/book/src/dev/continuous-integration.md#fixing-duplicate-dependencies-in-check-denytoml-bans)
 - [ ] If needed, remove resolved duplicate dependencies from `deny.toml`
 - [ ] Open a separate PR with the changes
@@ -41,11 +43,12 @@ Once you are ready to tag a release, copy the draft changelog into `CHANGELOG.md
 We use [the Release Drafter workflow](https://github.com/marketplace/actions/release-drafter) to automatically create a [draft changelog](https://github.com/ZcashFoundation/zebra/releases). We follow the [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format.
 
 To create the final change log:
+
 - [ ] Copy the [**latest** draft
-  changelog](https://github.com/ZcashFoundation/zebra/releases) into
-  `CHANGELOG.md` (there can be multiple draft releases)
+      changelog](https://github.com/ZcashFoundation/zebra/releases) into
+      `CHANGELOG.md` (there can be multiple draft releases)
 - [ ] Delete any trivial changes
-    - [ ] Put the list of deleted changelog entries in a PR comment to make reviewing easier
+  - [ ] Put the list of deleted changelog entries in a PR comment to make reviewing easier
 - [ ] Combine duplicate changes
 - [ ] Edit change descriptions so they will make sense to Zebra users
 - [ ] Check the category for each change
@@ -56,12 +59,14 @@ To create the final change log:
 README updates can be skipped for urgent releases.
 
 Update the README to:
+
 - [ ] Remove any "Known Issues" that have been fixed since the last release.
 - [ ] Update the "Build and Run Instructions" with any new dependencies.
       Check for changes in the `Dockerfile` since the last tag: `git diff <previous-release-tag> docker/Dockerfile`.
 - [ ] If Zebra has started using newer Rust language features or standard library APIs, update the known working Rust version in the README, book, and `Cargo.toml`s
 
 You can use a command like:
+
 ```sh
 fastmod --fixed-strings '1.58' '1.65'
 ```
@@ -88,20 +93,11 @@ This check runs automatically on pull requests with the `A-release` label. It mu
 
 Zebra follows [semantic versioning](https://semver.org). Semantic versions look like: MAJOR.MINOR.PATCH[-TAG.PRE-RELEASE]
 
-Choose a release level for `zebrad`. Release levels are based on user-visible changes:
+Choose a release level for `zebrad`. Release levels are based on user-visible changes from the changelog:
 
-- Major release: breaking changes to RPCs (fields changed or removed), config files (fields
-  changed or removed), command line (arguments changed or removed), features
-  (features changed or removed), environment variables (changed or removed)
-  or any other external interface of Zebra
-- Minor release: new features are `minor` releases
-- Patch release: otherwise
-
-Update the version using:
-
-```
-cargo release version --verbose --execute --allow-branch '*' -p zebrad patch # [ major | minor ]
-```
+- Mainnet Network Upgrades are `major` releases
+- significant new features or behaviour changes; changes to RPCs, command-line, or configs; and deprecations or removals are `minor` releases
+- otherwise, it is a `patch` release
 
 ## Update Crate Versions and Crate Change Logs
 
@@ -144,6 +140,7 @@ cargo release replace --verbose --execute --allow-branch '*' -p <crate>
 
 The end of support height is calculated from the current blockchain height:
 
+- [ ] Find where the Zcash blockchain tip is now by using a [Zcash Block Explorer](https://mainnet.zcashexplorer.app/) or other tool.
 - [ ] Replace `ESTIMATED_RELEASE_HEIGHT` in [`end_of_support.rs`](https://github.com/ZcashFoundation/zebra/blob/main/zebrad/src/components/sync/end_of_support.rs) with the height you estimate the release will be tagged.
 
 <details>
@@ -160,7 +157,6 @@ The end of support height is calculated from the current blockchain height:
 
 - [ ] Push the version increments and the release constants to the release branch.
 
-
 # Publish the Zebra Release
 
 ## Create the GitHub Pre-Release
@@ -182,7 +178,7 @@ The end of support height is calculated from the current blockchain height:
 ## Test the Pre-Release
 
 - [ ] Wait until the Docker binaries have been built on `main`, and the quick tests have passed:
-    - [ ] [zfnd-ci-integration-tests-gcp.yml](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-ci-integration-tests-gcp.yml?query=branch%3Amain)
+  - [ ] [zfnd-ci-integration-tests-gcp.yml](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-ci-integration-tests-gcp.yml?query=branch%3Amain)
 - [ ] Wait until the [pre-release deployment machines have successfully launched](https://github.com/ZcashFoundation/zebra/actions/workflows/zfnd-deploy-nodes-gcp.yml?query=event%3Arelease)
 
 ## Publish Release

.github/actions/setup-zebra-build/action.yml

@@ -0,0 +1,23 @@
+name: 'Setup Zebra Build Environment'
+description: 'Install protoc as system library'
+runs:
+  using: 'composite'
+  steps:
+    - name: Install protoc on Ubuntu
+      if: runner.os == 'Linux'
+      shell: bash
+      run: |
+        sudo apt-get -qq update
+        sudo apt-get -qq install -y --no-install-recommends protobuf-compiler
+
+    - name: Install protoc on macOS
+      if: runner.os == 'macOS'
+      shell: bash
+      run: |
+        brew install protobuf
+
+    - name: Install protoc on Windows
+      if: runner.os == 'Windows'
+      shell: bash
+      run: |
+        choco install protoc -y

.github/copilot-instructions.md

@@ -0,0 +1,119 @@
+# Zebra — Copilot Review Instructions
+
+You are reviewing PRs for Zebra, a Zcash full node in Rust. Prioritize correctness, consensus safety, DoS resistance, and maintainability. Stay consistent with existing Zebra conventions. Avoid style-only feedback unless it clearly prevents bugs.
+
+If the diff or PR description is incomplete, ask questions before making strong claims.
+
+## Contribution Process Checks
+
+Before reviewing code quality, verify:
+
+- [ ] PR links to a pre-discussed issue
+- [ ] PR description includes Motivation, Solution, and Tests sections
+- [ ] PR title follows conventional commits
+- [ ] If AI tools were used, disclosure is present in the PR description
+
+If these are missing, note it in the review.
+
+## Architecture Constraints
+
+```text
+zebrad (CLI orchestration)
+  → zebra-consensus (verification)
+  → zebra-state (storage/service boundaries)
+  → zebra-chain (data types; sync-only)
+  → zebra-network (P2P)
+  → zebra-rpc (JSON-RPC)
+```
+
+- Dependencies flow **downward only** (lower crates must not depend on higher crates)
+- `zebra-chain` is **sync-only** (no async / tokio / Tower services)
+- State uses `ReadRequest` for queries, `Request` for mutations
+
+## High-Signal Checks
+
+### Tower Service Pattern
+
+If the PR touches a Tower `Service` implementation:
+
+- Bounds must include `Send + Clone + 'static` on services, `Send + 'static` on futures
+- `poll_ready` must call `poll_ready` on all inner services
+- Services must be cloned before moving into async blocks
+
+### Error Handling
+
+- Prefer `thiserror` with `#[from]` / `#[source]`
+- `expect()` messages must explain **why** the invariant holds:
+  ```rust
+  .expect("block hash exists because we just inserted it")  // good
+  .expect("failed to get block")                            // bad
+  ```
+- Don't turn invariant violations into misleading `None`/defaults
+
+### Numeric Safety
+
+- External/untrusted values: prefer `saturating_*` / `checked_*`
+- All `as` casts must have a comment justifying safety
+
+### Async & Concurrency
+
+- CPU-heavy crypto/proof work: must use `tokio::task::spawn_blocking`
+- All external waits need timeouts and must be cancellation-safe
+- Prefer `watch` channels over `Mutex` for shared async state
+- Progress tracking: prefer freshness ("time since last change") over static state
+
+### DoS / Resource Bounds
+
+- Anything from attacker-controlled data must be bounded
+- Use `TrustedPreallocate` for deserialization lists/collections
+- Avoid unbounded loops/allocations
+
+### Performance
+
+- Prefer existing indexed structures (maps/sets) over iteration
+- Avoid unnecessary clones (structs may grow over time)
+
+### Complexity (YAGNI)
+
+When the PR adds abstraction, flags, generics, or refactors:
+
+- Ask: "Is the difference important enough to complicate the code?"
+- Prefer minimal, reviewable changes; suggest splitting PRs when needed
+
+### Testing
+
+- New behavior needs tests
+- Async tests: `#[tokio::test]` with timeouts for long-running tests
+- Test configs must use realistic network parameters
+
+### Observability
+
+- Metrics use dot-separated hierarchical names with existing prefixes (`checkpoint.*`, `state.*`, `sync.*`, `rpc.*`, `peer.*`, `zcash.chain.*`)
+- Use `#[instrument(skip(large_arg))]` for tracing spans
+
+### Changelog & Release Process
+
+- User-visible changes need a `CHANGELOG.md` entry under `[Unreleased]`
+- Library-consumer-visible changes need the crate's `CHANGELOG.md` updated
+- PR labels must match the intended changelog category (`C-bug`, `C-feature`, `C-security`, etc.)
+- PR title follows conventional commits (squash-merged to main)
+
+## Extra Scrutiny Areas
+
+- **`zebra-consensus` / `zebra-chain`**: Consensus-critical; check serialization, edge cases, overflow, test coverage
+- **`zebra-state`**: Read/write separation, long-lived locks, timeouts, database migrations
+- **`zebra-network`**: All inputs are attacker-controlled; check bounds, rate limits, protocol compatibility
+- **`zebra-rpc`**: zcashd compatibility (response shapes, errors, timeouts), user-facing behavior
+- **`zebra-script`**: FFI memory safety, lifetime/ownership across boundaries
+
+## Output Format
+
+Categorize findings by severity:
+
+- **BLOCKER**: Must fix (bugs, security, correctness, consensus safety)
+- **IMPORTANT**: Should fix (maintainability, likely future bugs)
+- **SUGGESTION**: Optional improvement
+- **NITPICK**: Minor style/clarity (keep brief)
+- **QUESTION**: Clarification needed
+
+For each finding, include the file path and an actionable suggestion explaining the "why".