Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .coverage-baseline
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Wormhole Go Coverage Baseline
# Auto-generated by: coverage-check -init
# Format: <package> <coverage-percentage>
#
# Coverage must not regress below these values.
# Update with: make coverage-update

# node/ packages
github.com/certusone/wormhole/node/cmd/ccq 15.5
github.com/certusone/wormhole/node/cmd/spy 31.5
github.com/certusone/wormhole/node/pkg/accountant 47.4
github.com/certusone/wormhole/node/pkg/adminrpc 46.6
github.com/certusone/wormhole/node/pkg/altpub 86.3
github.com/certusone/wormhole/node/pkg/common 63.2
github.com/certusone/wormhole/node/pkg/db 73.6
github.com/certusone/wormhole/node/pkg/devnet 41.2
github.com/certusone/wormhole/node/pkg/governor 48.1
github.com/certusone/wormhole/node/pkg/guardiansigner 43.1
github.com/certusone/wormhole/node/pkg/gwrelayer 31.9
github.com/certusone/wormhole/node/pkg/node 65.4
github.com/certusone/wormhole/node/pkg/notary 36.8
github.com/certusone/wormhole/node/pkg/p2p 53.3
github.com/certusone/wormhole/node/pkg/processor 6.1
github.com/certusone/wormhole/node/pkg/publicrpc 19.4
github.com/certusone/wormhole/node/pkg/query 69.9
github.com/certusone/wormhole/node/pkg/telemetry 16.0
github.com/certusone/wormhole/node/pkg/telemetry/prom_remote_write 59.5
github.com/certusone/wormhole/node/pkg/txverifier 54.7
github.com/certusone/wormhole/node/pkg/watchers/algorand 10.9
github.com/certusone/wormhole/node/pkg/watchers/evm 22.2
github.com/certusone/wormhole/node/pkg/watchers/evm/connectors 45.4
github.com/certusone/wormhole/node/pkg/watchers/ibc 22.0
github.com/certusone/wormhole/node/pkg/watchers/near 75.4
github.com/certusone/wormhole/node/pkg/watchers/near/nearapi 61.8
github.com/certusone/wormhole/node/pkg/watchers/solana 15.1
github.com/certusone/wormhole/node/pkg/watchers/sui 7.7

# sdk/ packages
github.com/wormhole-foundation/wormhole/sdk 70.2
github.com/wormhole-foundation/wormhole/sdk/vaa 49.7

4 changes: 4 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,7 @@
## Audit Reports

/audits @djb15 @SEJeff @johnsaigle @mdulin2 @pleasew8t @bemic

## Code coverage tools
scripts/coverage-check @evan-gray @kcsongor @johnsaigle @mdulin2 @pleasew8t @bemic
.coverage-baseline @evan-gray @kcsongor @panoel @johnsaigle @mdulin2 @pleasew8t @bemic
6 changes: 4 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -328,8 +328,10 @@ jobs:
with:
go-version: "1.23.3"
# The go-ethereum and celo-blockchain packages both implement secp256k1 using the exact same header, but that causes duplicate symbols.
- name: Run golang tests
run: cd node && go test -v -timeout 5m -race ./...
# Test flags (-race, -cover, -timeout) are defined in the root Makefile's
# coverage-test target. CI calls Make so there's a single source of truth.
- name: Run tests with coverage and check baseline
run: make coverage

# Run Rust lints and tests
rust-lint-and-tests:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ sui/examples/wrapped_coin
*.prof
# Only used for internal testing
*.testnet.yaml
coverage.txt
/coverage-check
39 changes: 39 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,42 @@ $(BIN)/guardiand: dirs generate
cd node && go build -ldflags "-X github.com/certusone/wormhole/node/pkg/version.version=${VERSION}" \
-mod=readonly -o ../$(BIN)/guardiand \
github.com/certusone/wormhole/node

# ── Coverage targets ──────────────────────────────────────────────
# All coverage targets use the coverage-* prefix.
# Tab-complete "make coverage" to discover them.
#
# coverage — run tests + check against baseline (main entry point)
# coverage-init — create baseline from scratch (first-time setup)
# coverage-update — update baseline after improvements
# coverage-test — just run tests with coverage, no baseline check
#
# Use VERBOSE=1 for detailed output: VERBOSE=1 make coverage
COVERAGE_FLAGS := $(if $(VERBOSE),-v,)

# Internal: build the coverage checker binary (not shown in help)
.PHONY: build-coverage-check
build-coverage-check:
@cd scripts/coverage-check && go build -o ../../coverage-check .

.PHONY: coverage-test
## Run tests with coverage for node/ and sdk/ (no baseline check)
coverage-test:
@echo "Running tests with coverage for node and sdk..."
@(cd node && go test -v -timeout 5m -race -cover ./...) 2>&1 | tee coverage.txt
@(cd sdk && go test -v -timeout 5m -race -cover ./...) 2>&1 | tee -a coverage.txt

.PHONY: coverage
## Run tests and check coverage against baseline
coverage: build-coverage-check coverage-test
@./coverage-check $(COVERAGE_FLAGS)

.PHONY: coverage-init
## Create coverage baseline from scratch (first-time setup)
coverage-init: build-coverage-check coverage-test
@./coverage-check -init

.PHONY: coverage-update
## Update coverage baseline with current values
coverage-update: build-coverage-check coverage-test
@./coverage-check -u
1 change: 1 addition & 0 deletions cspell-custom-words.txt
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,7 @@ Solana's
spydk
Starport
statesync
stdlib
struct
structs
subdenom
Expand Down
15 changes: 15 additions & 0 deletions node/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,18 @@ lint:
test:
go test -v ./...
timeout 10s go test -fuzz=FuzzMessagePublicationUnmarshalBinary -fuzztime=5s ./pkg/common || true

.PHONY: coverage-test
coverage-test:
@echo "Note: Use 'make coverage-test' from repo root to test both node and sdk"
@cd .. && $(MAKE) coverage-test

.PHONY: coverage
coverage:
@echo "Note: Use 'make coverage' from repo root to check both node and sdk"
@cd .. && $(MAKE) coverage

# Backward-compatible aliases
.PHONY: test-coverage check-coverage
test-coverage: coverage-test
check-coverage: coverage
115 changes: 115 additions & 0 deletions scripts/coverage-check/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Go Coverage Check Tool

A minimal Go tool to enforce incremental test coverage improvements without blocking development.

## Rules

1. **No regression**: Packages in the baseline must maintain their coverage (within 0.1% tolerance)
2. **New packages need tests**: Any new package not in the baseline must have at least 10% coverage
3. **Improvements must be committed**: Coverage gains trigger a required baseline update

### What's Excluded from New Package Checks

- `cmd/`, `hack/`, `tools/`, `proto/`, `mock/` — infrastructure/generated code
- `*abi` — generated ABI bindings
- Root `node` package

These can still appear in the baseline if you want to track them.

## Usage

All coverage targets use the `coverage-*` prefix. Tab-complete `make coverage` to discover them:

```bash
make coverage # Run tests + check against baseline (main entry point)
make coverage-init # Create baseline from scratch (first-time setup)
make coverage-update # Update baseline after improvements
make coverage-test # Just run tests with coverage, no baseline check

VERBOSE=1 make coverage # Detailed per-package breakdown
```

CI runs `make coverage` — the same command you run locally. Test flags (`-race`, `-cover`, `-timeout`) are defined once in the Makefile.

### CLI Flags (if running the binary directly)

```bash
./coverage-check # Quiet mode (only shows failures and improvements)
./coverage-check -v # Verbose mode (shows all checks)
./coverage-check -u # Update baseline with current coverage
./coverage-check -init # Create baseline from scratch (first-time setup)
```

The tool expects `coverage.txt` at repo root, generated by `make coverage-test`.

## Scenarios

| Situation | Result | Fix |
|-----------|--------|-----|
| You add tests, coverage improves | Exit 1 — "Coverage improved!" | `make coverage-update` to lock in gains |
| Coverage drops accidentally | Exit 1 — regression detected | Add tests to restore coverage |
| Coverage drops intentionally (dead code removal) | Exit 1 — regression detected | `make coverage-update` + commit with explanation |
| New package with <10% coverage | Exit 1 — below minimum | Add basic tests to reach 10% |
| New package with ≥10% coverage | Exit 1 — new package detected | `make coverage-update` to add to baseline |
| New `cmd/` or `hack/` code, no tests | Pass — excluded from checks | Nothing needed |

## Baseline File

The baseline (`.coverage-baseline` at repo root) maps packages to minimum coverage:

```
# Format: <package> <coverage-percentage>
github.com/certusone/wormhole/node/pkg/db 73.8
github.com/certusone/wormhole/node/pkg/query 69.9
```

**To update**: `make coverage-update` (preserves comments and structure).

**To manually edit**: Change the number, run `make coverage` to verify, commit with explanation.

**To create from scratch**: `make coverage-init` (one-time setup).

## Configuration

Edit constants in `scripts/coverage-check/main.go`:

```go
const (
baselineFile = ".coverage-baseline" // Baseline file location (repo root)
coverageOutputFile = "coverage.txt" // Where to read test coverage from
minNewPkgCoverage = 10.0 // Minimum for new packages
coverageTolerance = 0.1 // Floating point tolerance
)
```

To add exclusions for generated code, add patterns to the `init()` function's pattern list.

## Troubleshooting

**"No coverage data found"**: Package was renamed or deleted. Update the baseline to match.

**"Minimum required: 10.0%" for generated code**: Add an exclusion pattern in `init()`.

**Passes locally, fails in CI (or vice versa)**: Both use `make coverage`. Run from repo root. If they still diverge, check Go version matches (`1.23.3`).

## FAQ

**Q: Why 10% minimum for new packages?**
Low enough to not block development, high enough to ensure basic tests exist.

**Q: Ship urgently without tests?**
Add the package to the baseline with 0% coverage. File a follow-up issue.

**Q: Why per-package baselines instead of total coverage?**
Total coverage drops as you add new code, which blocks development. Per-package is more granular and fair.

## Development

```
scripts/coverage-check/
├── main.go # All logic (~600 lines)
├── go.mod # Module declaration (stdlib only)
└── README.md # This file
```

Key functions: `parseCoverageOutput()`, `loadBaseline()`, `checkBaseline()`, `checkNewPackages()`, `shouldExclude()`, `writeInitialBaseline()`, `writeUpdatedBaseline()`.
3 changes: 3 additions & 0 deletions scripts/coverage-check/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module github.com/certusone/wormhole/scripts/coverage-check

go 1.23.3
Loading
Loading