Pages

docs: record lessons 91 and 92 and two rules #615

Workflow file for this run

	name: Pages

	# Builds the mdBook documentation site, runs bca on this very repo to
	# produce HTML + Markdown quality reports, and (on push to main) deploys
	# both as a single GitHub Pages site. The bca self-scan acts as a
	# threshold gate for the workspace; PR runs exercise the gate without
	# deploying.

	on:
	push:
	branches: [main]
	paths:
	- 'big-code-analysis-book/**'
	- 'src/**'
	- 'big-code-analysis-cli/**'
	- 'big-code-analysis-web/**'
	- 'big-code-analysis-py/**'
	# Root manifest + lockfile track workspace-wide dependency state.
	# Grammar-pin bumps (e.g. `bca-tree-sitter-mozjs = "=0.23.x"`)
	# shift metric values across every module that parses them, so a
	# bump must refresh the published report and re-run the gate.
	- 'Cargo.toml'
	- 'Cargo.lock'
	# Vendored grammar crates: scanner.{c,cc} changes can alter node
	# categorisation and therefore Halstead / cyclomatic counts.
	- 'tree-sitter-ccomment/**'
	- 'tree-sitter-mozcpp/**'
	- 'tree-sitter-mozjs/**'
	- 'tree-sitter-preproc/**'
	- 'tree-sitter-tcl/**'
	- 'bca.toml'
	- '.bca-baseline.toml'
	- '.github/workflows/pages.yml'
	pull_request:
	paths:
	- 'big-code-analysis-book/**'
	- 'src/**'
	- 'big-code-analysis-cli/**'
	- 'big-code-analysis-web/**'
	- 'big-code-analysis-py/**'
	- 'Cargo.toml'
	- 'Cargo.lock'
	- 'tree-sitter-ccomment/**'
	- 'tree-sitter-mozcpp/**'
	- 'tree-sitter-mozjs/**'
	- 'tree-sitter-preproc/**'
	- 'tree-sitter-tcl/**'
	- 'bca.toml'
	- '.bca-baseline.toml'
	- '.github/workflows/pages.yml'
	workflow_dispatch:

	# Default to read-only. The `deploy` job escalates to pages:write +
	# id-token:write so the official `actions/deploy-pages` flow can
	# upload + claim the Pages environment.
	permissions:
	contents: read

	# Action SHAs are pinned to commit hashes to eliminate supply-chain
	# risk from action-release spoofing. Dependabot (configured for the
	# github-actions ecosystem in .github/dependabot.yml) keeps these in
	# sync with upstream releases. The mdBook tarball is downloaded
	# outside the actions/ pipeline and is additionally verified by
	# sha256 (see the "Install mdbook" step) — bump the version and the
	# checksum together.
	#
	# `bca` itself is built from the current checkout (see the `scan`
	# job's `cargo install` step). A pinned-release install used to live
	# here too but it broke whenever main shipped a CLI-artifact schema
	# bump the last release didn't understand (e.g., the v3 baseline
	# schema from #376 was unreadable to v1.1.0's bca). Building from
	# source closes that drift gap; Swatinem/rust-cache keeps cold-build
	# cost bounded. The trade-off vs the previous sha256-pinned install
	# is real but matches how every other Rust workflow in this repo
	# already trusts its own cargo build — see STABILITY.md for the
	# schema-bump operational contract this approach enforces.
	env:
	MDBOOK_VERSION: "0.4.40"
	MDBOOK_SHA256: "9ef07fd288ba58ff3b99d1c94e6d414d431c9a61fdb20348e5beb74b823d546b"
	# Provides the mdbook-gettext preprocessor for the Japanese book
	# build (big-code-analysis-book/po/ja.po). Must stay on the 0.3.x
	# line while MDBOOK_VERSION is 0.4.x — mdbook-i18n-helpers 0.4.0+
	# targets the mdbook 0.5 preprocessor crates. Bump the two pins
	# together. Installed via `cargo install --locked` (no prebuilt
	# binaries are published upstream), matching how this workflow
	# already trusts its own cargo builds (see the note above).
	MDBOOK_I18N_HELPERS_VERSION: "0.3.6"
	# Workspace-wide Rust env, mirrored from ci.yml so the `bca` build
	# in this workflow gates on the same warnings, retries, and
	# incremental settings as the rest of CI. Without these, a warning
	# introduced on main can compile green here while failing ci.yml
	# for the same commit.
	CARGO_TERM_COLOR: always
	CARGO_INCREMENTAL: 0
	CARGO_NET_RETRY: 10
	RUSTUP_MAX_RETRIES: 10
	RUSTFLAGS: "-D warnings"

	jobs:
	build-book:
	name: build mdBook
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	fetch-depth: 1
	submodules: false

	- name: Install mdbook
	run: \|
	set -euo pipefail
	tarball="mdbook-v${MDBOOK_VERSION}-x86_64-unknown-linux-gnu.tar.gz"
	url="https://github.com/rust-lang/mdBook/releases/download/v${MDBOOK_VERSION}/${tarball}"
	mkdir -p "$HOME/.local/bin"
	curl -fsSL --proto '=https' --tlsv1.2 -o "/tmp/${tarball}" "$url"
	echo "${MDBOOK_SHA256} /tmp/${tarball}" \| sha256sum --check --strict -
	tar -xzf "/tmp/${tarball}" -C "$HOME/.local/bin"
	rm -f "/tmp/${tarball}"
	echo "$HOME/.local/bin" >> "$GITHUB_PATH"

	- uses: dtolnay/rust-toolchain@6c977a6ca4077a0ceb28ffbe03f59d46e9ac8772 # stable tip
	with:
	toolchain: stable
	- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2

	- name: Install mdbook-i18n-helpers
	run: cargo install mdbook-i18n-helpers --version "=${MDBOOK_I18N_HELPERS_VERSION}" --locked

	- name: Build book (English)
	run: mdbook build big-code-analysis-book

	# The Japanese book is served from /ja/ under the same Pages
	# site. Build order matters: the English build cleans book/,
	# so the ja build (which only cleans book/ja/) must run second.
	- name: Build book (Japanese)
	run: MDBOOK_BOOK__LANGUAGE=ja mdbook build big-code-analysis-book -d book/ja

	# Belt-and-braces: `actions/deploy-pages` already bypasses
	# Jekyll, but the same artifact directory is also what the
	# manual fallback script (`utils/deploy-book-to-gh-pages.sh`)
	# consumes when publishing to a `gh-pages` branch — where
	# Jekyll WOULD strip `_FontAwesome/`, the search index, etc.
	# Keeping `.nojekyll` here means both publish paths produce
	# an identical, Jekyll-safe artifact.
	- name: Disable Jekyll processing
	run: touch big-code-analysis-book/book/.nojekyll

	# Uploaded as a plain workflow artifact (not the Pages artifact)
	# so the `assemble` job can extend it with the bca reports
	# before the single Pages artifact is produced.
	# `include-hidden-files: true` is REQUIRED here: upload-artifact
	# v4+ strips dotfiles by default, which would silently drop
	# `.nojekyll` from the artifact and break the gh-pages fallback
	# invariant the step above guards.
	- name: Upload book artifact
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
	with:
	name: book-html
	path: big-code-analysis-book/book
	include-hidden-files: true
	if-no-files-found: error
	# PR-built artifacts are never consumed downstream (assemble
	# only runs on main pushes). Keep them around for one day so
	# a reviewer can poke at the book preview if needed, but
	# avoid burning the per-repo artifact-storage quota under
	# steady PR traffic. Main pushes keep the standard 7-day
	# retention so a deploy investigation has the source artifact
	# in reach.
	retention-days: ${{ github.event_name == 'pull_request' && 1 \|\| 7 }}

	scan:
	name: bca self-scan
	runs-on: ubuntu-latest
	# Generous to accommodate cold-cache release builds of the whole
	# bca workspace (many tree-sitter grammar C crates plus the Rust
	# workspace itself). Warm-cache runs finish in 3-5 min; the
	# ceiling absorbs the worst-case "Cargo.lock churned + Swatinem
	# cache evicted" combo without killing useful runs.
	timeout-minutes: 30
	# security-events:write lets the upload-sarif step publish bca's own
	# self-scan offenders to GitHub code scanning (Security tab + PR
	# annotations) — dogfooding the SARIF path we document for adopters
	# in commands/check.md and recipes/ci.md. actions:read is required by
	# upload-sarif on private/internal repos (harmless on public ones)
	# and matches ci.yml's clippy job. The workflow-level default is
	# contents:read.
	permissions:
	contents: read
	security-events: write
	actions: read
	# The walk deny-set — build output, vendored trees, prose, and the
	# third-party corpora under `tests/repositories/` — lives in
	# `.bcaignore` at the repo root and is wired in via `exclude_from`
	# in the auto-discovered `bca.toml` manifest, so the invocations
	# below pick it up without an explicit `--exclude-from` flag. If you
	# edit `.bcaignore`, also refresh `.bca-baseline.toml` in the same
	# commit — the baseline keys are sensitive to which files the
	# walker actually visits.
	#
	# The manifest's `[check] exclude` is a separate, gate-only list
	# (#1146): the dev-tooling trees named there are walked, so they
	# appear in the reports and hotspot tables published below, and are
	# exempt only from the threshold gate step.
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	# `--since "origin/${GITHUB_BASE_REF:-main}"` (set via
	# BCA_SINCE on the threshold-gate step below, #387) resolves a
	# merge-base, which needs full history; the default shallow
	# `fetch-depth: 1` would make that ref unreachable. `bca`
	# soft-warns and falls back to an unpartitioned footer if the
	# base is still unresolvable, so this is a readability win, not
	# a correctness dependency.
	fetch-depth: 0
	submodules: false

	# Build `bca` from the current checkout so the gate always
	# runs against HEAD. This avoids the schema-skew failure mode
	# the previous "download pinned release" approach hit when
	# main shipped a CLI artifact bump (e.g., the v3 baseline
	# from #376 was unreadable to the last v1.1.0 release).
	# Swatinem/rust-cache below keeps the cold-build cost
	# bounded — warm runs are dominated by incremental relink,
	# not full recompile.
	- uses: dtolnay/rust-toolchain@6c977a6ca4077a0ceb28ffbe03f59d46e9ac8772 # stable tip
	with:
	toolchain: stable
	# `shared-key` (vs `key`) lets Swatinem fall back to
	# restore-keys when the exact Cargo.lock+rustc fingerprint
	# changes, so a dep bump still warm-restores most of the
	# workspace. The `--release` profile here is intentionally
	# distinct from ci.yml's default-profile builds — mixing them
	# in one cache would thrash both.
	- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
	with:
	shared-key: pages-scan-release
	# `cargo install --path …` lands the binary in `~/.cargo/bin`
	# (already on PATH on every GHA runner) so we don't have to
	# hand-roll a PATH prepend that would break if a future
	# `CARGO_TARGET_DIR` override moved the build artefacts.
	# `--locked` matches the rest of CI's `--locked` builds.
	- name: Install bca from current checkout
	run: cargo install --path big-code-analysis-cli --locked --bin bca

	- name: bca version
	run: bca --version

	# The deny-set (`exclude_from = ".bcaignore"`) is auto-discovered
	# from the repo-root `bca.toml` manifest, so the report
	# invocations no longer pass `--exclude-from` — workflow, recipe,
	# and local bootstrap all share the same list (issue #355) by
	# reading the manifest. `bca` resolves the manifest by climbing
	# from the working directory to the repo root.
	#
	# Path selection is left to the manifest's `paths = ["."]` key.
	# The manifest resolves it to an absolute root, but the walker
	# re-anchors that root to the `./`-prefixed form (#488), so the
	# `./`-anchored `.bcaignore` deny-set matches and the
	# `--strip-prefix "./"` below still lands on the emitted prefix.
	# The report-generation steps below run BEFORE the threshold gate.
	# A failure here must not mask the gate (skipping it would hide
	# real threshold violations), but it must still fail the job (a
	# broken report is a real defect). So the steps are left as normal
	# (no `continue-on-error`) — a failure correctly reds the job — and
	# the gate itself carries `if: ${{ !cancelled() }}`, so it runs
	# regardless of a prior report failure. (An earlier revision used
	# `continue-on-error` on the reports; that masked a single broken
	# report on PR runs, where `assemble` does not run to catch it.)
	# `--vcs` folds a "Change-history risk" section into the flagship
	# reports, so the AST hotspots and the VCS risk ranking sit
	# side-by-side in one page (issue #573) — the project's primary
	# dogfood of `bca report --vcs`. It builds the history index with
	# default windows from the same `fetch-depth: 0` checkout the VCS
	# JSON step below relies on. SECURITY: `--vcs` never
	# emits author-identity hashes (that is `bca vcs
	# --emit-author-details` only), so nothing author-sensitive reaches
	# the public Pages site.
	- name: Generate Markdown report
	run: \|
	set -euo pipefail
	bca \
	report markdown \
	--vcs \
	--top 30 \
	--strip-prefix "./" \
	--output report.md

	- name: Generate HTML report
	run: \|
	set -euo pipefail
	bca \
	report html \
	--vcs \
	--top 50 \
	--strip-prefix "./" \
	--output index.html

	# The rendered change-history report is now folded into the
	# flagship `report.md` / `index.html` above via `--vcs`. This step
	# additionally publishes the full top-100 ranking as a
	# machine-readable JSON download (deeper than the flagship report's
	# top-30/50 section), using the same `bca.toml` path selection and
	# the job's `fetch-depth: 0` full history.
	#
	# SECURITY: this report is published to the public Pages site.
	# Do NOT add `--emit-author-details` here: it would publish
	# SHA-256 hashes of author emails. The counts/paths/scores emitted
	# without it are all derivable from the public git history.
	#
	# `--output` writes a single whole-repo file (#573).
	- name: Generate VCS risk JSON
	run: \|
	set -euo pipefail
	bca vcs --top 100 --format json --output vcs-report.json

	# Run the gate last so the report artifacts above always upload
	# even when thresholds fail — the report is most useful when CI
	# is red. `if: ${{ !cancelled() }}` makes the gate run even if a
	# report step above failed, so a report defect can never mask a
	# threshold violation (while still failing the job on its own).
	# Delegate to `make self-scan` so the local pre-commit
	# gate and the CI gate share one canonical invocation; the
	# repo-root `bca.toml` manifest owns path selection,
	# `exclude_from`, the per-function thresholds, the cyclomatic `?`
	# policy, and the baseline file, while `--jobs` defaults to
	# `auto`. Drift between local and CI is now impossible by
	# construction.
	#
	# `BCA_SINCE` opts the gate into the #356 diff-aware footer
	# partition (#387): the per-file rollup splits into "Files in this
	# range:" (offenders this PR touched) vs "Other offenders:", so a
	# reviewer skimming a red gate sees what the PR introduced. On a
	# `push` to main `GITHUB_BASE_REF` is empty, so it resolves to
	# `origin/main` (a near-empty range — harmless). `--github-annotations`,
	# the `$GITHUB_STEP_SUMMARY` digest, and the remediation block all
	# auto-enable from GitHub Actions env vars, so no flag is needed for
	# them. `--since` is presentational only; it never changes the
	# gate's pass/fail.
	- name: Threshold gate (baseline-ratcheted, diff-aware)
	if: ${{ !cancelled() }}
	run: make self-scan
	env:
	BCA_SINCE: origin/${{ github.base_ref \|\| 'main' }}

	# Dogfood bca's own SARIF writer: re-run the gate as a pure
	# emitter (`--no-fail` keeps exit 0 regardless of offenders) so
	# this step runs even when the gate above is red, and upload the
	# result to Code Scanning below. Path selection, thresholds, and
	# the baseline all come from the auto-discovered `bca.toml`
	# manifest — same config the gate uses — so the published alerts
	# match the gate's offender set. `if: always()` mirrors the report
	# steps: the SARIF is most useful precisely when CI is red.
	#
	# We deliberately DROP `--report-suppressed` here: GitHub code
	# scanning does not honor the SARIF `suppressions` property
	# natively (it is absent from GitHub's supported-SARIF subset), so
	# offenders emitted as `suppressions` entries surface as open
	# alerts anyway — the opposite of the intent. Without the flag the
	# SARIF carries only genuinely-active offenders (those not silenced
	# by an in-source `bca: suppress` marker or covered by the
	# baseline); suppressed/baselined debt is tracked where it lives —
	# the markers in source and `.bca-baseline.toml` — not as phantom
	# Code Scanning alerts. `--tier=soft=0.95` keeps the
	# published alert set aligned with the soft (headroom) gate, so an
	# unsuppressed function creeping into the 95-100% band shows up as
	# an early warning before the hard gate trips.
	- name: Generate self-scan SARIF
	if: always()
	run: >-
	bca check --output-format sarif --no-fail
	--tier=soft=0.95 --output bca.sarif

	# Fork PRs get a read-only GITHUB_TOKEN without security-events
	# scope, so the upload would error; skip it there (and when no
	# SARIF was produced). `category: bca` keeps these alerts a
	# distinct analysis from ci.yml's `category: clippy` upload.
	- name: Upload self-scan SARIF to code scanning
	if: always() && hashFiles('bca.sarif') != '' && (github.event_name != 'pull_request' \|\| github.event.pull_request.head.repo.full_name == github.repository)
	uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
	with:
	sarif_file: bca.sarif
	category: bca

	- name: Upload reports artifact
	if: always()
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
	with:
	name: bca-reports
	path: \|
	report.md
	index.html
	vcs-report.json
	if-no-files-found: error
	# Same PR-vs-main retention split as the book artifact above —
	# see that comment for the rationale.
	retention-days: ${{ github.event_name == 'pull_request' && 1 \|\| 7 }}

	assemble:
	name: assemble Pages artifact
	# PRs validate scan + book build but do not deploy.
	if: github.event_name != 'pull_request'
	needs: [build-book, scan]
	runs-on: ubuntu-latest
	timeout-minutes: 5
	steps:
	- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
	with:
	name: book-html
	path: site
	- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
	with:
	name: bca-reports
	path: reports-staging
	- name: Merge reports into site
	run: \|
	set -euo pipefail
	mkdir -p site/reports
	cp reports-staging/index.html site/reports/index.html
	cp reports-staging/report.md site/reports/report.md
	# Change-history (VCS) risk (issues #328, #573): the rendered
	# ranking is folded into index.html / report.md via `--vcs`
	# above; this is the deeper top-100 machine-readable download.
	cp reports-staging/vcs-report.json site/reports/vcs-report.json
	# Belt-and-braces in two places:
	# * `site/.nojekyll` should already have travelled with
	# book-html (build-book uploads with include-hidden-files
	# true), but recreate it here to harden against a future
	# upload-artifact regression silently dropping dotfiles.
	# * `site/reports/.nojekyll` keeps the gh-pages fallback
	# script from running Jekyll over the bca reports subtree.
	touch site/.nojekyll site/reports/.nojekyll
	# `configure-pages` exposes the Pages environment metadata
	# (base URL, etc.) to downstream steps and populates the
	# `environment.url` output that `actions/deploy-pages` surfaces
	# on the run page. It is also where Pages-source enablement is
	# validated; without it, `deploy-pages` can succeed against a
	# mis-configured repo and produce an empty URL.
	- name: Configure Pages
	uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
	- name: Upload Pages artifact
	uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
	with:
	path: site

	deploy:
	name: deploy to GitHub Pages
	if: github.event_name != 'pull_request'
	needs: assemble
	runs-on: ubuntu-latest
	timeout-minutes: 10
	permissions:
	pages: write
	id-token: write
	environment:
	name: github-pages
	url: ${{ steps.deployment.outputs.page_url }}
	# Serialize Pages deployments at the JOB level (not workflow
	# level) so a slow deploy cannot overwrite a newer commit's
	# output, without serialising the upstream scan/build-book jobs
	# for every PR. cancel-in-progress=false: finishing the current
	# deploy is preferable to interrupting it mid-upload.
	concurrency:
	group: pages-deploy
	cancel-in-progress: false
	steps:
	- id: deployment
	uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0

Navigation Menu

docs: record lessons 91 and 92 and two rules #615

Workflow file

docs: record lessons 91 and 92 and two rules #615

Uh oh!

Workflow file for this run