🏗️✨：watch the vendored files for drift

A copy of somebody else's file goes stale without saying so. The
analytics script sat six upstream releases behind for sixteen months,
and what found it was somebody thinking to look.

A copy here is upstream's bytes and nothing else, so the check is a
comparison and needs no version recorded anywhere. Adding another file
to watch is three lines of data.

Weekly rather than on every pull request: it reaches the network, and
`verify.all` runs everything in its directory, so an upstream that is
slow or moved would fail changes that have nothing to do with it. It
sits beside `verify-pull-request.mts`, out of that directory for the
same kind of reason.

It opens an issue rather than a pull request. One raised with
`GITHUB_TOKEN` does not start the checks, so the queue could never land
it; and a file fetched from the internet is worth a person reading
before it arrives, which is why the dependency scanners are here.

Drift and an upstream nobody could reach are separate bits of the exit
code, so neither hides the other: an outage leaves a red run and no
issue rather than a report headed "has drifted" with nothing under it,
and a file nobody can reach cannot hold back a report about one that
drifted.

Signed-off-by: Derek Lewis <DerekNonGeneric@inf.is>
Assisted-by: Claude-Code:claude-opus-5
PR-URL: #1865
Refs: #1548

Author: DerekNonGeneric

.github/workflows/vendored-sync.yml

@@ -0,0 +1,94 @@
+# Watching the third-party files kept in this repository for drift.
+#
+# A vendored copy is upstream's bytes and nothing else, so whether it has
+# fallen behind is a comparison rather than a judgement. Weekly rather than on
+# every pull request, because this reaches the network: an upstream that is
+# slow, moved or unreachable would otherwise fail changes that have nothing to
+# do with it.
+#
+# It opens an issue rather than a pull request, for two reasons. A pull
+# request raised with `GITHUB_TOKEN` does not start the checks, so the queue
+# could never land it. And a file fetched from the internet is worth a person
+# reading before it arrives, which is why the dependency scanners are here at
+# all.
+#
+# Actions are pinned by commit, never by tag.
+name: Vendored sync
+
+on:
+  schedule:
+    # Wednesday, clear of the other two scheduled runs.
+    - cron: '0 5 * * 3'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  issues: write
+
+# A scheduled run and a hand-started one should not both file the same report.
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: false
+
+jobs:
+  check:
+    name: Check vendored files
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out project repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Set up Node.js runtime
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version-file: 'package.json'
+      # The task reads only what node ships with, so there is nothing to
+      # install and no lockfile to resolve before it can run.
+      - name: Compare against upstream
+        id: compare
+        run: |
+          node build/tasks/check-vendored.mts > report.md || code=$?
+          cat report.md
+
+          # Both bits are read: one file drifting says nothing about whether
+          # another was reachable, so neither answer is allowed to hide the
+          # other. Reported after the issue is filed, so a file nobody could
+          # reach cannot hold back a report about one that drifted.
+          echo "drifted=$(( (${code:-0} & 1) != 0 ))" >> "$GITHUB_OUTPUT"
+          echo "unchecked=$(( (${code:-0} & 2) != 0 ))" >> "$GITHUB_OUTPUT"
+      - name: Say so, once
+        if: steps.compare.outputs.drifted == '1'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          TITLE: 📦 a vendored file has drifted from upstream
+        run: |
+          # Matched against the open issues themselves rather than through
+          # search, which is an index and lags behind what was just written.
+          # `gh` pages until it has as many as asked for, and the number is
+          # far past what this repository will hold, so the report cannot be
+          # missed and a duplicate filed beside it.
+          # One issue at a time: a weekly comment on a report nobody has acted
+          # on yet says nothing the report did not.
+          open=$(gh issue list --state open --limit 1000 --json number,title \
+            --jq 'map(select(.title == env.TITLE)) | .[0].number // empty')
+
+          if [ -n "$open" ]; then
+            echo "already reported in #${open}"
+            exit 0
+          fi
+
+          {
+            echo 'A copy kept in this repository no longer matches what'
+            echo 'upstream serves. Read what changed before taking it.'
+            echo
+            cat report.md
+          } > body.md
+
+          gh issue create --title "$TITLE" --body-file body.md \
+            --label '📦 Type: Dependencies'
+      - name: Fail if anything could not be compared
+        # `always()`, so this runs after a report has been filed rather than
+        # instead of one.
+        if: always() && steps.compare.outputs.unchecked == '1'
+        run: |
+          echo '::error::a vendored file could not be compared'
+          exit 1

build/tasks/check-vendored.mts

@@ -0,0 +1,153 @@
+/**
+ * @file Compare vendored third-party files against what upstream serves.
+ * @author The OpenINF Authors & Friends
+ * @license MIT OR Apache-2.0 OR BlueOak-1.0.0
+ * @module {type ES6Module} build/tasks/check-vendored
+ *
+ * Outside `verify/` on purpose, the way `verify-pull-request.mts` is: it
+ * reaches the network, and every task in that directory runs on every pull
+ * request. An upstream that is slow, moved or unreachable would fail changes
+ * that have nothing to do with it.
+ *
+ * The exit code carries both answers at once, since one file drifting says
+ * nothing about whether another was reachable: bit 1 is set when a copy has
+ * drifted, bit 2 when one could not be compared. Drift is a thing to act on
+ * and an upstream nobody can reach is not, so neither hides the other.
+ */
+
+import { readFile } from 'node:fs/promises';
+
+/** One vendored file and where it comes from. */
+type Vendored = {
+  /** Where the copy lives, relative to the repository root. */
+  file: string;
+  /** What upstream serves, which the copy is expected to equal byte for byte. */
+  upstream: string;
+  /** Where to read about a change. */
+  project: string;
+};
+
+/**
+ * A copy here is upstream's bytes and nothing else. Anything this project
+ * needs to say about a file goes beside it rather than inside it, so that
+ * telling whether it has drifted stays a comparison rather than a judgement.
+ */
+const VENDORED: Vendored[] = [
+  {
+    file: '_assets/js/vendor/count.js',
+    upstream: 'https://gc.zgo.at/count.js',
+    project: 'https://github.com/arp242/goatcounter',
+  },
+];
+
+/** How long to wait on an upstream before giving up, in milliseconds. */
+const TIMEOUT = 30_000;
+
+/** Bits of the exit code. Both can be set; neither masks the other. */
+const MATCHED = 0;
+const DRIFTED = 1;
+const UNCHECKED = 2;
+
+/**
+ * Says what went wrong in a sentence rather than a stack trace.
+ * @param {unknown} error Whatever was thrown.
+ * @returns {string} Its message.
+ */
+const reasonOf = (error: unknown) =>
+  error instanceof Error ? error.message : String(error);
+
+/**
+ * Reports the first line each version differs at, since a whole diff of a
+ * long file says less than where to start looking.
+ * @param {string} ours What is in the repository.
+ * @param {string} theirs What upstream serves.
+ * @returns {string} A description of the first difference.
+ */
+function firstDifference(ours: string, theirs: string) {
+  const a = ours.split('\n');
+  const b = theirs.split('\n');
+
+  for (let index = 0; index < Math.max(a.length, b.length); index += 1) {
+    if (a[index] !== b[index]) {
+      return [
+        `first differs at line ${index + 1}:`,
+        `  ours:     ${a[index] ?? '(end of file)'}`,
+        `  upstream: ${b[index] ?? '(end of file)'}`,
+      ].join('\n');
+    }
+  }
+
+  return 'the files differ in how they end';
+}
+
+const drifted: string[] = [];
+const unchecked: string[] = [];
+
+for (const { file, upstream, project } of VENDORED) {
+  let ours: string;
+  let theirs: string;
+
+  try {
+    ours = await readFile(file, 'utf8');
+  } catch (error) {
+    unchecked.push(`\`${file}\` could not be read: ${reasonOf(error)}`);
+    continue;
+  }
+
+  try {
+    const response = await fetch(upstream, {
+      signal: AbortSignal.timeout(TIMEOUT),
+    });
+
+    if (!response.ok) throw new Error(`HTTP ${response.status}`);
+
+    theirs = await response.text();
+  } catch (error) {
+    unchecked.push(`\`${file}\`: ${upstream} — ${reasonOf(error)}`);
+    continue;
+  }
+
+  if (ours === theirs) {
+    console.log(`\`${file}\` matches ${upstream}`);
+    continue;
+  }
+
+  drifted.push(
+    [
+      `### \`${file}\``,
+      '',
+      `Upstream: ${upstream}`,
+      `Project: ${project}`,
+      '',
+      `Ours is ${ours.length} bytes, upstream is ${theirs.length}.`,
+      '',
+      '```text',
+      firstDifference(ours, theirs),
+      '```',
+      '',
+      'To take what upstream serves:',
+      '',
+      '```bash',
+      `curl -fsSL ${upstream} -o ${file}`,
+      '```',
+    ].join('\n')
+  );
+}
+
+// Everything goes to stdout, including what went wrong: whatever runs this
+// keeps only that, and a reason written anywhere else is a reason lost.
+if (unchecked.length > 0) {
+  console.log('');
+  console.log('Could not be compared:');
+  for (const problem of unchecked) console.log(`- ${problem}`);
+}
+
+if (drifted.length > 0) {
+  console.log('');
+  console.log(drifted.join('\n\n'));
+}
+
+process.exitCode =
+  MATCHED |
+  (drifted.length > 0 ? DRIFTED : MATCHED) |
+  (unchecked.length > 0 ? UNCHECKED : MATCHED);

package-scripts.yml

@@ -21,6 +21,9 @@ scripts:
     # Outside verify/ on purpose: it needs a pull request in the environment,
     # and verify.all runs everything in that directory.
     pullRequest: node build/tasks/verify-pull-request.mts
+    # Outside it for the same reason: this one reaches the network, and an
+    # upstream that is slow or moved would fail unrelated changes.
+    vendored: node build/tasks/check-vendored.mts
     svg: node build/tasks/verify/verify-svg.mts
     toml: node build/tasks/verify/verify-toml.mts
     ts: node build/tasks/verify/verify-ts.mts