Skip to content

Commit daa6d9e

Browse files
KristjanESPERANTOKristjanESPERANTO
committed
feat: add rule detecting modules that import third-party dependencies without declaring them
1 parent 3c86664 commit daa6d9e

File tree

8 files changed

+404
-34
lines changed

8 files changed

+404
-34
lines changed

‎cspell.config.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,15 @@
1717
"depandabot",
1818
"dependencie",
1919
"diffing",
20+
"envsub",
21+
"feedme",
2022
"Fenner",
2123
"fontawesome",
2224
"Gitea",
2325
"Goldens",
2426
"hoster",
27+
"ical",
28+
"ipfilter",
2529
"Itay",
2630
"Kristjan",
2731
"magicmirror",
@@ -41,11 +45,13 @@
4145
"refspecs",
4246
"rollup",
4347
"smarthome",
48+
"suncalc",
4449
"testconfig",
4550
"trumpetx",
4651
"Updat",
4752
"venv",
4853
"virtualenv",
54+
"weathericons",
4955
"workstream",
5056
"workstreams",
5157
"Workstreams"

‎docs/pipeline-refactor-roadmap.md‎

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -46,21 +46,21 @@ This document captures the long-term improvements we want to implement in the mo
4646

4747
### 4. Checks & Developer Experience
4848

49-
| Task | Description | Dependencies | Effort |
50-
| ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------ |
51-
| P4.1 | Split checks into a registry with metadata (category, severity, auto-fixable) ✅ Completed Oct 2025 | P2.3 | M |
52-
| P4.2 | Add configuration file to toggle check groups (`fast`, `deep`, optional ESLint/ncu`) ✅ Completed Oct 2025 | P4.1 | S |
53-
| P4.3 | Create sample dataset + regression tests for check outputs (golden files), reusing the curated fixtures where possible ✅ Completed Oct 2025 | P4.1 | M |
54-
| P4.4 | Provide CLI progress UI and Markdown summary per run ✅ Completed Oct 2025 | P1.2 | S |
55-
| P4.5 | Add rule detecting modules that rely on MagicMirror core dependencies without declaring them ([#78](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/78)) | P4.1 | M |
56-
| P4.6 | Check README install/update sections for copyable fenced command blocks ([#54](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/54)) | P4.1 | S |
57-
| P4.7 | Recommend `npm ci --omit=dev` when modules expose devDependencies in instructions ([#53](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/53)) | P4.1 | S |
58-
| P4.8 | Flag modules with multi-year inactivity that are not marked `outdated` and nudge maintainers to review status | P4.1 | M |
59-
| P4.9 | Inspect Dependabot configs for schedule scope (quarterly cadence, production-only) and suggest adjustments | P4.1 | M |
60-
| P4.10 | Evaluate migrating the `ntl` task menu into a `pipeline` subcommand (interactive launcher built on the orchestrator CLI) _(low priority)_ | P1.2 | S |
61-
| P4.11 | Extend the rule registry to cover every pipeline check stage (legacy JS script + future additions) ✅ Completed Oct 2025 | P4.1 | L |
62-
| P4.R1 | Audit every rule in the registry for relevance and clarity | P4.11 | S |
63-
| P4.R2 | Audit every recommendation in the registry for relevance and consistency | P4.11 | S |
49+
| Task | Description | Dependencies | Effort |
50+
| ----- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ------ |
51+
| P4.1 | Split checks into a registry with metadata (category, severity, auto-fixable) ✅ Completed Oct 2025 | P2.3 | M |
52+
| P4.2 | Add configuration file to toggle check groups (`fast`, `deep`, optional ESLint/ncu`) ✅ Completed Oct 2025 | P4.1 | S |
53+
| P4.3 | Create sample dataset + regression tests for check outputs (golden files), reusing the curated fixtures where possible ✅ Completed Oct 2025 | P4.1 | M |
54+
| P4.4 | Provide CLI progress UI and Markdown summary per run ✅ Completed Oct 2025 | P1.2 | S |
55+
| P4.5 | Add rule detecting modules that import third-party dependencies without declaring them ([#78](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/78)) ✅ Completed Oct 2025 | P4.1 | M |
56+
| P4.6 | Check README install/update sections for copyable fenced command blocks ([#54](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/54)) | P4.1 | S |
57+
| P4.7 | Recommend `npm ci --omit=dev` when modules expose devDependencies in instructions ([#53](https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules/issues/53)) | P4.1 | S |
58+
| P4.8 | Flag modules with multi-year inactivity that are not marked `outdated` and nudge maintainers to review status | P4.1 | M |
59+
| P4.9 | Inspect Dependabot configs for schedule scope (quarterly cadence, production-only) and suggest adjustments | P4.1 | M |
60+
| P4.10 | Evaluate migrating the `ntl` task menu into a `pipeline` subcommand (interactive launcher built on the orchestrator CLI) _(low priority)_ | P1.2 | S |
61+
| P4.11 | Extend the rule registry to cover every pipeline check stage (legacy JS script + future additions) ✅ Completed Oct 2025 | P4.1 | L |
62+
| P4.R1 | Audit every rule in the registry for relevance and clarity | P4.11 | S |
63+
| P4.R2 | Audit every recommendation in the registry for relevance and consistency | P4.11 | S |
6464

6565
### 5. Documentation & Collaboration
6666

@@ -99,10 +99,10 @@ Routine reminders for keeping the written guidance in sync with the code:
9999

100100
Immediate action items:
101101

102-
1. Add the dependency-declaration rule for core MagicMirror usage (P4.5).
103-
2. Audit README install/update sections for copyable fenced command blocks (P4.6).
104-
3. Recommend `npm ci --omit=dev` when modules list devDependencies in instructions (P4.7).
105-
4. Flag modules with multi-year inactivity that are not marked `outdated` and nudge maintainers to review status (P4.8).
102+
1. Audit README install/update sections for copyable fenced command blocks (P4.6).
103+
2. Recommend `npm ci --omit=dev` when modules list devDependencies in instructions (P4.7).
104+
3. Flag modules with multi-year inactivity that are not marked `outdated` and nudge maintainers to review status (P4.8).
105+
4. Inspect Dependabot configs for schedule scope (quarterly cadence, production-only) and suggest adjustments (P4.9).
106106

107107
---
108108

‎docs/pipeline/check-modules-reference.md‎

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Check Modules Reference
22

3-
_Last updated: October 3, 2025_
3+
_Last updated: October 4, 2025_
44

55
This page consolidates the material that previously lived in the P2.3 rollout documents. It should stay up to date as we evolve Stage 5 (`scripts/check-modules/index.ts`), the comparison harness, and the curated fixture set.
66

@@ -9,6 +9,7 @@ This page consolidates the material that previously lived in the P2.3 rollout do
99
- ✅ TypeScript implementation is the default Stage 5 runner.
1010
- ✅ Comparison harness (`npm run checkModules:compare`) can execute multiple commands, capture artifacts, and (when two runs complete) produce diffs for analysis.
1111
- ✅ CLI progress indicator renders live module throughput and emits a per-run Markdown summary under `.pipeline-runs/check-modules/`.
12+
- ✅ Stage 5 flags modules that import any non built-in dependency without declaring it in their own `package.json` (Node built-ins and the allowlist `express`, `node_helper`, `logger` are ignored).
1213
- 🔄 Follow-ups tracked here: extend harness diff coverage (README/HTML artifacts) and define warning/failure thresholds ahead of diff gating in CI.
1314

1415
## Check group configuration
@@ -77,16 +78,17 @@ These are the rule IDs currently implemented by the TypeScript checker. Keep thi
7778

7879
### `package.json` rules
7980

80-
| Rule ID | Pattern | Category | Notes |
81-
| ---------------------------------------- | -------------------------------------------- | -------------- | --------------------------------- |
82-
| pkg-deprecated-electron-rebuild | `"electron-rebuild"` | Deprecated | Use `@electron/rebuild`. |
83-
| pkg-deprecated-eslint-config-airbnb | `eslint-config-airbnb` | Deprecated | Seek modern configuration. |
84-
| pkg-recommend-eslint-plugin-json | `"eslint-plugin-json"`/`eslint-plugin-jsonc` | Recommendation | Suggest `@eslint/json`. |
85-
| pkg-deprecated-grunt | `"grunt"` | Deprecated | Tool largely unmaintained. |
86-
| pkg-outdated-husky-install | `husky install` | Outdated | Husky v9 no longer needs it. |
87-
| pkg-recommend-needle | `"needle"` | Recommendation | Suggest fetch. |
88-
| pkg-deprecated-rollup-banner | `rollup-plugin-banner` | Deprecated | Replace with built-in banner. |
89-
| pkg-deprecated-stylelint-config-prettier | `stylelint-config-prettier` | Deprecated | Remove in newer Stylelint setups. |
81+
| Rule ID | Pattern | Category | Notes |
82+
| ---------------------------------------- | -------------------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
83+
| pkg-deprecated-electron-rebuild | `"electron-rebuild"` | Deprecated | Use `@electron/rebuild`. |
84+
| pkg-deprecated-eslint-config-airbnb | `eslint-config-airbnb` | Deprecated | Seek modern configuration. |
85+
| pkg-recommend-eslint-plugin-json | `"eslint-plugin-json"`/`eslint-plugin-jsonc` | Recommendation | Suggest `@eslint/json`. |
86+
| pkg-deprecated-grunt | `"grunt"` | Deprecated | Tool largely unmaintained. |
87+
| pkg-outdated-husky-install | `husky install` | Outdated | Husky v9 no longer needs it. |
88+
| pkg-recommend-needle | `"needle"` | Recommendation | Suggest fetch. |
89+
| pkg-missing-dependency | _n/a (detected via usage scan)_ | Recommendation | Flags modules that import third-party packages without declaring them in `package.json` (built-ins and the default allowlist `express`, `node_helper`, `logger` are ignored). |
90+
| pkg-deprecated-rollup-banner | `rollup-plugin-banner` | Deprecated | Replace with built-in banner. |
91+
| pkg-deprecated-stylelint-config-prettier | `stylelint-config-prettier` | Deprecated | Remove in newer Stylelint setups. |
9092

9193
### `package-lock.json` rules
9294

‎scripts/check-modules/__tests__/dependency-usage.test.js‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
import {
2+
MISSING_DEPENDENCY_RULE_ID,
3+
detectUsedDependencies,
4+
extractDeclaredDependencyNames,
5+
findMissingDependencies,
6+
shouldAnalyzeFileForDependencyUsage
7+
} from "../dependency-usage.js";
8+
9+
import assert from "node:assert/strict";
10+
import test from "node:test";
11+
12+
function toSortedArray (iterable) {
13+
return Array.from(iterable).sort((a, b) => a.localeCompare(b));
14+
}
15+
16+
test("exports the shared rule id", () => {
17+
assert.equal(MISSING_DEPENDENCY_RULE_ID, "pkg-missing-dependency");
18+
});
19+
20+
test("detects third-party dependency imports via require and import", () => {
21+
const sample = [
22+
"import dayjs from 'dayjs';",
23+
"const tz = require('moment-timezone/builds');",
24+
"await import('@scope/example/utils');",
25+
"const fetchImpl = await import('node-fetch');",
26+
"const express = require('express');"
27+
].join("\n");
28+
29+
const detected = detectUsedDependencies(sample);
30+
assert.deepEqual(
31+
toSortedArray(detected),
32+
["@scope/example", "dayjs", "moment-timezone", "node-fetch"]
33+
);
34+
});
35+
36+
test("ignores relative and built-in dependencies", () => {
37+
const sample = [
38+
"const fs = require('fs');",
39+
"import path from 'node:path';",
40+
"const http = await import('node:http');",
41+
"import util from './util.js';",
42+
"const styles = require('../styles.css');",
43+
"const helper = require('node_helper');",
44+
"const logger = require('logger');"
45+
].join("\n");
46+
47+
const detected = detectUsedDependencies(sample);
48+
assert.deepEqual(Array.from(detected), []);
49+
});
50+
51+
test("ignores unrelated content", () => {
52+
const detected = detectUsedDependencies("const value = 42;");
53+
assert.deepEqual(Array.from(detected), []);
54+
});
55+
56+
test("only analyzes supported source files", () => {
57+
assert.ok(shouldAnalyzeFileForDependencyUsage("node_helper.js"));
58+
assert.ok(shouldAnalyzeFileForDependencyUsage("src/helpers/util.ts"));
59+
assert.ok(!shouldAnalyzeFileForDependencyUsage("README.md"));
60+
assert.ok(!shouldAnalyzeFileForDependencyUsage("vendor/moment.min.js"));
61+
assert.ok(!shouldAnalyzeFileForDependencyUsage("examples/demo.js"));
62+
});
63+
64+
test("extracts declared dependencies across sections", () => {
65+
const declared = extractDeclaredDependencyNames({
66+
dependencies: {
67+
moment: "^2.29.4",
68+
axios: "^1.7.0"
69+
},
70+
devDependencies: {
71+
eslint: "^9.0.0"
72+
},
73+
optionalDependencies: {
74+
express: "^4.18.2"
75+
},
76+
peerDependencies: {
77+
"socket.io": "^4.7.5"
78+
}
79+
});
80+
81+
assert.deepEqual(toSortedArray(declared), ["axios", "eslint", "express", "moment", "socket.io"]);
82+
});
83+
84+
test("finds missing dependencies", () => {
85+
const declared = new Set(["moment", "express"]);
86+
const missing = findMissingDependencies({
87+
declaredDependencies: declared,
88+
usedDependencies: new Set(["moment", "moment-timezone", "express"])
89+
});
90+
91+
assert.deepEqual(missing, ["moment-timezone"]);
92+
});

0 commit comments

Comments
 (0)