Skip to content

Commit fa03d38

Browse files
CopilotCopilot
authored
build(bundle-size-tools): replace pako with fflate (microsoft#25924)
## Summary **Replaced `pako` with `fflate` in bundle-size-tools:** - Removed `pako` and `@types/pako` dependencies - Added `fflate` (already used in build-cli, consolidates compression libraries) - Updated `decompressStatsFile.ts` to use `gunzipSync` from fflate - **Impact**: Reduced lockfile by 8 lines, removed 2 dev/dependencies **Added dependency reduction plan document:** - Created `build-tools/plans/` folder for planning documentation - Detailed plan covering glob library consolidation opportunities - Risk assessments and test coverage analysis - Specific risk reduction steps including: - Integration test recommendations - Compatibility wrapper code for glob migration - Incremental migration approach
1 parent eae3cde commit fa03d38

File tree

5 files changed

+331
-16
lines changed

5 files changed

+331
-16
lines changed

‎build-tools/packages/bundle-size-tools/package.json‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@
4141
},
4242
"dependencies": {
4343
"azure-devops-node-api": "^11.2.0",
44+
"fflate": "^0.8.2",
4445
"jszip": "^3.10.1",
4546
"msgpack-lite": "^0.1.26",
46-
"pako": "^2.1.0",
4747
"typescript": "~5.4.5",
4848
"webpack": "^5.103.0"
4949
},
@@ -55,7 +55,6 @@
5555
"@microsoft/api-extractor": "^7.55.1",
5656
"@types/msgpack-lite": "^0.1.12",
5757
"@types/node": "^22.19.1",
58-
"@types/pako": "^2.0.4",
5958
"copyfiles": "^2.4.1",
6059
"eslint": "~8.57.0",
6160
"rimraf": "^6.1.2"

‎build-tools/packages/bundle-size-tools/src/utilities/decompressStatsFile.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,17 @@
33
* Licensed under the MIT License.
44
*/
55

6+
import { gunzipSync } from "fflate";
67
import { decode } from "msgpack-lite";
7-
import { inflate } from "pako";
88
import type { StatsCompilation } from "webpack";
99

1010
/**
1111
* To save storage space, we store stats files as gzipped mspack files. This method takes
1212
* in a compressed file path and outputs the webpack stats object.
1313
*/
1414
export function decompressStatsFile(buffer: Buffer): StatsCompilation {
15-
// Inflate the gzipped data to get the mspack data
16-
const mspackData = inflate(buffer);
15+
// Decompress the gzipped data to get the msgpack data
16+
const mspackData = gunzipSync(buffer);
1717

1818
return decode(mspackData);
1919
}

‎build-tools/plans/README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Plans
2+
3+
This folder contains planning documents for build-tools development and maintenance.
4+
5+
## Documents
6+
7+
| Document | Description |
8+
|----------|-------------|
9+
| [dependency-reduction.md](./dependency-reduction.md) | Plan for reducing dependencies in the build-tools workspace |

‎build-tools/plans/dependency-reduction.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
# Dependency Reduction Plan for build-tools Workspace
2+
3+
This document outlines opportunities to reduce dependencies in the build-tools workspace, along with risk assessments and suggested approaches to reduce risk before implementation.
4+
5+
## Current State
6+
7+
The build-tools workspace has accumulated multiple dependencies serving similar purposes, particularly around:
8+
- Globbing/path matching
9+
- Compression
10+
- Command execution
11+
- File system operations
12+
13+
## Test Coverage Assessment
14+
15+
Understanding test coverage is critical for assessing risk:
16+
17+
| Package | Test Files | Coverage Level | Risk for Changes |
18+
|---------|------------|----------------|------------------|
19+
| version-tools | 5 | Good | Low |
20+
| build-infrastructure | 6 | Good | Low |
21+
| build-cli | 20+ | Moderate | Medium |
22+
| build-tools | 2 | Low | High |
23+
| bundle-size-tools | 0 | None | Very High |
24+
25+
## Completed Changes
26+
27+
### ✅ Compression Library Consolidation (Partial)
28+
29+
**Status:** Completed
30+
31+
**Change:** Replaced `pako` with `fflate` in bundle-size-tools
32+
33+
**Impact:**
34+
- Removed `pako` and `@types/pako` dependencies
35+
- `fflate` was already in use in build-cli
36+
- Lockfile reduced by 8 lines
37+
38+
---
39+
40+
## Planned Opportunities
41+
42+
### 1. Globbing Library Consolidation
43+
44+
**Priority:** Medium
45+
**Risk Level:** Medium-High
46+
**Estimated Impact:** Remove 3-4 direct dependencies
47+
48+
#### Current State
49+
50+
Six libraries serve similar globbing/matching purposes:
51+
52+
| Package | Version | Used In | Purpose |
53+
|---------|---------|---------|---------|
54+
| `glob` | 7.2.3 | build-tools | `globFn()` wrapper in taskUtils.ts |
55+
| `globby` | 11.1.0 | build-cli, build-infrastructure, build-tools | File matching with gitignore support |
56+
| `multimatch` | 5.0.0 | build-tools | Biome config filtering |
57+
| `micromatch` | 4.0.8 | build-infrastructure | Package filtering |
58+
| `picomatch` | 2.3.1 | build-tools | Pattern scanning in DepCruiseTask |
59+
| `minimatch` | 7.4.6 | build-cli | Path matching in repoConfig |
60+
61+
#### Recommended Consolidation
62+
63+
Consolidate on `tinyglobby` for file system globbing and `picomatch` for pattern matching:
64+
65+
1. **Replace `glob` with `tinyglobby`** in build-tools/taskUtils.ts
66+
2. **Replace `minimatch` with `picomatch`** in build-cli/repoConfig.ts
67+
3. **Replace `multimatch` with `picomatch`** in build-tools/biomeConfig.ts
68+
69+
Note: `picomatch` is already used in `miscTasks.ts` for pattern scanning, so consolidating on it reduces total dependencies rather than adding new ones.
70+
71+
#### API Migration Patterns
72+
73+
| Original | Replacement |
74+
|----------|-------------|
75+
| `minimatch(path, pattern)` returns boolean | `picomatch(pattern)(path)` returns boolean |
76+
| `multimatch(paths, patterns)` returns filtered array | `paths.filter(picomatch(patterns))` |
77+
| `glob(pattern, options, callback)` | `tinyglobby.glob(pattern, options)` returns Promise |
78+
79+
#### Key Migration Considerations
80+
81+
Option name differences between `glob` and `tinyglobby`:
82+
83+
| glob option | tinyglobby/fast-glob equivalent | Used in |
84+
|-------------|----------------------------|---------|
85+
| `nodir: true` | `onlyFiles: true` (default) | miscTasks.ts, ts2EsmTask.ts |
86+
| `follow: true` | `followSymbolicLinks: true` | miscTasks.ts (CopyfilesTask) |
87+
| `ignore: "pattern"` | `ignore: ["pattern"]` (array required) | fluidRepoBuild.ts |
88+
| `cwd` | `cwd` (same) | prettierTask.ts, ts2EsmTask.ts |
89+
| `absolute: true` | `absolute: true` (same) | ts2EsmTask.ts |
90+
| `dot: true` | `dot: true` (same) | miscTasks.ts (CopyfilesTask) |
91+
92+
#### Risk Reduction Steps
93+
94+
1. **Add integration tests for glob-dependent functionality:**
95+
- Create test cases in `build-tools/src/test/` covering:
96+
- `CopyfilesTask` glob behavior with various options (`dot`, `follow`, `ignore`)
97+
- `TypeValidationTask` output file discovery
98+
- `GoodFence` input file enumeration
99+
- `DepCruiseTask` pattern matching
100+
- Test edge cases: dot files, symlinks, nested directories, ignore patterns
101+
102+
2. **Add tests for pattern matching:**
103+
- Test `repoConfig.ts` branch pattern matching with various branch names
104+
- Test `biomeConfig.ts` include/ignore filtering with multiple patterns
105+
106+
3. **Create a compatibility wrapper:**
107+
```typescript
108+
// Temporary adapter that accepts old glob options and converts to tinyglobby
109+
export function globFn(pattern: string, options: GlobCompatOptions = {}): Promise<string[]> {
110+
return glob(pattern, {
111+
onlyFiles: options.nodir ?? true,
112+
followSymbolicLinks: options.follow ?? false,
113+
ignore: Array.isArray(options.ignore) ? options.ignore : options.ignore ? [options.ignore] : [],
114+
cwd: options.cwd,
115+
absolute: options.absolute,
116+
dot: options.dot,
117+
});
118+
}
119+
```
120+
121+
4. **Migrate incrementally by file:**
122+
- Start with files that have test coverage
123+
- Manually verify behavior for untested files
124+
125+
5. **Migration code for minimatch → picomatch** (in repoConfig.ts):
126+
```typescript
127+
// Before
128+
import { minimatch } from "minimatch";
129+
if (minimatch(branch, branchPattern) === true) { ... }
130+
131+
// After
132+
import picomatch from "picomatch";
133+
const isMatch = picomatch(branchPattern);
134+
if (isMatch(branch)) { ... }
135+
```
136+
137+
6. **Migration code for multimatch → picomatch** (in biomeConfig.ts):
138+
```typescript
139+
// Before
140+
import multimatch from "multimatch";
141+
const includedPaths = multimatch([...gitLsFiles], prefixedIncludes);
142+
143+
// After
144+
import picomatch from "picomatch";
145+
const isMatch = picomatch(prefixedIncludes);
146+
const includedPaths = [...gitLsFiles].filter(isMatch);
147+
```
148+
149+
7. **Gitignore support with tinyglobby:**
150+
151+
Unlike `globby`, `tinyglobby` does not have built-in gitignore support. If gitignore filtering is needed, use this pattern (from [e18e.dev](https://e18e.dev/docs/replacements/globby.html)):
152+
153+
```typescript
154+
import { execSync } from 'node:child_process';
155+
import { escapePath, glob } from 'tinyglobby';
156+
157+
async function globWithGitignore(patterns, options = {}) {
158+
const { cwd = process.cwd(), ...restOptions } = options;
159+
160+
try {
161+
const gitIgnored = execSync(
162+
'git ls-files --others --ignored --exclude-standard --directory',
163+
{ cwd, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }
164+
)
165+
.split('\n')
166+
.filter(Boolean)
167+
.map(p => escapePath(p));
168+
169+
return glob(patterns, {
170+
...restOptions,
171+
cwd,
172+
ignore: [...(restOptions.ignore || []), ...gitIgnored]
173+
});
174+
} catch {
175+
return glob(patterns, options);
176+
}
177+
}
178+
```
179+
180+
Note: The current `globFn` usage in build-tools does not appear to rely on gitignore support, so this may not be needed for the initial migration.
181+
182+
---
183+
184+
### 2. Additional Compression Consolidation
185+
186+
**Priority:** Low
187+
**Risk Level:** Low-Medium
188+
**Estimated Impact:** Remove 1 dependency
189+
190+
#### Current State
191+
192+
| Package | Used In | Purpose |
193+
|---------|---------|---------|
194+
| `fflate` | build-cli, bundle-size-tools | Gzip decompression |
195+
| `jszip` | build-cli, bundle-size-tools | ZIP file handling |
196+
197+
#### Recommendation
198+
199+
Consider replacing `jszip` with `fflate` for ZIP handling:
200+
- `fflate` has ZIP support via `unzipSync`/`zipSync`
201+
- However, `jszip` provides streaming and more features
202+
203+
#### Risk Reduction Steps
204+
205+
1. Audit all `jszip` usage patterns
206+
2. Verify `fflate` can handle all use cases
207+
3. If not, keep both (different purposes)
208+
209+
---
210+
211+
### 3. Command Execution (execa)
212+
213+
**Priority:** Deferred
214+
**Risk Level:** High
215+
**Estimated Impact:** Minimal (transitive dependency reduction only)
216+
217+
#### Current State
218+
219+
`execa` v5.1.1 is used in 11+ files across:
220+
- build-infrastructure (2 files)
221+
- build-cli (9+ files)
222+
223+
#### Usage Patterns
224+
225+
```typescript
226+
// Async command execution
227+
await execa('npm', ['publish', ...args], { cwd, stdio })
228+
229+
// Sync command execution
230+
execa.sync('git', ['rev-parse', '--show-toplevel'], { cwd, stdio })
231+
```
232+
233+
#### Recommendation
234+
235+
**Keep `execa` for now.** Reasons:
236+
- Cross-platform compatibility is essential
237+
- Extensively tested in the npm ecosystem
238+
- Native `child_process` alternatives require significant boilerplate
239+
- Risk outweighs minimal benefit
240+
241+
#### Future Consideration
242+
243+
When upgrading to execa v6+, evaluate `tinyexec` as a lighter alternative for simple use cases.
244+
245+
---
246+
247+
### 4. File System Utilities (fs-extra)
248+
249+
**Priority:** Low
250+
**Risk Level:** Low
251+
**Estimated Impact:** Minimal
252+
253+
#### Current State
254+
255+
`fs-extra` used in 7+ files for:
256+
- `readJsonSync`, `writeJson`, `writeJsonSync`
257+
- `mkdirpSync`
258+
- `copySync`
259+
260+
#### Recommendation
261+
262+
**Keep `fs-extra`.** Reasons:
263+
- Well-maintained with minimal footprint
264+
- Node.js alternatives require more code
265+
- Not a significant source of bloat
266+
267+
#### Gradual Migration Path (Optional)
268+
269+
If desired, these can be replaced with native Node.js equivalents:
270+
271+
| fs-extra | Native equivalent |
272+
|----------|-------------------|
273+
| `mkdirpSync` | `fs.mkdirSync(path, { recursive: true })` |
274+
| `readJsonSync` | `JSON.parse(fs.readFileSync(path, 'utf8'))` |
275+
| `writeJsonSync` | `fs.writeFileSync(path, JSON.stringify(data, null, 2))` |
276+
277+
---
278+
279+
## Implementation Roadmap
280+
281+
### Phase 1: Add Test Coverage (Prerequisite)
282+
- [ ] Add glob-related tests to build-tools
283+
- [ ] Add basic integration tests for bundle-size-tools decompression
284+
285+
### Phase 2: Low-Risk Changes
286+
- [x] Replace `pako` with `fflate` in bundle-size-tools ✅
287+
288+
### Phase 3: Glob Consolidation
289+
- [ ] Create compatibility wrapper for `globFn`
290+
- [ ] Migrate build-tools from `glob` to `tinyglobby`
291+
- [ ] Replace `minimatch` with `picomatch` in build-cli
292+
- [ ] Replace `multimatch` with `picomatch` in build-tools
293+
- [ ] Remove `glob`, `minimatch`, `multimatch` dependencies
294+
295+
### Phase 4: Evaluate and Defer
296+
- [ ] Re-evaluate `jszip` vs `fflate` for ZIP handling
297+
- [ ] Monitor for `execa` alternatives during future upgrades
298+
299+
---
300+
301+
## Success Metrics
302+
303+
- Reduce direct dependency count by 4-6
304+
- Maintain lockfile line count reduction
305+
- Zero regressions in build functionality
306+
- All existing tests continue to pass
307+
308+
---
309+
310+
## References
311+
312+
- [tinyglobby documentation](https://github.com/SuperchupuDev/tinyglobby)
313+
- [picomatch documentation](https://github.com/micromatch/picomatch)
314+
- [fast-glob options](https://github.com/mrmlnc/fast-glob#options-3)
315+
- [fflate documentation](https://github.com/101arrowz/fflate)

0 commit comments

Comments
 (0)