Skip to content

Commit 299080a

Browse files
committed
🤖 Use iterative fs.opendir for memory efficiency and early termination
Replace fs.readdir() with fs.opendir() async iterator: - More memory efficient: doesn't allocate full array for large directories - Early termination: stops reading if we collect 2x the limit (accounts for filtering) - Proper cleanup: uses finally block to ensure dirHandle.close() For huge directories (1000+ files), this prevents unnecessary memory allocation when we only need the first 64 entries. The 2x multiplier ensures we read enough entries to account for gitignore filtering and pattern matching.
1 parent a9d16dc commit 299080a

File tree

1 file changed

+21
-2
lines changed

1 file changed

+21
-2
lines changed

src/services/tools/file_list.ts

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -166,12 +166,31 @@ export function createFileListTool(config: { cwd: string }) {
166166
return { entries: [], totalCount: currentCount.value, exceeded: true };
167167
}
168168

169-
let dirents;
169+
// Use opendir for iterative reading - more memory efficient and allows early termination
170+
let dirHandle;
171+
const dirents = [];
170172
try {
171-
dirents = await fs.readdir(dir, { withFileTypes: true });
173+
dirHandle = await fs.opendir(dir);
174+
175+
// Read directory entries iteratively to avoid allocating large arrays
176+
// and to allow early termination if we reach the limit
177+
for await (const dirent of dirHandle) {
178+
dirents.push(dirent);
179+
180+
// Early termination: stop reading if we've collected enough entries
181+
// (accounts for filtering, so we read a bit more than the limit)
182+
if (dirents.length > options.maxEntries * 2) {
183+
break;
184+
}
185+
}
172186
} catch {
173187
// If we can't read the directory (permissions, etc.), skip it
174188
return { entries: [], totalCount: currentCount.value, exceeded: false };
189+
} finally {
190+
// Always close the directory handle
191+
if (dirHandle) {
192+
await dirHandle.close();
193+
}
175194
}
176195

177196
// Sort: directories first, then files, alphabetically within each group

0 commit comments

Comments
 (0)