Add fswatch package, ported from @parcel/watcher with heavy modification

[jakebailey]

For #3611

In the old compiler, our file watching was provided by Node's fs.watch function wrapped with heuristics and workarounds to make it function for our needs. But, now we're in Go, and we don't have Node or libuv to give these APIs. Go does not have file watching built in, so we need to do something ourselves.

For the editor, we can currently rely on LSP client-side watching. In VS Code this is provided by @parcel/watcher, which works very well and has fewer warts than fs.watch. But, @parcel/watcher is a N-API module written in C++. we can't cleanly depend on it directly as that would involve cgo and therefore a truly painful building and shipping setup, on top of having to adapt the library for use outside of Node.

So instead, I ported @parcel/watcher to Go, with heavy modifications to better fit our own needs, retaining attribution/license. This is possible since of course C++ and Go operate at the same level (the bottom!). We can make our own syscalls and futz around with pointers just fine.

---

Many things are unchanged, but not everything is. The differences are outlined in CHANGES.md, including some bugs that were found along the way I intend to upstream (and so eventually make its way to @parcel/watcher users, e.g. VS Code proper).

Some notable differences (see CHANGES.md for a deeper description):

• The only events are Update and Delete. TS does not care about Create, since us not knowing about a file and seeing an event for it is equivalent. This makes the watcher faster as it does not need to keep an in-memory copy of the entire FS, and simplifies the code greatly.
• The watcher can also watch dirs non-recursively, and watch files.
• Globbing has been removed; there's just an ignore func for that. TypeScript never actually filtered at the watcher level anyway.
• No watchman support. Maybe we could pull that in some day; I had a working version but dropped it to scope this down.
• We don't attempt to watch for or filter attribute changes. The watchers don't explicitly ask for these events, though on some platforms (like macOS), you still have to in order to detect file truncation. In those cases, a chmod might appear as an Update event, but that's in practice not different from a user saving a file with identical contents to what was already on disk.
• On Linux, the package makes use of fanotify over inotify, if available, which allows for virtually unlimited file watches. There is actually an open PR for this in @parcel/watcher, but I only compared after the fact. CHANGES.md includes a handful of bugs in the unmerged upstream PR (to comment on there later 😄).

---

Now, one gotcha is fsevents on macOS. This watcher is generally seen to be better than kqueue, but it's not a simple syscall API; it's a dynamically loaded C library.

Calling into C from Go on macOS is a well understood problem. Go provides //go:cgo_import_dynamic to load libraries, and the runtime, syscall and x/sys/unix packages, etc, provide everything to use them, as they are preferred, or often required to do anything.

The real problem is that fsevents uses callbacks to send you events, whereas the other platforms' watchers do things like creating file descriptors to read from.

This poses a problem for our "no cgo" goal. We don't want to actually write C, since as I said before, we'd then need a C compiler (for something other than race mode), figure out multiplatform, and get it shipped. That's arguably better than dynamically linking @parcel/watcher in, but still problematic.

I could have used purego for this, but that seemed too risky; I want us to be forward compatible with newer versions of Go, and purego works effectively by copy/pasting the cgo package and then initializing it. I don't know how forward compatible that is.

Thankfully, there's another trick we can use: assembly.

Instead of writing C code, letting cgo compile that, depend on the C toolchain, we can instead write amd64 and arm64 assembly with the C calling conventions, and then give that to fsevents. But that's not enough to actually do anything interesting with; we still have to send the event over to Go.

To do that, we can (sorta) steal the same thing I did for the sync RPC API and use pipes! When setting up the callback, Go makes a pipe, then spawns a goroutine that waits for that pipe (which, doesn't consume any threads thanks to the netpoller).

When an event happens, the assembly copies the data into a fresh C-allocated buffer, places the event data into the Go callback struct, then wakes the receiving goroutine by writing to a pipe (returning immediately). Go then does the callback, pulls out the data, frees it, and does its callback, then loops back around. From fsevents' perspective, it called a regular C function. From Go's perspective, it just read data from a pipe. That is all the synchronization needed as the pipe write from C to Go synchronizes, and the data only flows one way.

This means we don't actually need C -> Go calling at all, only Go -> C, and everything works! Neat!

This wasn't trivial to get right, however. Thankfully copilot knows better than me how to write amd64/arm64 assembly. (I'm more of a MIPS guy. 😉)

---

Long term, we're going to have to track changes and synchronize fixes. Not everything will be applicable due to the simplifications, but some things are certainly going to be shared, and I'm sure the methods I used here will be helpful to someone else trying to tackle problems like this.

I have also gone significant lengths to make this not flaky, and even work for the BSDs. Everything appears stable, and a regular package test on my machine takes about 4 seconds despite having waits and so on.

---

There are still some things I want to do, but not now:

• The paths are filepath paths i.e. OS specific ones. TS does not work with these at all. Could be worth changing over, but likely it'll get too annoying to do conversions everywhere.
• Polling fallback, somehow
• Illumos/Solaris support? 😄 (AIX???)

[Copilot]

Pull request overview

Introduces a new internal/fswatch package: a pure-Go, cgo-free filesystem watcher ported (with heavy modification and bugfixes) from @parcel/watcher. It provides a unified Watcher API with per-platform backends (inotify, fanotify, kqueue, FSEvents, Windows ReadDirectoryChangesW), debounced batched event delivery, and a per-platform walkDir implementation. The macOS FSEvents backend uses //go:cgo_import_dynamic plus hand-written amd64/arm64 assembly trampolines so the C callback can hand events to Go via a pipe without entering Go ABI from the GCD thread. Intended to be the foundation for native --watch mode (issue #3611).

Changes:

• New fswatch package with Watcher/Watch API, event coalescing (eventList), process-wide debouncer, and per-subscriber WithIgnore/WithRecursive options.
• Per-platform backends: inotify_linux.go, fanotify_linux.go (default on kernels ≥ 5.13), kqueue.go, windows.go, plus fsevents_darwin*.go with assembly trampolines and a static register-safety test.
• Per-platform walkDir (native getdents/FindFirstFile fast paths plus portable fallback) with shared tests, NFC path canonicalization on darwin, README/CHANGES docs, and cgmanifest.json recording upstream attribution.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
internal/fswatch/watcher.go	Public Watcher/Watch/WatchOption API, dirWatch lifecycle, base watcherImpl
internal/fswatch/event.go	Event kinds and coalescing primitive eventList
internal/fswatch/eventlist_test.go	Unit tests for coalescing/drain semantics
internal/fswatch/debounce.go	Process-wide debounced callback dispatcher
internal/fswatch/inotify_linux.go	Inotify backend
internal/fswatch/fanotify_linux.go	Fanotify (FID-based) backend with FAN_RENAME probe + fallback
internal/fswatch/fanotify_linux_test.go	Fanotify-specific tests including fallback path
internal/fswatch/kqueue.go	kqueue backend (darwin + BSDs)
internal/fswatch/windows.go	ReadDirectoryChangesW backend
internal/fswatch/fsevents_darwin.go	FSEvents event classification and stream lifecycle
internal/fswatch/fsevents_darwin_ffi.go	cgo-free CoreFoundation/CoreServices FFI plumbing
internal/fswatch/fsevents_darwin_ffi.s	Shared trampolines (CoreFoundation, libdispatch, FSEvents)
internal/fswatch/fsevents_darwin_ffi_amd64.s	amd64 callback assembly + FSEventStreamCreate latency shuffle
internal/fswatch/fsevents_darwin_ffi_arm64.s	arm64 callback assembly + FSEventStreamCreate latency shuffle
internal/fswatch/fsevents_darwin_ffi_arm64_test.go	Static check that callback asm clobbers only safe registers
internal/fswatch/fsevents_darwin_nfd_test.go	NFC normalization tests for darwin paths
internal/fswatch/canonicalize_darwin.go / canonicalize_other.go	NFC canonicalization on darwin, no-op elsewhere
internal/fswatch/walkdir.go / walkdir_unix.go / walkdir_windows.go / walkdir_other.go	Per-platform recursive directory walker + portable fallback
internal/fswatch/walkdir_dirent_*.go	Per-OS Dirent reclen/ino accessor shims
internal/fswatch/walkdir_test.go	Shared tests for walkDir/walkDirGeneric
internal/fswatch/README.md / CHANGES.md	Usage docs and upstream-divergence notes
internal/fswatch/LICENSE / cgmanifest.json	Upstream attribution

Comments suppressed due to low confidence (1)

internal/fswatch/watcher.go:320

• Same panic-on-bad-input concern as in WatchDirectory: a non-absolute path argument panics rather than returning an error. Prefer an error return for consistency with the other validation paths in this file.

	if !filepath.IsAbs(path) {
		panic("fswatch: path must be an absolute path")
	}

[jakebailey]

Oh great, I forgot to check linting when I moved the code over, sigh

[Copilot]

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated no new comments.