feat(worker-bundler): add typescript support for worker-bundler #1277
feat(worker-bundler): add typescript support for worker-bundler #1277threepointone merged 13 commits intomainfrom
Conversation
🦋 Changeset detectedLatest commit: 7d5d661 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
Add DurableObjectKVFileSystem write overlay that buffers writes in memory and only persists to KV on flush(), avoiding expensive per-write I/O. Reads fall back through the overlay to KV so callers see their own writes immediately. Switch read() to return string|null instead of empty string so callers can distinguish missing files from empty ones without a separate exists() check. Add vitest tests for both InMemoryFileSystem and DurableObjectKVFileSystem using runInDurableObject for real KV storage.
…tion Replace direct Record<string,string> access throughout bundler.ts, resolver.ts, transformer.ts, config.ts, utils.ts, and app.ts with FileSystem.read() / FileSystem.write() calls. Both createWorker and createApp now accept a plain Files object or any FileSystem implementation for their 'files' option — plain objects are automatically wrapped in InMemoryFileSystem. Removes the stale 'files = installResult.files' pattern from app.ts (the installer already mutates the provided FileSystem in-place via write()). Add e2e tests covering the full bundle+install pipeline driven by both InMemoryFileSystem and DurableObjectKVFileSystem, asserting that installed node_modules are readable from the filesystem after createWorker returns and that flushing a DurableObjectKVFileSystem persists them to KV.
…talled packages Export installDependencies, hasDependencies, and InstallResult so callers can pre-warm a FileSystem with npm packages independently of createWorker/createApp. Add skip-if-already-exists guard in installPackage: if node_modules/<name>/ package.json is already present in the FileSystem, the package is skipped without hitting the network. This makes the internal installDependencies call inside createWorker a no-op for packages that were pre-installed. Add tests covering standalone install, the skip behavior using a pre-seeded filesystem, and the full pre-install-then-createWorker pipeline — including a vi.spyOn(globalThis, 'fetch') assertion that proves no network request is made during the second pass.
…ctRawFileSystem Extract a generic OverlayFileSystem (internal, not exported) that buffers writes in memory over any FileSystem inner. flush() drains the buffer via inner.write() then inner.flush(), keeping the overlay path-agnostic with no prefix awareness. Add DurableObjectRawFileSystem (exported) — a thin FileSystem backed directly by Durable Object KV with no buffering. Every write is committed synchronously and flush() is a no-op. Useful for read-heavy access or when writing a small number of files where per-write durability is preferred over batching. Retool DurableObjectKVFileSystem to compose OverlayFileSystem(DurableObjectRawFileSystem) rather than reimplementing the overlay inline. Public API is unchanged. Add DurableObjectRawFileSystem to the package exports and add a test suite covering direct write persistence, read-back, no-op flush, and custom prefix.
…ementations Add list(prefix?: string): string[] to the FileSystem interface. All implementations return logical paths (without any storage prefix): - InMemoryFileSystem: filters the backing Map's keys by prefix - DurableObjectRawFileSystem: iterates kv.list() (Iterable<[key, value]>) and strips the storage prefix from each key for consistency - OverlayFileSystem: unions inner.list() with matching overlay keys via a Set, so paths present in both sources appear exactly once - DurableObjectKVFileSystem: delegates to its inner OverlayFileSystem
`createTypescriptLanguageService` wraps a `FileSystem` in a `TypescriptFileSystem` that mirrors every write and delete into an underlying virtual TypeScript environment. Diagnostics returned by the language service always reflect the current state of the filesystem — an edit that fixes a type error immediately clears `getSemanticDiagnostics`. TypeScript is pre-bundled as a browser-safe artifact so it runs inside the Workers runtime without Node.js APIs. Lib declarations are fetched from the TypeScript npm tarball at runtime. Exposed under a separate `./typescript` subpath to keep the TypeScript bundle out of the main import path.
Add a _pendingEnqueueCount to track requests that have passed the concurrency decision but are not yet enqueued (bridging the gap caused by awaiting persist/broadcast). Increment before the async persist step and decrement right before entering the queued turn, and include this pending count in _getSubmitConcurrencyDecision so subsequent submits don't misread queuedCount() as zero. Also reset the counter when resetting turn state. This prevents a race where overlapping submits skip proper concurrency handling.
|
working on this |
Add createFileSystemSnapshot to snapshot async iterables into an InMemoryFileSystem so async backends (e.g. Workspaces, DO storage) can be used with the bundler and TypeScript language service. Export the helper from the package entry, add README documentation for the FileSystem APIs, pre-installing dependencies, and language-service usage. Update TypescriptFileSystem.write to create a TS source file when it didn't previously exist so the in-process language service sees newly written files. Add tests for createFileSystemSnapshot and the TypeScript language-service write behavior. Also update THIRD_PARTY_LICENSES and refresh package-lock.json metadata.
Update package docs to reflect a third concrete filesystem implementation: DurableObjectRawFileSystem — a thin DO KV-backed filesystem that commits every write synchronously for per-write durability. Add documentation for createFileSystemSnapshot which builds an InMemoryFileSystem from sync or async iterables of [path, content], enabling bridging async storage backends to the synchronous FileSystem interface. Also clarify an internal comment in file-system.ts to state the write-overlay is not re-exported from the package entry point.
9c53d77 to
620442d
Compare
|
Made a few changes on top of this: Bug fix: New utility: Docs: Updated README with sections on the Cleanup: Fixed |
Wrap chat message broadcast, persistence and queued-message merge in a try/finally so _pendingEnqueueCount is always decremented even if persistence or merge throws. This prevents leaking the pending enqueue counter on errors while preserving existing broadcast/persist/merge behavior before running the exclusive chat turn.
Replace the direct -- decrement with Math.max(0, _pendingEnqueueCount - 1) to prevent the pending enqueue counter from becoming negative. This ensures the internal pending count remains consistent during concurrent or exceptional flows.
Adds support for the TypeScript language service to
@cloudflare/worker-bundleras well a new virtual file-system abstraction for interfacing with the code for bundled workers. This is primarily to allow efficient storage of Worker code in Durable Objects.Example
Separate
./typescriptexportThe TypeScript bundle is large (~5 MB minified). Importing it unconditionally from the main entry point would penalise every caller that only needs bundling. The language service is therefore exported under its own subpath: