Skip to content

Commit 1131350

Browse files
DIYgodclaude
andauthored
feat: deploy RSSHub to Cloudflare Workers (#20804)
* feat: deploy RSSHub to Cloudflare Workers - Create Worker entry point (lib/worker.ts) with polyfills - Implement Worker-specific app configuration (lib/app.worker.tsx) - Add automatic .worker.ts resolution plugin for cleaner config - Simplify build configuration with only 3 essential shims - Enable static asset serving via Cloudflare Static Assets feature - Support dynamic route loading with proper module aliasing 🤖 Generated with Claude Code Co-Authored-By: Claude Haiku 4.5 <[email protected]> * feat(worker): add puppeteer and request-rewriter support - Add @cloudflare/puppeteer for Browser Rendering API support - Create puppeteer.worker.ts with Cloudflare Browser binding - Add request-rewriter Worker version with static browser headers - Add vm module shim to node-module.ts for JSDOM compatibility - Configure __dirname/__filename in tsdown for CommonJS compat - Update wrangler.toml with BROWSER binding configuration - Dynamically extract namespaces from foloAnalysisTop100 for routes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix(cache): restore synchronous initialization for tests The async initialization caused cache tests to fail because the cache module wasn't ready when tests ran. Restored synchronous imports while keeping Worker-specific no-op behavior. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * test(worker): add automated Worker integration tests - Add miniflare for Worker environment simulation - Create lib/worker.worker.test.ts with integration tests - Test basic routes (/test/1, /, unknown routes) - Test RSS feed routes (hackernews, v2ex) - Test error handling for puppeteer routes without BROWSER binding - Add worker-test npm script 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * refactor(worker): use test routes for Worker integration tests - Replace third-party routes (hackernews, v2ex, weibo) with /test/* routes - Remove miniflare dependency (wrangler includes it internally) - Simplify test setup using wrangler's unstable_dev API 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix(test): make header-generator test less flaky The header-generator library may return different platform values due to internal randomness. Update test to check for valid format instead of exact value. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix(test): exclude worker tests from vitest coverage run Worker tests require the dist-worker bundle to be built first, which is not part of the regular CI test workflow. These tests should be run separately using 'pnpm worker-test'. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * test(worker): add automated Worker integration tests - Add worker-build step to CI workflow before running tests - Revert exclusion of worker tests from vitest coverage run - Worker tests now run as part of the regular test suite 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix(worker): fix routes.js alias path in worker build config Change alias key from absolute path to relative import path to match how it's imported in lib/registry.ts. This fixes the worker build failing with "Could not resolve '../assets/build/routes.js'" error. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * ci: trigger CI re-run 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix(ci): build worker routes before worker-build The worker build requires routes-worker.js which is generated by running build:routes with WORKER_BUILD=true. This was missing in CI, causing the worker build to fail with "Could not resolve routes.js". 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix(test): increase timeout for Worker integration tests Worker tests need more time in CI environments. Increase individual test timeouts from default 10s to 30s for each test case. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> --------- Co-authored-by: Claude Haiku 4.5 <[email protected]>
1 parent 472082e commit 1131350

26 files changed

+1700
-24
lines changed

.github/workflows/test.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,10 @@ jobs:
4242
run: pnpm rb && pnpx rebrowser-puppeteer browsers install chrome
4343
- name: Build routes
4444
run: pnpm build
45+
- name: Build worker routes
46+
run: WORKER_BUILD=true pnpm build:routes
47+
- name: Build worker
48+
run: pnpm worker-build
4549
- name: Test all and generate coverage
4650
run: pnpm run vitest:coverage --reporter=github-actions
4751
env:

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ node_modules
2828
tmp
2929
dist
3030
dist-lib
31+
dist-worker
32+
.wrangler
3133

3234
Session.vim
3335
combined.log

lib/app.worker.tsx

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
// Worker-specific app configuration
2+
// This is a simplified version of app-bootstrap.tsx for Cloudflare Workers
3+
// Heavy middleware and API routes are excluded
4+
5+
import { Hono } from 'hono';
6+
import { jsxRenderer } from 'hono/jsx-renderer';
7+
import { trimTrailingSlash } from 'hono/trailing-slash';
8+
9+
import { errorHandler, notFoundHandler } from '@/errors';
10+
import accessControl from '@/middleware/access-control';
11+
import debug from '@/middleware/debug';
12+
import header from '@/middleware/header';
13+
import mLogger from '@/middleware/logger';
14+
import template from '@/middleware/template';
15+
import trace from '@/middleware/trace';
16+
import registry from '@/registry';
17+
import { setBrowserBinding } from '@/utils/puppeteer';
18+
19+
// Define Worker environment bindings
20+
type Bindings = {
21+
BROWSER?: any; // Browser Rendering API binding
22+
};
23+
24+
const app = new Hono<{ Bindings: Bindings }>();
25+
26+
// Set browser binding for puppeteer
27+
app.use(async (c, next) => {
28+
if (c.env?.BROWSER) {
29+
setBrowserBinding(c.env.BROWSER);
30+
}
31+
await next();
32+
});
33+
34+
app.use(trimTrailingSlash());
35+
36+
// Cloudflare Workers handles compression at the edge, no need for compress()
37+
38+
app.use(
39+
jsxRenderer(({ children }) => <>{children}</>, {
40+
docType: '<?xml version="1.0" encoding="UTF-8"?>',
41+
stream: {},
42+
})
43+
);
44+
app.use(mLogger);
45+
app.use(trace);
46+
47+
// Heavy middleware excluded in Worker build:
48+
// - sentry: @sentry/node
49+
// - antiHotlink: cheerio
50+
// - parameter: cheerio, sanitize-html, @postlight/parser
51+
// - cache: ioredis
52+
53+
app.use(accessControl);
54+
app.use(debug);
55+
app.use(template);
56+
app.use(header);
57+
58+
app.route('/', registry);
59+
60+
// API routes not available in Worker environment
61+
62+
app.notFound(notFoundHandler);
63+
app.onError(errorHandler);
64+
65+
export default app;

lib/assets/favicon.ico

2.61 KB
Binary file not shown.

lib/registry.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import metrics from '@/routes/metrics';
1212
import robotstxt from '@/routes/robots.txt';
1313
import type { APIRoute, Namespace, Route } from '@/types';
1414
import { directoryImport } from '@/utils/directory-import';
15+
import { isWorker } from '@/utils/is-worker';
1516
import logger from '@/utils/logger';
1617

1718
const __dirname = import.meta.dirname;
@@ -260,7 +261,7 @@ if (config.debugInfo) {
260261
// Only enable tracing in debug mode
261262
app.get('/metrics', metrics);
262263
}
263-
if (!config.isPackage && !process.env.VERCEL_ENV) {
264+
if (!config.isPackage && !process.env.VERCEL_ENV && !isWorker) {
264265
app.use(
265266
'/*',
266267
serveStatic({

lib/shims/dotenv-config.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
// No-op shim for dotenv/config in Cloudflare Workers
2+
// Environment variables are set via wrangler.toml or wrangler secrets
3+
// No need to load from .env file

lib/shims/node-module.ts

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
// Shim for node:module in Cloudflare Workers
2+
// Provides a createRequire that returns pre-imported modules
3+
4+
import * as assert from 'node:assert';
5+
import * as async_hooks from 'node:async_hooks';
6+
import * as buffer from 'node:buffer';
7+
import * as child_process from 'node:child_process';
8+
import * as console_module from 'node:console';
9+
import * as constants from 'node:constants';
10+
import * as crypto from 'node:crypto';
11+
import * as diagnostics_channel from 'node:diagnostics_channel';
12+
import * as dns from 'node:dns';
13+
// For events, we need the default export (EventEmitter class) for CJS compatibility
14+
// CJS require('events') returns EventEmitter class directly
15+
import events, * as eventsNamespace from 'node:events';
16+
// Pre-import Node.js builtins that CJS modules might require
17+
import * as fs from 'node:fs';
18+
import * as fs_promises from 'node:fs/promises';
19+
import * as http from 'node:http';
20+
import * as https from 'node:https';
21+
import * as net from 'node:net';
22+
import * as os from 'node:os';
23+
import path from 'node:path';
24+
import * as perf_hooks from 'node:perf_hooks';
25+
import * as process from 'node:process';
26+
import * as punycode from 'node:punycode';
27+
import * as querystring from 'node:querystring';
28+
import * as readline from 'node:readline';
29+
import * as stream from 'node:stream';
30+
import * as stream_promises from 'node:stream/promises';
31+
import * as stream_web from 'node:stream/web';
32+
import * as string_decoder from 'node:string_decoder';
33+
import * as timers from 'node:timers';
34+
import * as timers_promises from 'node:timers/promises';
35+
import * as tls from 'node:tls';
36+
import * as tty from 'node:tty';
37+
import * as url from 'node:url';
38+
// eslint-disable-next-line unicorn/import-style -- need full util module for CJS compatibility
39+
import * as util from 'node:util';
40+
import * as util_types from 'node:util/types';
41+
import * as worker_threads from 'node:worker_threads';
42+
import * as zlib from 'node:zlib';
43+
44+
// VM shim for Cloudflare Workers
45+
// JSDOM and some other libraries require vm module
46+
class ScriptShim {
47+
private code: string;
48+
constructor(code: string) {
49+
this.code = code;
50+
}
51+
runInContext() {
52+
throw new Error('vm.Script.runInContext is not supported in Workers');
53+
}
54+
runInNewContext() {
55+
throw new Error('vm.Script.runInNewContext is not supported in Workers');
56+
}
57+
runInThisContext() {
58+
throw new Error('vm.Script.runInThisContext is not supported in Workers');
59+
}
60+
}
61+
62+
const vmShim = {
63+
createContext: (sandbox?: object) => sandbox || {},
64+
runInContext: () => {
65+
throw new Error('vm.runInContext is not supported in Workers');
66+
},
67+
runInNewContext: () => {
68+
throw new Error('vm.runInNewContext is not supported in Workers');
69+
},
70+
runInThisContext: () => {
71+
throw new Error('vm.runInThisContext is not supported in Workers');
72+
},
73+
Script: ScriptShim,
74+
isContext: () => false,
75+
compileFunction: () => {
76+
throw new Error('vm.compileFunction is not supported in Workers');
77+
},
78+
};
79+
80+
// Create a CJS-compatible events module
81+
// In CJS, require('events') returns EventEmitter class directly (the default export)
82+
// but also has named exports attached to it
83+
const eventsModule = Object.assign(events, eventsNamespace);
84+
85+
// Map of module names to their exports
86+
const builtinModules: Record<string, unknown> = {
87+
fs,
88+
path,
89+
90+
util,
91+
stream,
92+
events: eventsModule,
93+
buffer,
94+
crypto,
95+
http,
96+
https,
97+
url,
98+
querystring,
99+
zlib,
100+
os,
101+
assert,
102+
tty,
103+
net,
104+
dns,
105+
child_process,
106+
string_decoder,
107+
timers,
108+
process,
109+
perf_hooks,
110+
async_hooks,
111+
worker_threads,
112+
tls,
113+
readline,
114+
115+
punycode,
116+
117+
constants,
118+
diagnostics_channel,
119+
console: console_module,
120+
vm: vmShim,
121+
// Also support node: prefix
122+
'node:fs': fs,
123+
'node:path': path,
124+
'node:util': util,
125+
'node:stream': stream,
126+
'node:events': eventsModule,
127+
'node:buffer': buffer,
128+
'node:crypto': crypto,
129+
'node:http': http,
130+
'node:https': https,
131+
'node:url': url,
132+
'node:querystring': querystring,
133+
'node:zlib': zlib,
134+
'node:os': os,
135+
'node:assert': assert,
136+
'node:tty': tty,
137+
'node:net': net,
138+
'node:dns': dns,
139+
'node:child_process': child_process,
140+
'node:string_decoder': string_decoder,
141+
'node:timers': timers,
142+
'node:process': process,
143+
'node:perf_hooks': perf_hooks,
144+
'node:async_hooks': async_hooks,
145+
'node:worker_threads': worker_threads,
146+
'node:tls': tls,
147+
'node:readline': readline,
148+
'node:punycode': punycode,
149+
'node:constants': constants,
150+
'node:diagnostics_channel': diagnostics_channel,
151+
'node:console': console_module,
152+
'node:fs/promises': fs_promises,
153+
'fs/promises': fs_promises,
154+
'node:stream/promises': stream_promises,
155+
'stream/promises': stream_promises,
156+
'node:stream/web': stream_web,
157+
'stream/web': stream_web,
158+
'node:util/types': util_types,
159+
'util/types': util_types,
160+
'node:timers/promises': timers_promises,
161+
'timers/promises': timers_promises,
162+
'node:vm': vmShim,
163+
};
164+
165+
export function createRequire(_filename: string | URL) {
166+
return function require(id: string): unknown {
167+
if (id in builtinModules) {
168+
return builtinModules[id];
169+
}
170+
// For non-builtin modules, throw an error
171+
throw new Error(`require() is not available in Workers. Attempted to require: ${id}`);
172+
};
173+
}
174+
175+
export default {
176+
createRequire,
177+
};

lib/shims/sentry-node.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
// No-op shim for @sentry/node in Cloudflare Workers
2+
export const withScope = (callback: (scope: unknown) => void) => callback({});
3+
export const captureException = () => {};

lib/utils/cache/index.ts

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { config } from '@/config';
2+
import { isWorker } from '@/utils/is-worker';
23
import logger from '@/utils/logger';
34

45
import type CacheModule from './base';
@@ -15,7 +16,18 @@ const globalCache: {
1516

1617
let cacheModule: CacheModule;
1718

18-
if (config.cache.type === 'redis') {
19+
if (isWorker) {
20+
// No-op cache for Cloudflare Workers
21+
cacheModule = {
22+
init: () => null,
23+
get: () => null,
24+
set: () => null,
25+
status: {
26+
available: false,
27+
},
28+
clients: {},
29+
};
30+
} else if (config.cache.type === 'redis') {
1931
cacheModule = redis;
2032
cacheModule.init();
2133
const { redisClient } = cacheModule.clients;

lib/utils/cache/index.worker.ts

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
// Worker-specific cache module - no-op implementation
2+
// This file is used instead of index.ts when building for Cloudflare Workers
3+
4+
import { config } from '@/config';
5+
6+
import type CacheModule from './base';
7+
8+
const globalCache: {
9+
get: (key: string) => Promise<string | null | undefined> | string | null | undefined;
10+
set: (key: string, value?: string | Record<string, any>, maxAge?: number) => any;
11+
} = {
12+
get: () => null,
13+
set: () => null,
14+
};
15+
16+
// No-op cache module for Worker
17+
const cacheModule: CacheModule = {
18+
init: () => null,
19+
get: () => null,
20+
set: () => null,
21+
status: {
22+
available: false,
23+
},
24+
clients: {},
25+
};
26+
27+
export default {
28+
...cacheModule,
29+
/**
30+
* Try to get the cache. If the cache does not exist, the `getValueFunc` function will be called to get the data, and the data will be cached.
31+
* @param key The key used to store and retrieve the cache. You can use `:` as a separator to create a hierarchy.
32+
* @param getValueFunc A function that returns data to be cached when a cache miss occurs.
33+
* @param maxAge The maximum age of the cache in seconds. This should left to the default value in most cases which is `CACHE_CONTENT_EXPIRE`.
34+
* @param refresh Whether to renew the cache expiration time when the cache is hit. `true` by default.
35+
* @returns
36+
*/
37+
tryGet: async <T extends string | Record<string, any>>(key: string, getValueFunc: () => Promise<T>, _maxAge = config.cache.contentExpire, _refresh = true) => {
38+
if (typeof key !== 'string') {
39+
throw new TypeError('Cache key must be a string');
40+
}
41+
// In Worker environment, always call getValueFunc since cache is not available
42+
const value = await getValueFunc();
43+
return value;
44+
},
45+
globalCache,
46+
};

0 commit comments

Comments
 (0)