Skip to content

Commit 5715ee0

Browse files
Merge remote-tracking branch 'refs/remotes/duan/master'
# Conflicts: # CLAUDE.md # README.md # README.zh.md # digests/web-state.json # feed.xml # manifest.json # package.json # pnpm-lock.yaml # src/generate-manifest.ts # src/index.ts # src/notify.ts # src/prompts.ts # src/report.ts # src/rollup.ts
2 parents 402ba5c + 76033c1 commit 5715ee0

File tree

228 files changed

+152908
-2542
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

228 files changed

+152908
-2542
lines changed

.github/workflows/ci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,6 @@ jobs:
3333

3434
- name: Typecheck
3535
run: pnpm typecheck
36+
37+
- name: Test
38+
run: pnpm test

.github/workflows/daily-digest.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,9 @@ jobs:
7878
PAGES_URL: https://compasify.github.io/agents-radar
7979
run: pnpm notify
8080

81+
- name: Send Feishu notification
82+
env:
83+
FEISHU_WEBHOOK_URL: ${{ secrets.FEISHU_WEBHOOK_URL }}
84+
PAGES_URL: https://duanyytop.github.io/agents-radar
85+
run: pnpm notify:feishu
86+

.husky/pre-commit

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
pnpm lint
2+
pnpm format:check
3+
pnpm typecheck

CLAUDE.md

Lines changed: 45 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -2,35 +2,40 @@
22

33
## Project overview
44

5-
agents-radar is a daily digest generator for the AI open-source ecosystem. A GitHub Actions cron job runs at 00:00 UTC (08:00 CST) and produces five Chinese-language reports, published as GitHub Issues and committed Markdown files.
5+
agents-radar is a daily digest generator for the AI open-source ecosystem. A GitHub Actions cron job runs at 00:00 UTC (08:00 CST) and produces bilingual (Chinese + English) reports, published as GitHub Issues and committed Markdown files.
66

77
## Commands
88

99
```bash
1010
pnpm start # run the full digest locally
11+
pnpm test # vitest (unit tests)
1112
pnpm typecheck # tsc --noEmit
1213
pnpm lint # ESLint
1314
pnpm lint:fix # ESLint --fix
1415
pnpm format # Prettier --write src
1516
pnpm format:check # Prettier --check src
1617
```
1718

18-
Required env vars for local runs (set **one** LLM provider group):
19+
Required env vars for local runs:
1920

2021
```bash
2122
export GITHUB_TOKEN=ghp_xxxxx
23+
export DIGEST_REPO=owner/repo # omit to skip GitHub issue creation
2224

23-
# Option A — OpenAI-compatible (takes precedence when OPENAI_API_KEY is set)
24-
export OPENAI_API_KEY=sk-xxxxx
25-
export OPENAI_BASE_URL=https://your-provider/v1 # optional
26-
export OPENAI_MODEL=gpt-4o # optional, default: gpt-4o
25+
# LLM provider (default: anthropic)
26+
export LLM_PROVIDER=anthropic # anthropic | openai | github-copilot | openrouter
2727

28-
# Option B — Anthropic (default when OPENAI_API_KEY is absent)
28+
# Anthropic (default)
2929
export ANTHROPIC_API_KEY=sk-ant-xxxxx
3030
export ANTHROPIC_BASE_URL=https://api.kimi.com/coding/ # omit for Anthropic
31-
export ANTHROPIC_MODEL=claude-sonnet-4-6 # optional
3231

33-
export DIGEST_REPO=owner/repo # omit to skip GitHub issue creation
32+
# OpenAI
33+
# export OPENAI_API_KEY=sk-xxxxx
34+
35+
# GitHub Copilot — uses GITHUB_TOKEN
36+
37+
# OpenRouter
38+
# export OPENROUTER_API_KEY=sk-or-xxxxx
3439
```
3540

3641
## Architecture
@@ -40,16 +45,29 @@ The pipeline runs in four sequential phases, each implemented as a named async f
4045
1. **`fetchAllData`** — all network I/O in parallel: GitHub API (issues/PRs/releases) for 17 repos, Claude Code Skills, Anthropic/OpenAI sitemaps, GitHub Trending HTML + Search API, Hacker News Algolia API.
4146
2. **`generateSummaries`** — per-repo LLM calls, all in parallel, rate-limited to 5 concurrent requests by a queue in `src/report.ts`.
4247
3. **Comparisons** — two LLM calls: cross-tool CLI comparison and OpenClaw cross-ecosystem comparison.
43-
4. **Save phase**`buildCliReportContent` / `buildOpenclawReportContent` build Markdown strings; `saveWebReport` / `saveTrendingReport` / `saveHnReport` call LLM + write file + create GitHub Issue.
48+
4. **Save phase**`buildCliReportContent` / `buildOpenclawReportContent` (in `src/report-builders.ts`) build Markdown strings; `saveWebReport` / `saveTrendingReport` / `saveHnReport` (in `src/report-savers.ts`) call LLM + write file + create GitHub Issue.
4449

4550
## Source files
4651

4752
| File | Responsibility |
4853
|------|---------------|
4954
| `src/index.ts` | Orchestration: repo config, phase functions, `main()` |
50-
| `src/github.ts` | GitHub API helpers: `fetchRecentItems`, `fetchRecentReleases`, `fetchSkillsData`, `createGitHubIssue` |
51-
| `src/prompts.ts` | LLM prompt builders (one per report type) and `formatItem` |
52-
| `src/report.ts` | `callLlm` (with concurrency limiter), `saveFile`, `autoGenFooter` |
55+
| `src/i18n.ts` | Centralized bilingual strings: `Lang` type, report titles, issue labels, footer text, `REPORT_LABELS`, `NOTIFY_LABELS` |
56+
| `src/github.ts` | GitHub API helpers: `fetchRecentItems`, `fetchRecentReleases`, `fetchSkillsData`, `createGitHubIssue`; shared `RepoFetch` type |
57+
| `src/prompts.ts` | LLM prompt builders for repo reports: `buildCliPrompt`, `buildPeerPrompt`, `buildComparisonPrompt`, `buildPeersComparisonPrompt`, `buildSkillsPrompt` |
58+
| `src/prompts-data.ts` | LLM prompt builders for data-source reports: `buildTrendingPrompt`, `buildWebReportPrompt`, `buildHnPrompt`, `buildWeeklyPrompt`, `buildMonthlyPrompt` |
59+
| `src/report.ts` | `callLlm` (with concurrency limiter), `saveFile`, `autoGenFooter` (uses i18n), LLM token budget constants |
60+
| `src/report-builders.ts` | `buildCliReportContent`, `buildOpenclawReportContent` — assemble final Markdown strings for CLI and OpenClaw reports |
61+
| `src/report-savers.ts` | `saveWebReport`, `saveTrendingReport`, `saveHnReport` — LLM call + file save + optional GitHub issue |
62+
| `src/date.ts` | Date and timing utilities: `toCstDateStr`, `toUtcStr`, `sleep` |
63+
| `src/rollup.ts` | Weekly and monthly rollup report generator |
64+
| `src/providers/types.ts` | `LlmProvider` interface, `ProviderName` type, `VALID_PROVIDER_NAMES` |
65+
| `src/providers/openai-compatible.ts` | `OpenAICompatibleProvider` — shared base class for OpenAI-compatible providers |
66+
| `src/providers/anthropic.ts` | `AnthropicProvider` — Anthropic SDK wrapper |
67+
| `src/providers/openai.ts` | `OpenAIProvider` — extends `OpenAICompatibleProvider` |
68+
| `src/providers/github-copilot.ts` | `GitHubCopilotProvider` — extends `OpenAICompatibleProvider` |
69+
| `src/providers/openrouter.ts` | `OpenRouterProvider` — extends `OpenAICompatibleProvider` |
70+
| `src/providers/index.ts` | `createProvider` factory + barrel re-exports |
5371
| `src/web.ts` | Sitemap-based web content fetching; state persisted to `digests/web-state.json` |
5472
| `src/trending.ts` | GitHub Trending HTML scraper + Search API topic queries |
5573
| `src/hn.ts` | Hacker News top AI stories via Algolia HN Search API |
@@ -78,10 +96,13 @@ Files written to `digests/YYYY-MM-DD/`:
7896

7997
## Key conventions
8098

81-
- All LLM prompts are in `src/prompts.ts`. Each report type has its own builder function. Prompts are written in Chinese and produce Chinese output.
99+
- All bilingual strings (titles, labels, footers, messages) are centralized in `src/i18n.ts`. Use the `Lang` type (`"zh" | "en"`) and `Record<Lang, string>` maps. Do not add inline bilingual ternaries elsewhere.
100+
- LLM prompt builders are split across two files: `src/prompts.ts` (repo-level prompts) and `src/prompts-data.ts` (data-source and rollup prompts). Each report type has its own builder function.
82101
- `callLlm(prompt, maxTokens?)` defaults to 4096 tokens. Web report uses 8192, trending uses 6144. HN report uses the default 4096.
83102
- On 429 rate-limit errors `callLlm` retries up to 3 times with exponential backoff (5 s / 10 s / 20 s); the concurrency slot is released during the wait.
84-
- The concurrency limiter (`LLM_CONCURRENCY = 5`) prevents 429s when many parallel LLM calls fire. Do not bypass it by calling the Anthropic SDK directly.
103+
- The concurrency limiter (`LLM_CONCURRENCY = 5`) prevents 429s when many parallel LLM calls fire. Do not bypass it by calling SDK clients directly.
104+
- LLM provider is selected via `LLM_PROVIDER` env var (default: `anthropic`). Valid values: `anthropic`, `openai`, `github-copilot`, `openrouter`.
105+
- Provider implementations live in `src/providers/`. Each file implements the `LlmProvider` interface. The factory in `src/providers/index.ts` validates the provider name and logs only the provider name — never API keys or endpoint URLs.
85106
- GitHub issue label colors are defined in `LABEL_COLORS` in `src/github.ts`. Add new labels there.
86107
- `sampleNote(total, sampled)` in `src/prompts.ts` formats the "(共 N 条,展示前 M 条)" note. Reuse it — do not inline the same string format.
87108
- Web state (`digests/web-state.json`) is committed to git on every run. It is the source of truth for which URLs have been seen.
@@ -91,14 +112,16 @@ Files written to `digests/YYYY-MM-DD/`:
91112
- Web UI: `index.html` reads `manifest.json` to build the sidebar, then fetches `digests/YYYY-MM-DD/report.md` on demand.
92113
- RSS Feed: `feed.xml` at the repo root. Generated by `src/generate-manifest.ts` in the same `pnpm manifest` step. Contains the latest 30 items (newest first) across all report types. Item links use hash routing: `https://duanyytop.github.io/agents-radar/#YYYY-MM-DD/report`.
93114
- Both `manifest.json` and `feed.xml` are committed together in the "Commit manifest and feed" GHA step.
94-
- The `REPORT_LABELS` map in `generate-manifest.ts` must be kept in sync with the `LABELS` object in `index.html` when adding new report types.
115+
- The `REPORT_LABELS` map in `src/i18n.ts` must be kept in sync with the `LABELS` object in `index.html` when adding new report types.
95116

96117
## Adding a new report type
97118

98119
1. Create a data fetcher (or add to an existing one).
99-
2. Add a `buildXxxPrompt` function in `src/prompts.ts`.
100-
3. Wire into `fetchAllData`, `generateSummaries`, and a `saveXxxReport` function in `src/index.ts`.
101-
4. Add a label color entry in `LABEL_COLORS` in `src/github.ts`.
102-
5. Add the report ID and label to `REPORT_LABELS` in `src/generate-manifest.ts` and `LABELS` in `index.html`.
103-
6. Add the report file name to `REPORT_FILES` in `src/generate-manifest.ts`.
104-
7. Update both README files and this file.
120+
2. Add a `buildXxxPrompt` function in `src/prompts-data.ts` (for data-source prompts) or `src/prompts.ts` (for repo-level prompts).
121+
3. Add bilingual strings (titles, labels, issue title function) to `src/i18n.ts`.
122+
4. Add a `saveXxxReport` function in `src/report-savers.ts`.
123+
5. Wire into `fetchAllData`, `generateSummaries`, and the save phase in `src/index.ts`.
124+
6. Add a label color entry in `LABEL_COLORS` in `src/github.ts`.
125+
7. Add the report ID and label to `REPORT_LABELS` in `src/i18n.ts` and `LABELS` in `index.html`.
126+
8. Add the report file name to `REPORT_FILES` in `src/generate-manifest.ts`.
127+
9. Update both README files and this file.

0 commit comments

Comments
 (0)