Merge pull request #51 from Comfy-Org/sno-label-op-system

feat: implement long-running repository event monitoring system

Author: snomiao

.env

@@ -16,3 +16,6 @@ [email protected]
 AUTH_ADMINS=[email protected]
 
 SLACK_BOT_TOKEN="FILL_THIS_INTO_ .env.local"
+
+# default port for GITHUB_WEBHOOK_PORT for webhook
+GITHUB_WEBHOOK_PORT=14215

.gitignore

@@ -56,4 +56,10 @@ package-lock.json
 report
 
 # bun crash dump
-core
+core
+
+# gh-service sqlite cache
+gh-service/state.sqlite
+
+**/*.sqlite
+**/*.sqlite-journal

CLAUDE.md

@@ -0,0 +1,104 @@
+# Claude Development Notes
+
+## General Development Flow for All Scripts
+
+### Standard Development Pattern
+
+1. **Install Dependencies**: Use `bun add <package>` for runtime deps, `bun add -d <package>` for dev deps
+2. **Create Implementation**: Write the main functionality in TypeScript
+3. **Add Testing**: Create `.spec.ts` files with comprehensive test coverage
+4. **Test Execution**: Use `bun <file.ts>` to run scripts directly with `if (import.meta.main)` blocks
+5. **Type Safety**: Ensure full TypeScript coverage and proper type definitions
+6. **Error Handling**: Handle errors gracefully and don't cache failed responses
+7. **Documentation**: Update CLAUDE.md with implementation details and usage examples
+
+### Testing Standards
+
+- **File Naming**: Use `.spec.ts` suffix for test files
+- **Mocking**: Mock external APIs and services to avoid real calls during tests
+- **Coverage Areas**: Test happy path, error cases, edge cases, concurrent scenarios
+- **Setup/Teardown**: Clean up resources (files, caches, etc.) in test lifecycle hooks
+
+### Common Patterns
+
+- **Caching**: Use Keyv with SQLite for persistent caching in `node_modules/.cache/Comfy-PR/`
+- **Configuration**: Store config in environment variables with sensible defaults
+- **Logging**: Use console.log sparingly, prefer structured logging for debugging
+- **File Organization**: Keep related functionality together, use clear module exports
+
+### Repository Structure
+
+- **`src/`**: Core utilities and shared functionality
+- **`app/tasks/`**: Specific task implementations
+- **`gh-service/`**: GitHub webhook service components
+- **`run/`**: Executable scripts and services
+- **Tests**: Co-located with source files using `.spec.ts` suffix
+
+## Cached GitHub Client (ghc)
+
+### Overview
+
+Created a cached wrapper around the GitHub API client that transparently caches all API responses to improve performance and reduce API rate limiting.
+
+### Implementation
+
+- **File**: `src/ghc.ts`
+- **Cache Storage**: SQLite via Keyv in `node_modules/.cache/Comfy-PR/gh-cache.sqlite`
+- **Default TTL**: 5 minutes
+- **Cache Key Format**: `gh.{apiPath}({truncatedArgs})#{hash}`
+
+### Key Features
+
+1. **Transparent Caching**: Same interface as `gh` object, just import `ghc` instead
+2. **Smart Cache Keys**: Truncates long arguments but preserves full hash for accuracy
+3. **SQLite Storage**: Persistent cache across runs
+4. **Type Safety**: Maintains full TypeScript types from original GitHub client
+
+### Usage
+
+```typescript
+import { ghc } from "./src/ghc";
+
+// Same as gh.repos.get() but with caching
+const repo = await ghc.repos.get({
+  owner: "octocat",
+  repo: "Hello-World",
+});
+
+// Second call uses cache
+const cachedRepo = await ghc.repos.get({
+  owner: "octocat",
+  repo: "Hello-World",
+});
+```
+
+### Cache Management
+
+```typescript
+import { clearGhCache, getGhCacheStats } from "./src/ghc";
+
+// Clear all cached data
+await clearGhCache();
+
+// Get cache statistics
+const stats = await getGhCacheStats();
+```
+
+### Testing
+
+- **Test File**: `src/ghc.spec.ts`
+- **Coverage**: Cache hits/misses, error handling, concurrent requests, key generation
+- **Mocking**: Uses Jest mocks to avoid real API calls during tests
+
+### Development Flow
+
+1. Install dependencies: `bun add keyv @keyv/sqlite`
+2. Create cached wrapper with Proxy pattern
+3. Implement cache key generation with argument truncation
+4. Add comprehensive test suite
+5. Test with real API calls: `bun src/ghc.ts`
+
+### Cache Key Examples
+
+- Short args: `gh.repos.get({"owner":"octocat","repo":"Hello-World"})#b3117af2`
+- Long args: `gh.repos.get({"owner":"octocat","descripti...bbbbbbbbbb"})#4240f076`

WEBHOOK_SETUP.md

@@ -0,0 +1,81 @@
+# GitHub Repository Event Monitor - Webhook Setup
+
+This system now supports both polling and real-time webhook-based monitoring.
+
+## Quick Start
+
+### Polling Mode (Default)
+
+```bash
+bun run run/label-op.tsx
+```
+
+### Webhook Mode (Real-time)
+
+```bash
+# Set environment variables
+export USE_WEBHOOKS=true
+export GITHUB_WEBHOOK_SECRET=your-secure-secret-here
+export WEBHOOK_BASE_URL=https://your-domain.com  # Your public URL
+
+# Run the monitor
+bun run run/label-op.tsx
+```
+
+## Webhook Setup Instructions
+
+### 1. Environment Variables
+
+- `USE_WEBHOOKS=true` - Enable webhook mode
+- `GITHUB_WEBHOOK_SECRET` - Secret for webhook security (recommended)
+- `WEBHOOK_BASE_URL` - Your public URL (for webhook endpoint)
+- `GH_TOKEN` - GitHub personal access token with repo permissions
+
+### 2. Public URL Requirements
+
+For webhooks to work, your server needs to be accessible from the internet:
+
+- **Development**: Use ngrok, cloudflare tunnels, or similar
+- **Production**: Deploy to a cloud provider with a public domain
+
+### 3. GitHub Permissions
+
+The system will automatically attempt to create webhooks for repositories in `REPOLIST`. You need:
+
+- Admin or write access to the repositories
+- GitHub token with `repo` permissions
+
+### 4. Webhook Events Monitored
+
+- Issues (opened, closed, labeled, etc.)
+- Pull Requests (opened, closed, merged, labeled, etc.)
+- Comments (issue comments, PR comments, review comments)
+- Labels (added, removed)
+
+## Comparison: Polling vs Webhooks
+
+| Feature            | Polling (30s)     | Webhooks            |
+| ------------------ | ----------------- | ------------------- |
+| **Latency**        | ~30 seconds       | ~1 second           |
+| **API Usage**      | High (continuous) | Low (event-driven)  |
+| **Setup**          | Simple            | Requires public URL |
+| **Reliability**    | Always works      | Depends on network  |
+| **Resource Usage** | Higher            | Lower               |
+
+## Troubleshooting
+
+### Webhook Creation Failed
+
+- Check GitHub token permissions
+- Ensure you have admin access to repositories
+- Verify the WEBHOOK_BASE_URL is publicly accessible
+
+### Webhook Not Receiving Events
+
+- Test the webhook endpoint: `curl -X POST http://your-url/webhook`
+- Check GitHub webhook delivery logs in repo settings
+- Verify webhook secret matches if configured
+
+### Fallback Strategy
+
+The system gracefully falls back to polling mode if webhook setup fails for any repository.