Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ These commands are intended more for scripts, bots, and agent integrations than
```bash
ghcrawl threads owner/repo --numbers 42,43,44
ghcrawl threads owner/repo --numbers 42,43,44 --include-closed
ghcrawl pr-template owner/repo --template-file ./pull_request_template.md
ghcrawl pr-template owner/repo --max-distance 200
ghcrawl author owner/repo --login lqquan
ghcrawl close-thread owner/repo --number 42
ghcrawl close-cluster owner/repo --id 123
Expand All @@ -178,6 +180,8 @@ ghcrawl search owner/repo --query "download stalls"

Use `threads --numbers ...` when you want several specific issue or PR records in one CLI call instead of paying process startup overhead repeatedly.

Use `pr-template` when you want to flag pull requests that still contain the repository PR template verbatim or are only a small edit-distance away from it. Pass `--template-file` to force a local template snapshot, or omit it to let `ghcrawl` probe common GitHub PR-template paths for the target repo. When the body contains the `## Summary` ... `## Risks and Mitigations` template block, `levenshteinDistance` is computed on that extracted section and `fullBodyLevenshteinDistance` keeps the broader whole-description score.

Use `author --login ...` when you want all currently open issue/PR records from one user plus the strongest stored same-author similarity match for each item.

By default, JSON list commands filter out locally closed issues/PRs and completely closed clusters. Use `--include-closed` when you need to inspect those records too.
Expand Down Expand Up @@ -220,6 +224,7 @@ The skill is built around the stable JSON CLI surface and is intentionally conse
ghcrawl doctor --json
ghcrawl refresh owner/repo
ghcrawl threads owner/repo --numbers 42,43,44
ghcrawl pr-template owner/repo --max-distance 200
ghcrawl clusters owner/repo --min-size 10 --limit 20 --sort recent
ghcrawl cluster-detail owner/repo --id 123 --member-limit 20 --body-chars 280
```
Expand Down
5 changes: 5 additions & 0 deletions apps/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ These commands are intended more for scripts, bots, and agent integrations than

```bash
ghcrawl threads owner/repo --numbers 42,43,44
ghcrawl pr-template owner/repo --template-file ./pull_request_template.md
ghcrawl pr-template owner/repo --max-distance 200
ghcrawl author owner/repo --login lqquan
ghcrawl cluster owner/repo
ghcrawl clusters owner/repo --min-size 10 --limit 20
Expand All @@ -160,6 +162,8 @@ ghcrawl search owner/repo --query "download stalls"

Use `threads --numbers ...` when you want several specific issue or PR records in one CLI call instead of paying process startup overhead repeatedly.

Use `pr-template` when you want to flag pull requests that still contain the repository PR template verbatim or are only a small edit-distance away from it. Pass `--template-file` to force a local template snapshot, or omit it to let `ghcrawl` probe common GitHub PR-template paths for the target repo. When the body contains the `## Summary` ... `## Risks and Mitigations` template block, `levenshteinDistance` is computed on that extracted section and `fullBodyLevenshteinDistance` keeps the broader whole-description score.

Use `author --login ...` when you want all currently open issue/PR records from one user plus the strongest stored same-author similarity match for each item.

## Cost To Operate
Expand Down Expand Up @@ -195,6 +199,7 @@ The skill is built around the stable JSON CLI surface and is intentionally conse
ghcrawl doctor --json
ghcrawl refresh owner/repo
ghcrawl threads owner/repo --numbers 42,43,44
ghcrawl pr-template owner/repo --max-distance 200
ghcrawl clusters owner/repo --min-size 10 --limit 20 --sort recent
ghcrawl cluster-detail owner/repo --id 123 --member-limit 20 --body-chars 280
```
Expand Down
10 changes: 10 additions & 0 deletions apps/cli/src/main.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ test('run prints usage with no command', async () => {
assert.match(output, /\n version\n/);
assert.match(output, /refresh <owner\/repo>/);
assert.match(output, /threads <owner\/repo>/);
assert.match(output, /pr-template <owner\/repo>/);
assert.match(output, /author <owner\/repo> --login <user>/);
assert.match(output, /close-thread <owner\/repo> --number <thread>/);
assert.match(output, /close-cluster <owner\/repo> --id <cluster-id>/);
Expand All @@ -43,6 +44,7 @@ test('run prints usage for help flag', async () => {
assert.match(output, /\n version\n/);
assert.match(output, /refresh <owner\/repo>/);
assert.match(output, /threads <owner\/repo>/);
assert.match(output, /pr-template <owner\/repo>/);
assert.match(output, /author <owner\/repo> --login <user>/);
assert.match(output, /close-thread <owner\/repo> --number <thread>/);
assert.match(output, /tui \[owner\/repo\]/);
Expand Down Expand Up @@ -168,6 +170,14 @@ test('parseRepoFlags accepts kind filter for threads', () => {
assert.equal(parsed.repo, 'openclaw');
assert.equal(parsed.values.kind, 'pull_request');
});

test('parseRepoFlags accepts pr-template options', () => {
const parsed = parseRepoFlags(['openclaw/openclaw', '--template-file', './template.md', '--max-distance', '200']);
assert.equal(parsed.owner, 'openclaw');
assert.equal(parsed.repo, 'openclaw');
assert.equal(parsed.values['template-file'], './template.md');
assert.equal(parsed.values['max-distance'], '200');
});
test('resolveSinceValue keeps ISO timestamps', () => {
assert.equal(resolveSinceValue('2026-03-01T00:00:00Z'), '2026-03-01T00:00:00.000Z');
});
Expand Down
42 changes: 42 additions & 0 deletions apps/cli/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ type CommandName =
| 'sync'
| 'refresh'
| 'threads'
| 'pr-template'
| 'author'
| 'close-thread'
| 'close-cluster'
Expand Down Expand Up @@ -45,6 +46,7 @@ function usage(devMode = false): string {
' sync <owner/repo> [--since <iso|duration>] [--limit <count>] [--include-comments] [--full-reconcile]',
' refresh <owner/repo> [--no-sync] [--no-embed] [--no-cluster]',
' threads <owner/repo> [--numbers <n,n,...>] [--kind issue|pull_request] [--include-closed]',
' pr-template <owner/repo> [--template-file <path>] [--max-distance <count>] [--limit <count>] [--include-closed]',
' author <owner/repo> --login <user> [--include-closed]',
' close-thread <owner/repo> --number <thread>',
' close-cluster <owner/repo> --id <cluster-id>',
Expand Down Expand Up @@ -105,6 +107,8 @@ export function parseRepoFlags(args: string[]): { owner: string; repo: string; v
kind: { type: 'string' },
number: { type: 'string' },
numbers: { type: 'string' },
'template-file': { type: 'string' },
'max-distance': { type: 'string' },
login: { type: 'string' },
query: { type: 'string' },
mode: { type: 'string' },
Expand Down Expand Up @@ -207,6 +211,14 @@ function parsePositiveInteger(name: string, value: string): number {
return parsed;
}

function parseNonNegativeInteger(name: string, value: string): number {
const parsed = Number(value);
if (!Number.isSafeInteger(parsed) || parsed < 0) {
throw new Error(`Invalid ${name}: ${value}`);
}
return parsed;
}

function parsePositiveIntegerList(name: string, value: string): number[] {
const parts = value
.split(',')
Expand Down Expand Up @@ -371,6 +383,36 @@ export async function run(argv: string[], stdout: NodeJS.WritableStream = proces
stdout.write(`${JSON.stringify(result, null, 2)}\n`);
return;
}
case 'pr-template': {
const { owner, repo, values } = parseRepoFlags(rest);
const templatePath =
typeof values['template-file'] === 'string' && values['template-file'].trim().length > 0
? path.resolve(values['template-file'])
: null;
const template = templatePath
? {
text: readFileSync(templatePath, 'utf8'),
source: {
mode: 'file' as const,
label: templatePath,
},
}
: await getService().getPullRequestTemplate({ owner, repo });
const result = getService().findPullRequestTemplateMatches({
owner,
repo,
templateText: template.text,
templateSource: template.source,
maxDistance:
typeof values['max-distance'] === 'string'
? parseNonNegativeInteger('max-distance', values['max-distance'])
: undefined,
limit: typeof values.limit === 'string' ? parsePositiveInteger('limit', values.limit) : undefined,
includeClosed: values['include-closed'] === true,
});
stdout.write(`${JSON.stringify(result, null, 2)}\n`);
return;
}
case 'author': {
const { owner, repo, values } = parseRepoFlags(rest);
if (typeof values.login !== 'string' || values.login.trim().length === 0) {
Expand Down
67 changes: 66 additions & 1 deletion packages/api-contract/src/contracts.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
import test from 'node:test';
import assert from 'node:assert/strict';

import { actionRequestSchema, healthResponseSchema, neighborsResponseSchema, searchResponseSchema } from './contracts.js';
import {
actionRequestSchema,
healthResponseSchema,
neighborsResponseSchema,
prTemplateMatchesResponseSchema,
searchResponseSchema,
} from './contracts.js';

test('health schema accepts configured status payload', () => {
const parsed = healthResponseSchema.parse({
Expand Down Expand Up @@ -87,3 +93,62 @@ test('neighbors schema accepts repository, source thread, and neighbor list', ()

assert.equal(parsed.neighbors[0].number, 43);
});

test('pr template matches schema accepts heuristic match payload', () => {
const parsed = prTemplateMatchesResponseSchema.parse({
repository: {
id: 1,
owner: 'openclaw',
name: 'openclaw',
fullName: 'openclaw/openclaw',
githubRepoId: null,
updatedAt: new Date().toISOString(),
},
template: {
source: {
mode: 'github',
label: '.github/pull_request_template.md',
},
length: 128,
},
filters: {
exact: true,
maxDistance: 200,
includeClosed: false,
},
matches: [
{
thread: {
id: 11,
repoId: 1,
number: 43,
kind: 'pull_request',
state: 'open',
isClosed: false,
closedAtGh: null,
closedAtLocal: null,
closeReasonLocal: null,
title: 'Fix downloader hang',
body: 'Checklist here',
authorLogin: 'alice',
htmlUrl: 'https://github.com/openclaw/openclaw/pull/43',
labels: ['bug'],
updatedAtGh: new Date().toISOString(),
clusterId: null,
},
exactMatch: true,
exactMatchOffset: 12,
templateSectionFound: true,
templateSectionExactMatch: false,
templateSectionStartOffset: 12,
templateSectionEndOffset: 140,
levenshteinDistance: 42,
fullBodyLevenshteinDistance: 55,
bodyLength: 150,
},
],
});

assert.equal(parsed.matches[0].exactMatch, true);
assert.equal(parsed.matches[0].levenshteinDistance, 42);
});
35 changes: 35 additions & 0 deletions packages/api-contract/src/contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,41 @@ export const threadSchema = z.object({
});
export type ThreadDto = z.infer<typeof threadSchema>;

export const prTemplateSourceSchema = z.object({
mode: z.enum(['file', 'github']),
label: z.string(),
});
export type PrTemplateSourceDto = z.infer<typeof prTemplateSourceSchema>;

export const prTemplateMatchSchema = z.object({
thread: threadSchema,
exactMatch: z.boolean(),
exactMatchOffset: z.number().int().nonnegative().nullable(),
templateSectionFound: z.boolean(),
templateSectionExactMatch: z.boolean(),
templateSectionStartOffset: z.number().int().nonnegative().nullable(),
templateSectionEndOffset: z.number().int().nonnegative().nullable(),
levenshteinDistance: z.number().int().nonnegative().nullable(),
fullBodyLevenshteinDistance: z.number().int().nonnegative().nullable(),
bodyLength: z.number().int().nonnegative(),
});
export type PrTemplateMatchDto = z.infer<typeof prTemplateMatchSchema>;

export const prTemplateMatchesResponseSchema = z.object({
repository: repositorySchema,
template: z.object({
source: prTemplateSourceSchema,
length: z.number().int().positive(),
}),
filters: z.object({
exact: z.boolean(),
maxDistance: z.number().int().nonnegative().nullable(),
includeClosed: z.boolean(),
}),
matches: z.array(prTemplateMatchSchema),
});
export type PrTemplateMatchesResponse = z.infer<typeof prTemplateMatchesResponseSchema>;

export const healthResponseSchema = z.object({
ok: z.boolean(),
configPath: z.string(),
Expand Down
7 changes: 7 additions & 0 deletions packages/api-core/src/api/server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ test('health endpoint returns contract payload', async () => {
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -80,6 +81,7 @@ test('neighbors endpoint returns contract payload', async () => {
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -172,6 +174,7 @@ test('threads endpoint can filter by a bulk number list', async () => {
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -241,6 +244,7 @@ test('author-threads endpoint returns one author with strongest same-author matc
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -320,6 +324,7 @@ test('close-thread and includeClosed thread routes expose locally closed items',
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -403,6 +408,7 @@ test('server returns 400 for malformed request inputs', async () => {
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down Expand Up @@ -457,6 +463,7 @@ test('cluster summary and detail endpoints return contract payloads', async () =
github: {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down
1 change: 1 addition & 0 deletions packages/api-core/src/cluster/perf.integration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ function createGitHubStub(): GHCrawlService['github'] {
return {
checkAuth: async () => undefined,
getRepo: async () => ({}),
getFileContents: async () => { throw new Error("not expected"); },
listRepositoryIssues: async () => [],
getIssue: async () => ({}),
getPull: async () => ({}),
Expand Down
24 changes: 24 additions & 0 deletions packages/api-core/src/github/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,13 @@ import { Octokit } from 'octokit';
export type GitHubClient = {
checkAuth: (reporter?: GitHubReporter) => Promise<void>;
getRepo: (owner: string, repo: string, reporter?: GitHubReporter) => Promise<Record<string, unknown>>;
getFileContents: (
owner: string,
repo: string,
filePath: string,
ref?: string,
reporter?: GitHubReporter,
) => Promise<string>;
listRepositoryIssues: (
owner: string,
repo: string,
Expand Down Expand Up @@ -167,6 +174,23 @@ export function makeGitHubClient(options: RequestOptions): GitHubClient {
return response.data as Record<string, unknown>;
});
},
async getFileContents(owner, repo, filePath, ref, reporter) {
return request(`GET /repos/${owner}/${repo}/contents/${filePath}`, reporter, async (octokit) => {
const response = await octokit.rest.repos.getContent({
owner,
repo,
path: filePath,
ref,
mediaType: {
format: 'raw',
},
});
if (typeof response.data !== 'string') {
throw new Error(`GitHub content for ${filePath} was not returned as raw text.`);
}
return response.data;
});
},
async listRepositoryIssues(owner, repo, since, limit, reporter, state = 'open') {
return paginate(
`GET /repos/${owner}/${repo}/issues state=${state} per_page=100`,
Expand Down
Loading
Loading