Skip to content

Commit 8fb875b

Browse files
lpcoxCopilot
andauthored
perf: exclude browser tools and reduce turns in smoke-copilot (#1625)
* perf: exclude browser tools and reduce turns in smoke-copilot Implements recommendations from #1624: 1. Exclude 21 unused Playwright/browser tools via --excluded-tools in postprocess-smoke-workflows.ts (saves ~10,500 tokens/turn) 2. Remove redundant MCP verification call — pre-step already proves MCP connectivity, no need for agent to call list_pull_requests 3. Remove redundant bash echo test — bash functionality is already proven by the file write/read test in section 3 4. Drop repos toolset — only pull_requests tools are used, removing 4 unused tool schemas (~2,400 tokens/turn) Expected impact: 5→3 LLM turns, ~25-30% token reduction per run. Closes #1624 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR review feedback - Restore MCP call in section 1 so the workflow actually tests GitHub MCP connectivity (not just CLI/API) - Make --excluded-tools injection idempotent: strip any existing flag before re-injecting so updates to the excluded list are always applied correctly on re-runs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 2b175e2 commit 8fb875b

File tree

3 files changed

+51
-23
lines changed

3 files changed

+51
-23
lines changed

.github/workflows/smoke-copilot.lock.yml

Lines changed: 18 additions & 18 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.github/workflows/smoke-copilot.md

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ tools:
2222
bash:
2323
- "*"
2424
github:
25-
toolsets: [repos, pull_requests]
25+
toolsets: [pull_requests]
2626
safe-outputs:
2727
add-comment:
2828
hide-older-comments: true
@@ -100,7 +100,7 @@ post-steps:
100100
The following tests were already executed in a deterministic pre-agent step. Your job is to verify the results and produce the summary comment.
101101

102102
### 1. GitHub MCP Testing
103-
The last 2 merged pull requests have been fetched. Verify MCP connectivity by calling `github-list_pull_requests` for ${{ github.repository }} (limit 1, state merged) and confirm data is returned.
103+
Verify MCP connectivity by calling `github-list_pull_requests` for ${{ github.repository }} (limit 1, state merged). Confirm the result matches the pre-fetched data below.
104104

105105
### 2. GitHub.com Connectivity
106106
Pre-step result: HTTP ${{ steps.smoke-data.outputs.SMOKE_HTTP_CODE }} from github.com.
@@ -111,9 +111,6 @@ Pre-step wrote and read back: "${{ steps.smoke-data.outputs.SMOKE_FILE_CONTENT }
111111
File path: ${{ steps.smoke-data.outputs.SMOKE_FILE_PATH }}
112112
Verify by running `cat` on the file path using bash to confirm it exists.
113113

114-
### 4. Bash Tool Testing
115-
Run a simple bash command (e.g., `echo "bash works"`) to verify the bash tool is functional.
116-
117114
## Pre-Fetched PR Data
118115

119116
```

scripts/ci/postprocess-smoke-workflows.ts

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,37 @@ for (const workflowPath of workflowPaths) {
244244
console.log(` Replaced ${imageTagMatches.length} --image-tag/--skip-pull with --build-local`);
245245
}
246246

247+
// Exclude unused Playwright/browser tools from Copilot CLI for smoke-copilot.
248+
// The Copilot CLI includes 21 built-in browser_* tools when --allow-all-tools is set.
249+
// These tools are never used in smoke-copilot but add ~10,500 tokens/turn of dead weight.
250+
// We inject --excluded-tools after --allow-all-tools to suppress them.
251+
const isCopilotSmoke = workflowPath.includes('smoke-copilot.lock.yml');
252+
if (isCopilotSmoke) {
253+
const excludedToolsFlag =
254+
'--excluded-tools=browser_close,browser_resize,browser_console_messages,' +
255+
'browser_handle_dialog,browser_evaluate,browser_file_upload,browser_fill_form,' +
256+
'browser_press_key,browser_type,browser_navigate,browser_navigate_back,' +
257+
'browser_network_requests,browser_run_code,browser_take_screenshot,' +
258+
'browser_snapshot,browser_click,browser_drag,browser_hover,' +
259+
'browser_select_option,browser_tabs,browser_wait_for';
260+
// First, strip any existing --excluded-tools flag to make this idempotent
261+
const existingExcludedRegex = / --excluded-tools=[^\s'"]*/g;
262+
const existingMatches = content.match(existingExcludedRegex);
263+
if (existingMatches) {
264+
content = content.replace(existingExcludedRegex, '');
265+
console.log(` Removed ${existingMatches.length} existing --excluded-tools flag(s)`);
266+
}
267+
const allowAllToolsCount = (content.match(/--allow-all-tools/g) || []).length;
268+
if (allowAllToolsCount > 0) {
269+
content = content.replace(
270+
/--allow-all-tools/g,
271+
`--allow-all-tools ${excludedToolsFlag}`
272+
);
273+
modified = true;
274+
console.log(` Injected --excluded-tools (21 browser tools) in ${allowAllToolsCount} location(s)`);
275+
}
276+
}
277+
247278
// Remove unused "Setup Scripts" step from update_cache_memory jobs.
248279
// The step downloads a private action but is never used in these jobs,
249280
// causing 401 Unauthorized failures when permissions: {} is set.

0 commit comments

Comments
 (0)