[prompt-analysis] Copilot PR Prompt Analysis - 2026-01-08

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-01-08

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 767 (76.7%) | Closed: 227 (22.7%) | Open: 6

Success Rate: 77.2% of completed PRs were merged

---

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	569	428	75.2%
Feature	471	340	72.2%
Test	298	220	73.8%
Update	428	311	72.7%
Refactor	204	145	71.1%
Documentation	375	262	69.9%
Other	190	157	82.6%

---

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 99.9 words
• Most common keywords: github, workflow, githubnext, actions, runs, https, comments, issue, command, copilot
• Pattern: Concise, specific, mentions concrete elements (files, errors, commands)

Example successful prompts:

1. PR #9326: "Format lint fix links..." → ✅ Merged

2. PR #9324: "Add a new safe output type, missing_data, similar to missing_tool but designed to allow the model to report missing data required to achieve a goal.
   -..." → ✅ Merged

3. PR #9322: "Analyze error in safe outputs step:
   ✓ Loaded and initialized handler for: create_pull_request
   Loaded 1 handler(s)
   Processing 1 message(s) in order of ..." → ✅ Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 143.4 words
• Most common keywords: workflow, github, githubnext, actions, comments, issue, runs, section, https, copilot
• Pattern: Longer, more verbose, sometimes lacks clear success criteria

Example unsuccessful prompts:

1. PR #9327: "Suggest possible workaround for obtaining resolved urls when using github mcp projoects tools:
   Since the GitHub MCP Projects tools don't directly prov..." → ❌ Closed

2. PR #9320: "## Add Explicit Engine Declarations to Workflows..." → ❌ Closed

3. PR #9319: "Meta-orchestrator workflows using the GitHub MCP tool couldn't execute gh CLI commands because GH_TOKEN was not set in the agent job environment, ..." → ❌ Closed

---

💡 Key Insights

1. Shorter prompts succeed more: Merged PRs have 43.5% shorter prompts on average (99.9 words vs 143.4 words). Concise, focused prompts perform better.
2. 'other' prompts most successful: 82.6% success rate (157/190), while 'documentation' prompts have 69.9% success rate.
3. Successful prompts mention specific elements: Top words in merged PRs include 'error, file, tests', suggesting specificity helps.
4. Overall Copilot PR success rate: 77.2% of completed PRs get merged (767 merged out of 994 completed).

---

📋 Recommendations

Based on the analysis of 1000 Copilot PRs:

1. Be concise and specific: Keep prompts under 100 words and reference specific files, functions, or error messages
2. Prefer bug fixes over docs: Bug fix prompts have 75.2% success vs documentation at 69.9%
3. Include context: Mention workflow names, file paths, or specific GitHub Actions when relevant
4. Avoid vagueness: Generic improvement requests ('clean up code', 'make better') have lower success rates

---

📊 Detailed Findings

Word Frequency Analysis

Top Words in Merged PRs

• github: 1097 occurrences
• workflow: 1030 occurrences
• githubnext: 918 occurrences
• actions: 788 occurrences
• runs: 645 occurrences
• https: 632 occurrences
• comments: 612 occurrences
• issue: 558 occurrences
• command: 502 occurrences
• copilot: 491 occurrences
• update: 460 occurrences
• plan: 456 occurrences
• section: 433 occurrences
• files: 387 occurrences
• test: 378 occurrences

Top Words in Closed PRs

• workflow: 462 occurrences
• github: 428 occurrences
• githubnext: 389 occurrences
• actions: 325 occurrences
• comments: 296 occurrences
• issue: 269 occurrences
• runs: 267 occurrences
• section: 237 occurrences
• https: 236 occurrences
• copilot: 233 occurrences
• plan: 228 occurrences
• files: 222 occurrences
• command: 222 occurrences
• update: 192 occurrences
• workflows: 187 occurrences

Prompt Length Distribution

• Merged PRs: Average 99.9 words per prompt
• Closed PRs: Average 143.4 words per prompt
• Finding: Closed PR prompts are 43.5% longer on average

---

🎯 Action Items

For developers creating Copilot PRs:

1. ✅ Write concise prompts (aim for under 100 words)
2. ✅ Be specific (mention file names, function names, error messages)
3. ✅ Focus on bug fixes (highest success rate at 75.2%)
4. ✅ Provide clear success criteria (what should the PR accomplish?)
5. ❌ Avoid vague requests (e.g., "improve documentation" without specifics)
6. ❌ Don't over-explain (concise beats verbose)

---

Data Source: 1,000 Copilot PRs from githubnext/gh-aw (last 30 days)
Analysis Date: 2026-01-08 12:00 UTC

AI generated by Copilot PR Prompt Pattern Analysis

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-01-15T12:02:53.084Z.