Skip to content

Commit 4415eeb

Browse files
vim-zzvim-zz
authored
Merge pull request #716 from linear-b/add-rules-to-label-ai-usage
Add AI Rules section for Copilot and Cursor PRs
2 parents 0c6d20c + 7f3f76d commit 4415eeb

File tree

6 files changed

+252
-2
lines changed

6 files changed

+252
-2
lines changed

‎docs/automations/integrations/copilot/flag-copilot-pr/README.md‎

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,48 @@ starter_kits: [genai]
88
<!-- --8<-- [start:example]-->
99
Automatically apply labels to PRs that are assisted by GitHub Copilot. You can apply labels based on a known list of Copilot users, PR tags, or by prompting the PR author to indicate if they used Copilot.
1010

11-
=== "Label by Prompt"
11+
=== "Label by AI Rules"
12+
Automatically apply labels to PRs based on Copilot AI rules, providing a data-driven approach to track developer adoption of AI tools without requiring manual input from developers.
13+
14+
This solution automatically tracks AI tool usage without requiring developers to fill out surveys or take manual actions. Each AI-assisted edit is logged with unique identifiers in structured JSON format, setting the foundation for analytics across different AI tools while maintaining a consistent data model.
15+
16+
!!! info "Configuration Description"
17+
Conditions:
18+
19+
* A PR is created
20+
* Copilot AI was used in this PR
21+
* Copilot rule file was added to the repo
22+
* Changes to `.cm/ai.log` file are detected (this file contains entries for each AI-assisted action)
23+
24+
Automation Actions:
25+
26+
* PR is labeled with `ai-assisted`
27+
* Structured data about AI usage is logged for future analytics
28+
29+
!!! example "`.github/copilot-instructions.md`"
30+
```yaml+jinja
31+
--8<-- "docs/downloads/automation-library/integrations/copilot/copilot-instructions.md"
32+
```
33+
<div class="result" markdown>
34+
<span>
35+
[:octicons-download-24: Download Cursor rules file.](/downloads/automation-library/integrations/copilot/copilot-instructions.md){ .md-button }
36+
</span>
37+
</div>
38+
39+
!!! example "PR is labled with AI usage"
40+
```yaml+jinja
41+
--8<-- "docs/downloads/automation-library/integrations/copilot/label_copilot_by_rule.cm"
42+
```
43+
<div class="result" markdown>
44+
<span>
45+
[:octicons-download-24: Download this example as a CM file.](/downloads/automation-library/integrations/copilot/label_copilot_by_rule.cm){ .md-button }
46+
</span>
47+
</div>
48+
49+
!!! tip
50+
Add `.cm/ai.log linguist-generated=true` to your `.gitattributes` file to ensure that Copilot-generated files are not included in your repository.
51+
52+
=== "Label by Survey"
1253
Prompt PR authors to indicate if they used Copilot for the PR and automatically label the PR if they did. This requires two separate automation files to handle posting the prompt and labeling accordingly.
1354

1455
![Label Copilot by Prompt](/automations/integrations/copilot/flag-copilot-pr/label-copilot-by-prompt.png)

‎docs/automations/integrations/cursor/flag-cursor-pr/README.md‎

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,49 @@ category: [quality, genai, cursor, quickstart]
88
<!-- --8<-- [start:example]-->
99
Automatically apply labels to Pull Requests that are assisted by Cursor AI. This automation helps track the impact and usage of Cursor's AI capabilities across your development workflow.
1010

11-
=== "Label by Prompt"
11+
=== "Label by AI Rules"
12+
Automatically apply labels to PRs based on Cursor AI rules, providing a data-driven approach to track developer adoption of AI tools without requiring manual input from developers.
13+
14+
This solution automatically tracks AI tool usage without requiring developers to fill out surveys or take manual actions. Each AI-assisted edit is logged with unique identifiers in structured JSON format, setting the foundation for analytics across different AI tools while maintaining a consistent data model.
15+
16+
!!! info "Configuration Description"
17+
Conditions:
18+
19+
* A PR is created
20+
* Cursor AI was used in this PR
21+
* Cursor rule file was added to the repo
22+
* Changes to `.cm/ai.log` file are detected (this file contains entries for each AI-assisted action)
23+
24+
Automation Actions:
25+
26+
* PR is labeled with `ai-assisted`
27+
* Structured data about AI usage is logged for future analytics
28+
29+
!!! example "`.cursor/rules/cursor-ai-logging.mdc`"
30+
```yaml+jinja
31+
--8<-- "docs/downloads/automation-library/integrations/cursor/cursor-ai-logging.mdc"
32+
```
33+
<div class="result" markdown>
34+
<span>
35+
[:octicons-download-24: Download Cursor rules file.](/downloads/automation-library/integrations/cursor/cursor-ai-logging.mdc){ .md-button }
36+
</span>
37+
</div>
38+
39+
!!! example "PR is labled with AI usage"
40+
```yaml+jinja
41+
--8<-- "docs/downloads/automation-library/integrations/cursor/label_cursor_by_rule.cm"
42+
```
43+
<div class="result" markdown>
44+
<span>
45+
[:octicons-download-24: Download this example as a CM file.](/downloads/automation-library/integrations/cursor/label_cursor_by_rule.cm){ .md-button }
46+
</span>
47+
</div>
48+
49+
!!! tip
50+
Add `.cm/ai.log linguist-generated=true` to your `.gitattributes` file to ensure that AI-generated files are marked as generated content.
51+
52+
53+
=== "Label by Survey"
1254
Prompt PR authors to indicate if they used Cursor for the PR and automatically label the PR if they did. This requires two separate automation files to handle posting the prompt and labeling accordingly.
1355

1456
![Label Cursor by Prompt](/automations/integrations/cursor/flag-cursor-pr/label-cursor-by-prompt.png)

‎docs/downloads/automation-library/integrations/copilot/copilot-instructions.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Copilot Instructions for Tracking AI Usage
2+
3+
## Purpose
4+
5+
These instructions ensure that all edits, file creations, and terminal commands are logged accurately in `.cm/ai.log` with strict verification. Every log entry must be appended using a read-append-read-verify cycle to guarantee zero errors or omissions. Never deviate from these steps.
6+
7+
## Steps
8+
9+
1. Check for `.cm/ai.log`:
10+
11+
- If the directory does not exist, create it.
12+
- If the file does not exist, create it.
13+
14+
2. Log Every Action:
15+
16+
- After editing or creating a file, you must log the action.
17+
- The log entry must be one line, added to the bottom of the file on a new line, with only these fields:
18+
19+
```jinja
20+
[{{ timestamp }}] tool="copilot" model="{{ model }}"
21+
```
22+
23+
- timestamp: Timestamp of the action, in ISO 8601 datetime format with space separator (UTC), i.e. YYYY-MM-DD HH:MM:SS
24+
- tool: Ensure the tool name is "copilot"
25+
- model: The current model id configured for the chat session, e.g. "claude-3.5-sonnet".
26+
- Add quotes around String values.
27+
- Never add unexpected fields like "action", or "file".
28+
- The logging mechanism must not skip or suppress log entries that are identical to the previous entry.
29+
- Every action must result in a new line in the log.
30+
31+
3. Read-Append-Read-Verify Cycle (MANDATORY):
32+
33+
- Before appending:
34+
- Read and store the entire current content of `.cm/ai.log` in memory.
35+
- Append:
36+
- Use `insert_edit_into_file` to append the new log entry in the required format.
37+
- Append a new entry for every action, regardless of previous entries.
38+
- Never attempt to summarize, deduplicate, or add extra fields.
39+
- Strictly follow the read-append-read-verify cycle, even if the entry is identical to the previous one.
40+
- After appending:
41+
- Read the entire `.cm/ai.log` file again.
42+
- Verify:
43+
- All previous content is unchanged.
44+
- The new entry is appended as a new line that is correctly formatted.
45+
- No unexpected fields or formatting errors are present.
46+
- If any issue is detected:
47+
- Restore the file to the original content.
48+
- Re-append the correct log entry.
49+
- Repeat the verification until the log is correct.
50+
51+
4. Handle Errors:
52+
53+
If an error occurs (e.g., overwriting the file, appending an incorrect entry, or verification fails):
54+
- Restore the original content from memory.
55+
- Append the correct log entry again.
56+
- Re-verify as above.
57+
58+
5. Ensure Accuracy:
59+
60+
- Double-check that the `tool` name is exactly "copilot".
61+
- Use consistent formatting for all log entries.
62+
- Do not proceed to any other action until verification passes.
63+
64+
## Notes
65+
66+
- The read-append-read-verify cycle is mandatory and must never be skipped.
67+
- If in doubt, refer to the examples provided above.
68+
- These steps guarantee that `.cm/ai.log` is always accurate and compliant with logging requirements.
69+
- Treat this rule as the single source of truth and never rely on memory or previous behavior for critical values.

‎docs/downloads/automation-library/integrations/copilot/label_copilot_by_rule.cm‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
manifest:
2+
version: 1.0
3+
4+
automations:
5+
label_ai_tool_usage:
6+
if:
7+
- {{ files | includes(term='.cm/ai.log') }}
8+
run:
9+
- action: add-labels@v1
10+
args:
11+
labels:
12+
- ai-assisted

‎docs/downloads/automation-library/integrations/cursor/cursor-ai-logging.mdc‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
description: Log all Cursor actions
3+
globs:
4+
alwaysApply: true
5+
---
6+
# Cursor Instructions for Tracking AI Usage
7+
8+
## Purpose
9+
10+
These instructions ensure that all edits, file creations, and terminal commands are logged accurately in `.cm/ai.log` with strict verification. Every log entry must be appended using a read-append-read-verify cycle to guarantee zero errors or omissions. Never deviate from these steps.
11+
12+
## Steps
13+
14+
1. Check for `.cm/ai.log`:
15+
16+
- If the directory does not exist, create it.
17+
- If the file does not exist, create it.
18+
19+
2. Log Every Action:
20+
21+
- After editing or creating a file, you must log the action.
22+
- The log entry must be one line, added to the bottom of the file on a new line, with only these fields:
23+
24+
```jinja
25+
[{{ timestamp }}] tool="cursor" model="{{ model }}"
26+
```
27+
28+
- timestamp: Timestamp of the action, in ISO 8601 datetime format with space separator (UTC), i.e. YYYY-MM-DD HH:MM:SS
29+
- tool: Ensure the tool name is "cursor"
30+
- model: The current model id configured for the chat session, e.g. "claude-3.5-sonnet".
31+
- Add quotes around String values.
32+
- Never add unexpected fields like "action", or "file".
33+
- The logging mechanism must not skip or suppress log entries that are identical to the previous entry.
34+
- Every action must result in a new line in the log.
35+
36+
3. Read-Append-Read-Verify Cycle (MANDATORY):
37+
38+
- Before appending:
39+
- Read and store the entire current content of `.cm/ai.log` in memory.
40+
- Append:
41+
- Use `insert_edit_into_file` to append the new log entry in the required format.
42+
- Append a new entry for every action, regardless of previous entries.
43+
- Never attempt to summarize, deduplicate, or add extra fields.
44+
- Strictly follow the read-append-read-verify cycle, even if the entry is identical to the previous one.
45+
- After appending:
46+
- Read the entire `.cm/ai.log` file again.
47+
- Verify:
48+
- All previous content is unchanged.
49+
- The new entry is appended as a new line that is correctly formatted.
50+
- No unexpected fields or formatting errors are present.
51+
- If any issue is detected:
52+
- Restore the file to the original content.
53+
- Re-append the correct log entry.
54+
- Repeat the verification until the log is correct.
55+
56+
4. Handle Errors:
57+
58+
If an error occurs (e.g., overwriting the file, appending an incorrect entry, or verification fails):
59+
- Restore the original content from memory.
60+
- Append the correct log entry again.
61+
- Re-verify as above.
62+
63+
5. Ensure Accuracy:
64+
65+
- Double-check that the `tool` name is exactly "cursor".
66+
- Use consistent formatting for all log entries.
67+
- Do not proceed to any other action until verification passes.
68+
69+
## Notes
70+
71+
- The read-append-read-verify cycle is mandatory and must never be skipped.
72+
- If in doubt, refer to the examples provided above.
73+
- These steps guarantee that `.cm/ai.log` is always accurate and compliant with logging requirements.
74+
- Treat this rule as the single source of truth and never rely on memory or previous behavior for critical values.

‎docs/downloads/automation-library/integrations/cursor/label_cursor_by_rule.cm‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
manifest:
2+
version: 1.0
3+
4+
automations:
5+
label_ai_tool_usage:
6+
if:
7+
- {{ files | includes(term='.cm/ai.log') }}
8+
run:
9+
- action: add-labels@v1
10+
args:
11+
labels:
12+
- ai-assisted

0 commit comments

Comments
 (0)