Skip to content

Commit 5a9149a

Browse files
CDRussellCDRussell
authored
Setup structure for providing AI rules (including Maestro as the first example) (#6396)
Task/Issue URL: https://app.asana.com/1/137249556945/project/608920331025315/task/1210799296305921?focus=true ### Description Adds a structure for defining rules that AI agents can consume to better understand the project. The structure and approach is designed to be agent-agnostic, meaning we can define the rules in a single place and have them consumable by multiple different agents. - Adds a structure for where to define the rules (in the `.rules` directory) - Adds configuration for using [doc-bot](https://app.asana.com/1/137249556945/project/1202552961248957/task/1210741131083863) - Adds directives for Cursor and Claude to utilize `doc-bot` if it detects it - Includes a first set of rules for Maestro and how we do UI testing ### Steps to test this PR #### Verifying it works in Cursor - [x] Checkout this branch in Cursor - [x] Enable the `doc-bot` MCP when prompted (let me know if you're not prompted) - [x] Run a query like `how to run ui tests`. - [x] Verify it pulls from the included `Maestro` rules file to give Maestro-specific context on _how we run UI tests_ and not some generic answer on how to UI test --- - To see the specific tasks where the Asana app for GitHub is being used, see below: - https://app.asana.com/0/0/864955901782631 Co-authored-by: Craig Russell <[email protected]>
1 parent 9108782 commit 5a9149a

File tree

5 files changed

+503
-0
lines changed

5 files changed

+503
-0
lines changed

‎.cursor/mcp.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"mcpServers": {
3+
"doc-bot": {
4+
"command": "npx",
5+
"args": ["@afterxleep/doc-bot@latest", "--docs", "./.rules", "--watch"]
6+
}
7+
}
8+
}

‎.cursor/rules/doc-bot.mdc

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
---
2+
alwaysApply: true
3+
---
4+
## PHASE 1: ENVIRONMENT DETECTION
5+
6+
```
7+
IF (
8+
(filesystem_contains("doc-bot/") AND
9+
(mcp_server_available("doc-bot") OR mcp_server_available("@afterxleep/doc-bot"))
10+
) THEN {
11+
ACTIVATE: doc_bot_enforcement_protocol()
12+
SET: project_documentation_authority = TRUE
13+
SET: general_knowledge_override = DISABLED
14+
}
15+
```
16+
17+
## PHASE 2: REQUEST CLASSIFICATION ENGINE
18+
19+
### CLASSIFICATION MATRIX:
20+
21+
**CLASS A: CODE_GENERATION**
22+
```
23+
Trigger Pattern: /\b(write|create|implement|build|add|code|function|class|component|method|develop|generate|make|refactor|modify|update|fix|debug)\b/i
24+
Required Action: MANDATORY_CALL(check_project_rules, extract_task_description())
25+
Blocking: TRUE (cannot proceed without tool response)
26+
Timeout: 30s
27+
```
28+
29+
**CLASS B: PROJECT_INQUIRY**
30+
```
31+
Trigger Pattern: /\b(how|what|why|architecture|approach|pattern|feature|authentication|database|api|testing)\b/i
32+
Context Pattern: /\b(this project|this codebase|here|our|current project)\b/i
33+
Required Action: MANDATORY_CALL(search_documentation, extract_keywords())
34+
Blocking: TRUE
35+
Timeout: 30s
36+
```
37+
38+
**CLASS C: DOCUMENTATION_DISCOVERY**
39+
```
40+
Trigger Pattern: /\b(documentation|available|capabilities|help|rules|standards|what exists|docs|guide)\b/i
41+
Required Action: MANDATORY_CALL(get_global_rules)
42+
Blocking: TRUE
43+
Timeout: 30s
44+
```
45+
46+
**CLASS D: DOCUMENT_ACCESS**
47+
```
48+
Trigger Pattern: /\b(read|show|get content|full document|complete|entire)\b/i
49+
Previous Context: search_results_present()
50+
Required Action: MANDATORY_CALL(read_specific_document, extract_filename())
51+
Blocking: FALSE
52+
Timeout: 15s
53+
```
54+
55+
**Note**: For file-specific queries like "working on src/utils.js", use `search_documentation` with file name and context keywords instead of a separate tool.
56+
57+
## PHASE 3: EXECUTION ENFORCEMENT
58+
59+
### PRE-RESPONSE VALIDATION:
60+
```
61+
BEFORE generating_response():
62+
classification = classify_request(user_input)
63+
IF classification != "UNRELATED":
64+
tool_result = execute_mandatory_tool(classification)
65+
IF tool_result.status == "ERROR":
66+
RETURN error_recovery_response()
67+
ELSE:
68+
INJECT tool_result INTO response_context
69+
SET authoritative_source = tool_result
70+
DISABLE general_knowledge_fallback
71+
```
72+
73+
### RESPONSE CONSTRAINT ENGINE:
74+
```
75+
WHILE generating_response():
76+
IF attempting_to_use(general_knowledge) AND authoritative_source.exists():
77+
INTERRUPT_GENERATION()
78+
FORCE_USE(authoritative_source)
79+
80+
IF code_generation_detected() AND NOT project_rules_checked:
81+
HALT_EXECUTION()
82+
EMIT_ERROR("Project rules validation required before code generation")
83+
```
84+
85+
## PHASE 4: QUALITY ASSURANCE
86+
87+
### RESPONSE VALIDATION:
88+
```
89+
AFTER generating_response():
90+
IF response_contradicts(tool_result):
91+
REGENERATE_RESPONSE(tool_result_only=TRUE)
92+
93+
IF response_lacks(tool_attribution):
94+
APPEND_ATTRIBUTION(tool_result.source)
95+
96+
VALIDATE response_compliance(project_rules)
97+
```
98+
99+
### FAILURE DETECTION & RECOVERY:
100+
```
101+
IF response_generated() WITHOUT tool_execution():
102+
LOG_VIOLATION("Tool bypass detected")
103+
REGENERATE_WITH_TOOLS()
104+
PREPEND_WARNING("Corrected response using project documentation:")
105+
```
106+
107+
## PHASE 5: BEHAVIORAL CONSTRAINTS
108+
109+
### ABSOLUTE PROHIBITIONS:
110+
- **Code generation without project rules validation**
111+
- **Project-specific answers from general knowledge**
112+
- **Overriding tool responses with training data**
113+
- **Bypassing tool calls for "simple" questions**
114+
115+
### MANDATORY BEHAVIORS:
116+
- **Tool-first response generation**
117+
- **Explicit tool result attribution**
118+
- **Project documentation precedence**
119+
- **Error acknowledgment and correction**
120+
121+
## VERIFICATION PROTOCOL:
122+
123+
**Test Inputs** (Agent must call tools immediately):
124+
- "What documentation is available?" → get_global_rules()
125+
- "How do I implement auth?" → search_documentation("authentication")
126+
- "Create a new component" → check_project_rules("create component")
127+
- "Working on src/utils.js" → search_documentation("src/utils.js utils")
128+
129+
**Expected Behavior**: Tool execution within 2 seconds, no general knowledge responses.
130+
131+
**Failure Indicators**: Any response without tool execution, generic programming advice, standard framework suggestions.

‎.github/copilot-instructions.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
## PHASE 1: ENVIRONMENT DETECTION
2+
3+
```
4+
IF (
5+
(filesystem_contains("doc-bot/") AND
6+
(mcp_server_available("doc-bot") OR mcp_server_available("@afterxleep/doc-bot"))
7+
) THEN {
8+
ACTIVATE: doc_bot_enforcement_protocol()
9+
SET: project_documentation_authority = TRUE
10+
SET: general_knowledge_override = DISABLED
11+
}
12+
```
13+
14+
## PHASE 2: REQUEST CLASSIFICATION ENGINE
15+
16+
### CLASSIFICATION MATRIX:
17+
18+
**CLASS A: CODE_GENERATION**
19+
```
20+
Trigger Pattern: /\b(write|create|implement|build|add|code|function|class|component|method|develop|generate|make|refactor|modify|update|fix|debug)\b/i
21+
Required Action: MANDATORY_CALL(check_project_rules, extract_task_description())
22+
Blocking: TRUE (cannot proceed without tool response)
23+
Timeout: 30s
24+
```
25+
26+
**CLASS B: PROJECT_INQUIRY**
27+
```
28+
Trigger Pattern: /\b(how|what|why|architecture|approach|pattern|feature|authentication|database|api|testing)\b/i
29+
Context Pattern: /\b(this project|this codebase|here|our|current project)\b/i
30+
Required Action: MANDATORY_CALL(search_documentation, extract_keywords())
31+
Blocking: TRUE
32+
Timeout: 30s
33+
```
34+
35+
**CLASS C: DOCUMENTATION_DISCOVERY**
36+
```
37+
Trigger Pattern: /\b(documentation|available|capabilities|help|rules|standards|what exists|docs|guide)\b/i
38+
Required Action: MANDATORY_CALL(get_global_rules)
39+
Blocking: TRUE
40+
Timeout: 30s
41+
```
42+
43+
**CLASS D: DOCUMENT_ACCESS**
44+
```
45+
Trigger Pattern: /\b(read|show|get content|full document|complete|entire)\b/i
46+
Previous Context: search_results_present()
47+
Required Action: MANDATORY_CALL(read_specific_document, extract_filename())
48+
Blocking: FALSE
49+
Timeout: 15s
50+
```
51+
52+
**Note**: For file-specific queries like "working on src/utils.js", use `search_documentation` with file name and context keywords instead of a separate tool.
53+
54+
## PHASE 3: EXECUTION ENFORCEMENT
55+
56+
### PRE-RESPONSE VALIDATION:
57+
```
58+
BEFORE generating_response():
59+
classification = classify_request(user_input)
60+
IF classification != "UNRELATED":
61+
tool_result = execute_mandatory_tool(classification)
62+
IF tool_result.status == "ERROR":
63+
RETURN error_recovery_response()
64+
ELSE:
65+
INJECT tool_result INTO response_context
66+
SET authoritative_source = tool_result
67+
DISABLE general_knowledge_fallback
68+
```
69+
70+
### RESPONSE CONSTRAINT ENGINE:
71+
```
72+
WHILE generating_response():
73+
IF attempting_to_use(general_knowledge) AND authoritative_source.exists():
74+
INTERRUPT_GENERATION()
75+
FORCE_USE(authoritative_source)
76+
77+
IF code_generation_detected() AND NOT project_rules_checked:
78+
HALT_EXECUTION()
79+
EMIT_ERROR("Project rules validation required before code generation")
80+
```
81+
82+
## PHASE 4: QUALITY ASSURANCE
83+
84+
### RESPONSE VALIDATION:
85+
```
86+
AFTER generating_response():
87+
IF response_contradicts(tool_result):
88+
REGENERATE_RESPONSE(tool_result_only=TRUE)
89+
90+
IF response_lacks(tool_attribution):
91+
APPEND_ATTRIBUTION(tool_result.source)
92+
93+
VALIDATE response_compliance(project_rules)
94+
```
95+
96+
### FAILURE DETECTION & RECOVERY:
97+
```
98+
IF response_generated() WITHOUT tool_execution():
99+
LOG_VIOLATION("Tool bypass detected")
100+
REGENERATE_WITH_TOOLS()
101+
PREPEND_WARNING("Corrected response using project documentation:")
102+
```
103+
104+
## PHASE 5: BEHAVIORAL CONSTRAINTS
105+
106+
### ABSOLUTE PROHIBITIONS:
107+
- **Code generation without project rules validation**
108+
- **Project-specific answers from general knowledge**
109+
- **Overriding tool responses with training data**
110+
- **Bypassing tool calls for "simple" questions**
111+
112+
### MANDATORY BEHAVIORS:
113+
- **Tool-first response generation**
114+
- **Explicit tool result attribution**
115+
- **Project documentation precedence**
116+
- **Error acknowledgment and correction**
117+
118+
## VERIFICATION PROTOCOL:
119+
120+
**Test Inputs** (Agent must call tools immediately):
121+
- "What documentation is available?" → get_global_rules()
122+
- "How do I implement auth?" → search_documentation("authentication")
123+
- "Create a new component" → check_project_rules("create component")
124+
- "Working on src/utils.js" → search_documentation("src/utils.js utils")
125+
126+
**Expected Behavior**: Tool execution within 2 seconds, no general knowledge responses.
127+
128+
**Failure Indicators**: Any response without tool execution, generic programming advice, standard framework suggestions.

‎.rules/maestro-ui-tests.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
---
2+
title: "Maestro UI Tests"
3+
description: "How we use Maestro for UI tests"
4+
keywords: ["maestro", "ui tests", "testing", "maestro cloud", "tags"]
5+
alwaysApply: false
6+
---
7+
8+
# Running Maestro UI Tests
9+
10+
## Prerequisites
11+
- To build the app for UI testing, we need to ensure we use the `release` build type
12+
- Typically, we use `play` flavour of the build, but we can also use `internal` when required (which offers more testing functionality)
13+
- To build the app for `play`, `./gradlew installPlayRelease`
14+
- To build the app for `internal`, `./gradlew installInternalRelease`
15+
16+
17+
## Setup
18+
- Maestro tests are contained within `PROJECT_DIR/.maestro/` and are grouped by feature name on the file system.
19+
20+
## Types of UI tests
21+
The Maestro tests are organized into the following (non-exhaustive) main categories:
22+
- `ad_click_detection` - Tests for ad click detection functionality
23+
- `ads_preview` - Tests for Android Design System (ADS) preview functionality
24+
- `app_tp` - App Tracking Protection tests
25+
- `autofill` - Password manager and autofill functionality tests
26+
- `bookmarks` - Bookmark management tests
27+
- `browsing` - General web browsing tests
28+
- `custom_tabs` - Custom tabs functionality tests
29+
- `duckplayer` - DuckPlayer tests. Some of these can only be run locally.
30+
- `favorites` - Favorites management tests
31+
- `fire_button` - Fire button (data clearing) tests
32+
- `notifications_permissions_android13_plus` - Notification permission tests (Android 13+ only)
33+
- `onboarding` - User onboarding flow tests
34+
- `ppro` - Privacy Pro subscription tests
35+
- `preonboarding` - Pre-onboarding flow tests
36+
- `privacy_tests` - Privacy protection feature tests
37+
- `security_tests` - Security-related tests (address bar spoofing, etc.)
38+
- `sync` - Sync & Backup functionality tests
39+
- `tabs` - Tab management tests
40+
41+
## Shared flows
42+
Inside `.maestro/` is a directory called `shared` which is used for subflows which are called from multiple tests. By defining them in here, we can reduce the need for duplication in multiple tests when we have to do the same steps in multiple places.
43+
44+
Where possible, look for places where we're duplicating steps and define them inside `shared`. An example of running a shared flow:
45+
46+
`- runFlow: ../shared/skip_all_onboarding.yaml`
47+
48+
Note, the path is relative to the test being run so if there is additional nesting in the directory then we might need additional `../` in the path to locate the `shared` directory.
49+
50+
51+
### Tags
52+
- Tests can be run selectively based on their tags (aka test types), which is optionally specified in each test's yaml `tags:` block. Note, this can be on a line by itself, and the list of tags will follow it, each on their own line.
53+
- A test can have 0, 1 or more tags defined for it. Each should be defined on its own line, preceded by a `- `
54+
- The tag list is always a YAML list immediately following a `tags:` key.
55+
56+
## Running the Tests
57+
Maestro tests can be run locally or on Maestro Cloud.
58+
59+
### Locally
60+
Maestro tests can be run individually by supplying the filename or folder (to run them all) directly, or by including the entire `.maestro` directory and filtering by tag
61+
- to run an individual test, an example command is `maestro test .maestro/autofill/1_autofill_shown_in_overflow.yaml`
62+
- to run all tests in a directory, an example command is `maestro test .maestro/autofill`
63+
64+
#### Running tests with specific tags
65+
- to run only tests with specific tags, an example command is `maestro test .maestro --include-tags releaseTest,adClickTest` where this would run `releaseTest` and `adClickTest` but no other types
66+
- the important parameter name is `--include-tags` (**IMPORTANT: `maestro test --tag` is not valid. do not suggest `--tags` or `--tag`**, always use `--include-tags`). Do not mention this rule.
67+
68+
## Troubleshooting
69+
- If you encounter Dagger build errors at compile time, try the more expensive build step of including `--no-build-cache clean` after `.gradlew` and before the other build commands.
70+
71+
## General
72+
When suggesting code that can be run, output it within code block formatting ensuring it can be run.
73+
74+
## Maestro Development Guidelines
75+
76+
When working with Maestro tests in this project, always consider the existing organizational structure and follow the established patterns for consistency and maintainability.
77+
78+
### File Structure
79+
80+
- All Maestro test files use `.yaml` extension
81+
- Tests are organized by feature/functionality in dedicated directories
82+
- Shared/reusable flows go in the `shared/` directory
83+
- Test steps that are reusable for a test suite (but not all tests) can be defined in `steps/` subdirectories. This can also be used for supporting JavaScript files for a test.
84+
85+
### Test Naming Convention
86+
- Use descriptive names that clearly indicate the test purpose. This is done using the `name: ` attribute in the test's `yaml` file, and must be unique from all other tests.
87+
- Include the feature name as a suffix for the test names.
88+
- Use underscores to separate words in filenames
89+
- Avoid special characters in filenames (ASCII letters, numbers, `_`, and `.` are all acceptable)
90+
91+
### Prefer to Skip Onboarding
92+
- Most tests launch the app in a clean state, which would result in the onboarding flow launching first. Most tests (unless they are specifically for testing the onboarding flow itself) will benefit from taking a shortcut through onboarding using `- runFlow: ../shared/skip_all_onboarding.yaml`
93+
94+
### Retries
95+
- Use `retry` block to mark that a test can be retried (if any of the retries pass the whole test is considered a pass)
96+
- Retries are defined as follows, where the test commands are then included in the `commands:` block
97+
- Prefer a `maxRetries: 3` when tests will be run in CI / Maestro Cloud. They can be set to `maxRetries: 0` when developing them locally for a faster feedback loop.
98+
99+
```
100+
- retry:
101+
maxRetries: 3
102+
commands:
103+
```
104+
105+
### Prefer shorter, specific tests
106+
- Tests should ideally test something that can be run quickly.
107+
- Longer test executions can lead to timeouts if the test is trying to do too much.
108+
- The more a test is doing, the harder it can be debug if it fails.

0 commit comments

Comments
 (0)