Skip to content

Commit 898605f

Browse files
mvdbeekclaude
andcommitted
Add galaxy-test-runner skill
Adds an auto-triggered skill that guides Claude to use ./run_tests.sh instead of raw pytest for all Galaxy test types (unit, API, integration, framework, selenium, toolshed, framework-workflows). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent b5bfb6c commit 898605f

File tree

2 files changed

+95
-0
lines changed

2 files changed

+95
-0
lines changed

plugins/gx-arch-review/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,9 @@ galaxy-plugins/
145145
│ │ ├── py-review-code-structure.md
146146
│ │ ├── gx-vitest-review.md
147147
│ │ └── ...
148+
│ ├── skills/ # Auto-triggered skills
149+
│ │ └── galaxy-test-runner/
150+
│ │ └── SKILL.md
148151
│ └── README.md
149152
└── README.md
150153
```
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
---
2+
name: galaxy-test-runner
3+
description: "Run Galaxy test suites using ./run_tests.sh. Use this skill whenever the user asks to run tests, verify a fix, check if tests pass, or validate changes in the Galaxy codebase. This includes unit tests, API tests, integration tests, framework tool tests, framework workflow tests, selenium tests, and toolshed tests. Trigger on any mention of running tests, test failures, pytest, or test validation — even if the user just says 'run the tests' or 'does this pass?'"
4+
---
5+
6+
# Galaxy Test Runner
7+
8+
Galaxy uses `./run_tests.sh` as the primary entry point for all test types. Always prefer it over running pytest directly — it handles virtualenv setup, dependency installation, common startup, and environment configuration that raw pytest would miss.
9+
10+
## Choosing the right test type
11+
12+
Pick the test type based on what changed and what the user is asking:
13+
14+
| What changed | Test type | Flag |
15+
|---|---|---|
16+
| Backend logic, models, utilities | Unit | `-unit` |
17+
| API behavior, endpoint responses | API | `-api` |
18+
| Tool XML, tool evaluation | Framework | `-framework` |
19+
| Workflow evaluation logic | Framework workflows | `-framework-workflows` |
20+
| Special Galaxy configurations | Integration | `-integration` |
21+
| UI behavior | Selenium | `-selenium` |
22+
| ToolShed functionality | ToolShed | `-toolshed` |
23+
24+
If you're unsure, look at the test file's location:
25+
- `test/unit/` or doctests in `lib/``-unit`
26+
- `lib/galaxy_test/api/``-api`
27+
- `test/functional/tools/``-framework`
28+
- `test/integration/``-integration`
29+
- `lib/galaxy_test/selenium/``-selenium`
30+
31+
## Command patterns
32+
33+
### Running a specific test file
34+
```bash
35+
./run_tests.sh -api lib/galaxy_test/api/test_workflows.py
36+
```
37+
38+
### Running a specific test class
39+
```bash
40+
./run_tests.sh -api lib/galaxy_test/api/test_tools.py::TestToolsApi
41+
```
42+
43+
### Running a specific test method
44+
```bash
45+
./run_tests.sh -api lib/galaxy_test/api/test_tools.py::TestToolsApi::test_map_over_with_output_format_actions
46+
```
47+
48+
### Using pytest selectors with -k
49+
Pass extra pytest args after `--`:
50+
```bash
51+
./run_tests.sh -api lib/galaxy_test/api/test_workflows.py -- -k "test_run_workflow" -s
52+
```
53+
54+
### Unit tests with a specific path
55+
```bash
56+
./run_tests.sh -unit test/unit/data/test_galaxy_mapping.py
57+
```
58+
59+
### Framework tool tests by tool ID
60+
```bash
61+
./run_tests.sh -framework -id <tool_id>
62+
```
63+
64+
### Framework workflow tests by name
65+
```bash
66+
./run_tests.sh -framework-workflows -id <workflow_name>
67+
```
68+
69+
## Useful extra flags
70+
71+
- `-- -s` — show stdout/stderr (disable capture), useful for debugging
72+
- `-- -x` — stop on first failure
73+
- `-- -k "expression"` — select tests by name expression
74+
- `--verbose_errors` — more verbose error reporting
75+
- `--skip_flakey_fails` — skip known flakey tests
76+
77+
## Important notes
78+
79+
- API and integration tests start a full Galaxy server instance, so they take longer. Unit tests are fast.
80+
- The `--` separator is needed to pass extra pytest arguments through `run_tests.sh`.
81+
- Framework tests use `-id` (not `-k`) to select specific tools.
82+
- If a test needs output visibility for debugging, always add `-- -s`.
83+
- Tests produce an HTML report file (printed at the end of the run). The user rarely needs this but it's there if needed.
84+
85+
## Finding the right test to run
86+
87+
When the user says "run the tests" without specifying which ones, figure out what they changed:
88+
89+
1. Check `git diff` or recent file edits to see what was modified
90+
2. Search for existing tests that cover the changed code — look in the corresponding test directory
91+
3. If modifying `lib/galaxy/`, look for tests in `test/unit/`, `lib/galaxy_test/api/`, or `test/integration/`
92+
4. Run the most specific test possible rather than a whole suite — faster feedback

0 commit comments

Comments
 (0)