|
| 1 | +--- |
| 2 | +name: galaxy-test-runner |
| 3 | +description: "Run Galaxy test suites using ./run_tests.sh. Use this skill whenever the user asks to run tests, verify a fix, check if tests pass, or validate changes in the Galaxy codebase. This includes unit tests, API tests, integration tests, framework tool tests, framework workflow tests, selenium tests, and toolshed tests. Trigger on any mention of running tests, test failures, pytest, or test validation — even if the user just says 'run the tests' or 'does this pass?'" |
| 4 | +--- |
| 5 | + |
| 6 | +# Galaxy Test Runner |
| 7 | + |
| 8 | +Galaxy uses `./run_tests.sh` as the primary entry point for all test types. Always prefer it over running pytest directly — it handles virtualenv setup, dependency installation, common startup, and environment configuration that raw pytest would miss. |
| 9 | + |
| 10 | +## Choosing the right test type |
| 11 | + |
| 12 | +Pick the test type based on what changed and what the user is asking: |
| 13 | + |
| 14 | +| What changed | Test type | Flag | |
| 15 | +|---|---|---| |
| 16 | +| Backend logic, models, utilities | Unit | `-unit` | |
| 17 | +| API behavior, endpoint responses | API | `-api` | |
| 18 | +| Tool XML, tool evaluation | Framework | `-framework` | |
| 19 | +| Workflow evaluation logic | Framework workflows | `-framework-workflows` | |
| 20 | +| Special Galaxy configurations | Integration | `-integration` | |
| 21 | +| UI behavior | Selenium | `-selenium` | |
| 22 | +| ToolShed functionality | ToolShed | `-toolshed` | |
| 23 | + |
| 24 | +If you're unsure, look at the test file's location: |
| 25 | +- `test/unit/` or doctests in `lib/` → `-unit` |
| 26 | +- `lib/galaxy_test/api/` → `-api` |
| 27 | +- `test/functional/tools/` → `-framework` |
| 28 | +- `test/integration/` → `-integration` |
| 29 | +- `lib/galaxy_test/selenium/` → `-selenium` |
| 30 | + |
| 31 | +## Command patterns |
| 32 | + |
| 33 | +### Running a specific test file |
| 34 | +```bash |
| 35 | +./run_tests.sh -api lib/galaxy_test/api/test_workflows.py |
| 36 | +``` |
| 37 | + |
| 38 | +### Running a specific test class |
| 39 | +```bash |
| 40 | +./run_tests.sh -api lib/galaxy_test/api/test_tools.py::TestToolsApi |
| 41 | +``` |
| 42 | + |
| 43 | +### Running a specific test method |
| 44 | +```bash |
| 45 | +./run_tests.sh -api lib/galaxy_test/api/test_tools.py::TestToolsApi::test_map_over_with_output_format_actions |
| 46 | +``` |
| 47 | + |
| 48 | +### Using pytest selectors with -k |
| 49 | +Pass extra pytest args after `--`: |
| 50 | +```bash |
| 51 | +./run_tests.sh -api lib/galaxy_test/api/test_workflows.py -- -k "test_run_workflow" -s |
| 52 | +``` |
| 53 | + |
| 54 | +### Unit tests with a specific path |
| 55 | +```bash |
| 56 | +./run_tests.sh -unit test/unit/data/test_galaxy_mapping.py |
| 57 | +``` |
| 58 | + |
| 59 | +### Framework tool tests by tool ID |
| 60 | +```bash |
| 61 | +./run_tests.sh -framework -id <tool_id> |
| 62 | +``` |
| 63 | + |
| 64 | +### Framework workflow tests by name |
| 65 | +```bash |
| 66 | +./run_tests.sh -framework-workflows -id <workflow_name> |
| 67 | +``` |
| 68 | + |
| 69 | +## Useful extra flags |
| 70 | + |
| 71 | +- `-- -s` — show stdout/stderr (disable capture), useful for debugging |
| 72 | +- `-- -x` — stop on first failure |
| 73 | +- `-- -k "expression"` — select tests by name expression |
| 74 | +- `--verbose_errors` — more verbose error reporting |
| 75 | +- `--skip_flakey_fails` — skip known flakey tests |
| 76 | + |
| 77 | +## Important notes |
| 78 | + |
| 79 | +- API and integration tests start a full Galaxy server instance, so they take longer. Unit tests are fast. |
| 80 | +- The `--` separator is needed to pass extra pytest arguments through `run_tests.sh`. |
| 81 | +- Framework tests use `-id` (not `-k`) to select specific tools. |
| 82 | +- If a test needs output visibility for debugging, always add `-- -s`. |
| 83 | +- Tests produce an HTML report file (printed at the end of the run). The user rarely needs this but it's there if needed. |
| 84 | + |
| 85 | +## Finding the right test to run |
| 86 | + |
| 87 | +When the user says "run the tests" without specifying which ones, figure out what they changed: |
| 88 | + |
| 89 | +1. Check `git diff` or recent file edits to see what was modified |
| 90 | +2. Search for existing tests that cover the changed code — look in the corresponding test directory |
| 91 | +3. If modifying `lib/galaxy/`, look for tests in `test/unit/`, `lib/galaxy_test/api/`, or `test/integration/` |
| 92 | +4. Run the most specific test possible rather than a whole suite — faster feedback |
0 commit comments