Skip to content

Commit 9eb2db3

Browse files
NanoSectorjrfnl
NanoSector
authored and
jrfnl
committed
feat: Introduce bashunit test suite
This introduces the bashunit test suite as requested by @jrfnl. This tests both a simple happy and a simple unhappy flow, as well as a test case for #1112.
1 parent 9171ec6 commit 9eb2db3

File tree

10 files changed

+224
-3
lines changed

10 files changed

+224
-3
lines changed

‎.github/CONTRIBUTING.md

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@ Thank you for your interest in contributing to PHP_CodeSniffer!
1818
* [Finding Something to Work on](#finding-something-to-work-on)
1919
* [Getting Started](#getting-started)
2020
* [While Working on a Patch](#while-working-on-a-patch)
21-
* [Writing Tests](#writing-tests)
21+
* [Writing Unit/Integration Tests](#writing-unitintegration-tests)
22+
* [Writing End-to-End Tests](#writing-end-to-end-tests)
2223
* [Submitting Your Pull Request](#submitting-your-pull-request)
2324
* [Licensing](#licensing)
2425

@@ -268,7 +269,7 @@ To help you with this, a number of convenience scripts are available:
268269
N.B.: You can ignore any skipped tests as these are for external tools.
269270

270271

271-
### Writing Tests
272+
### Writing Unit/Integration Tests
272273

273274
Tests for the PHP_CodeSniffer engine can be found in the `tests/Core` directory.
274275
Tests for individual sniffs can be found in the `src/Standards/[StandardName]/Tests/[Category]/` directory.
@@ -376,7 +377,7 @@ To run the tests specific to the use of `PHP_CODESNIFFER_CBF === true`:
376377
vendor/bin/phpunit --group CBF --exclude-group nothing
377378
```
378379

379-
#### Other notes about writing tests
380+
#### Other notes about writing unit/integration tests
380381

381382
* The `Config` class uses a number of static properties and can have a performance impact on the tests too.
382383
To get round both these issues, use the `ConfigDouble` class instead.
@@ -387,6 +388,24 @@ To run the tests specific to the use of `PHP_CODESNIFFER_CBF === true`:
387388
* When using data providers, define them immediately below the corresponding test method.
388389
* When a test method has only one data provider, it is considered best practice to closely couple the test and data provider methods via their names. I.e. the data provider's name should match the test method name, replacing the "test" prefix with "data". For example, the data provider for a method named `testSomething()` should be `dataSomething()`.
389390

391+
392+
### Writing End-to-End Tests
393+
394+
Bash-based end-to-end tests can be written using the [Bashunit](https://bashunit.typeddevs.com/) test tooling.
395+
396+
To install bashunit, follow the [installation guide](https://bashunit.typeddevs.com/installation).
397+
398+
You can then run the bashunit tests on Linux/Mac/WSL, like so:
399+
```bash
400+
./lib/bashunit -p tests/EndToEnd
401+
```
402+
403+
> Note: these tests will not run in the Windows native CMD shell. When on Windows, either use WSL or use the "git bash" shell.
404+
405+
When writing end-to-end tests, please use fixtures for the "files under scan" to make the tests stable.
406+
These fixtures can be placed in the `tests/EndToEnd/Fixtures` subdirectory.
407+
408+
390409
### Submitting Your Pull Request
391410

392411
Some guidelines for submitting pull requests (PRs) and improving the chance that your PR will be merged:

‎.github/workflows/end-to-end-tests.yml

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: E2E Tests
2+
3+
on:
4+
# Run on pushes to `master`/`4.0` and on all pull requests.
5+
# Prevent the build from running when there are only irrelevant changes.
6+
push:
7+
branches:
8+
- master
9+
- 4.0
10+
tags:
11+
- '**'
12+
paths-ignore:
13+
- '**.md'
14+
pull_request:
15+
# Allow manually triggering the workflow.
16+
workflow_dispatch:
17+
18+
jobs:
19+
bash-tests:
20+
# Cancels all previous runs of this particular job for the same branch that have not yet completed.
21+
concurrency:
22+
# The concurrency group contains the workflow name, job name, job index and the branch name.
23+
group: ${{ github.workflow }}-${{ github.job }}-${{ strategy.job-index }}-${{ github.ref }}
24+
cancel-in-progress: true
25+
26+
runs-on: 'ubuntu-latest'
27+
28+
strategy:
29+
matrix:
30+
php: ['5.4', '5.5', '5.6', '7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3', '8.4', '8.5']
31+
32+
# yamllint disable-line rule:line-length
33+
name: "E2E PHP: ${{ matrix.php }}"
34+
35+
continue-on-error: ${{ matrix.php == '8.5' }}
36+
37+
steps:
38+
- name: Prepare git to leave line endings alone
39+
run: git config --global core.autocrlf input
40+
41+
- name: Checkout code
42+
uses: actions/checkout@v4
43+
44+
- name: Install PHP
45+
uses: shivammathur/setup-php@v2
46+
with:
47+
php-version: ${{ matrix.php }}
48+
ini-values: "error_reporting=-1, display_errors=On"
49+
coverage: none
50+
51+
- name: "Install bashunit"
52+
shell: bash
53+
run: |
54+
curl -s https://bashunit.typeddevs.com/install.sh > install.sh
55+
chmod +x install.sh
56+
./install.sh
57+
58+
- name: "Run bashunit tests"
59+
shell: bash
60+
run: "./lib/bashunit -p tests/EndToEnd"

‎.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,5 @@
88
composer.lock
99
phpstan.neon
1010
/node_modules/
11+
/tests/EndToEnd/Fixtures/*.fixed
12+
/lib/

‎phpcs.xml.dist

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
<exclude-pattern>*/src/Standards/*/Tests/*\.(inc|css|js)$</exclude-pattern>
1212
<exclude-pattern>*/tests/Core/*/*\.(inc|css|js)$</exclude-pattern>
1313
<exclude-pattern>*/tests/Core/*/Fixtures/*\.php$</exclude-pattern>
14+
<exclude-pattern>*/tests/EndToEnd/Fixtures/*\.inc$</exclude-pattern>
1415

1516
<arg name="basepath" value="."/>
1617
<arg name="colors"/>

‎tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
<?php
2+
3+
/**
4+
* Class containing no style errors according to the end-to-end tests phpcs.xml.dist.
5+
*
6+
* @copyright 2025 PHPCSStandards and contributors
7+
* @license https://github.com/PHPCSStandards/PHP_CodeSniffer/blob/master/licence.txt BSD Licence
8+
*/
9+
10+
namespace PHP_CodeSniffer\Tests\EndToEnd\Fixtures;
11+
12+
class ClassOneWithoutStyleError
13+
{
14+
private function foo()
15+
{
16+
return 'bar';
17+
}
18+
}

‎tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
<?php
2+
3+
/**
4+
* Class containing no style errors.
5+
*
6+
* @copyright 2025 PHPCSStandards and contributors
7+
* @license https://github.com/PHPCSStandards/PHP_CodeSniffer/blob/master/licence.txt BSD Licence
8+
*/
9+
10+
namespace PHP_CodeSniffer\Tests\EndToEnd\Fixtures;
11+
12+
class ClassTwoWithoutStyleError
13+
{
14+
/**
15+
* A property.
16+
*
17+
* @var string
18+
*/
19+
private $bar = 'baz';
20+
}

‎tests/EndToEnd/Fixtures/ClassWithStyleError.inc

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
<?php
2+
3+
/**
4+
* Class containing a simple style error that phpcbf can fix.
5+
*
6+
* @copyright 2025 PHPCSStandards and contributors
7+
* @license https://github.com/PHPCSStandards/PHP_CodeSniffer/blob/master/licence.txt BSD Licence
8+
*/
9+
10+
namespace PHP_CodeSniffer\Tests\EndToEnd\Fixtures;
11+
12+
class ClassWithStyleError
13+
{
14+
/**
15+
* The bracket for this function is misaligned and this can be automatically fixed by phpcbf.
16+
*
17+
* @return string
18+
*/
19+
private function foo() {
20+
return 'bar';
21+
}
22+
}

‎tests/EndToEnd/Fixtures/endtoend.xml.dist

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
<?xml version="1.0"?>
2+
<ruleset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="EndToEndTests" xsi:noNamespaceSchemaLocation="phpcs.xsd">
3+
<description>The coding standard for end to end tests.</description>
4+
5+
<rule ref="PSR12"/>
6+
7+
<file>.</file>
8+
9+
<arg name="basepath" value="."/>
10+
<arg name="colors"/>
11+
<arg name="parallel" value="75"/>
12+
<arg value="p"/>
13+
</ruleset>

‎tests/EndToEnd/phpcbf_test.sh

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/usr/bin/env bash
2+
3+
function tear_down() {
4+
rm -r tests/EndToEnd/Fixtures/*.fixed
5+
}
6+
7+
function test_phpcbf_is_working() {
8+
OUTPUT="$(bin/phpcbf --no-cache --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc)"
9+
10+
assert_successful_code
11+
assert_contains "No violations were found" "$OUTPUT"
12+
}
13+
14+
function test_phpcbf_is_working_in_parallel() {
15+
OUTPUT="$(bin/phpcbf --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc)"
16+
17+
assert_successful_code
18+
assert_contains "No violations were found" "$OUTPUT"
19+
}
20+
21+
function test_phpcbf_returns_error_on_issues() {
22+
OUTPUT="$(bin/phpcbf --no-colors --no-cache --suffix=.fixed --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassWithStyleError.inc)"
23+
assert_exit_code 1
24+
25+
assert_contains "F 1 / 1 (100%)" "$OUTPUT"
26+
assert_contains "A TOTAL OF 1 ERROR WERE FIXED IN 1 FILE" "$OUTPUT"
27+
}
28+
29+
function test_phpcbf_bug_1112() {
30+
# See https://github.com/PHPCSStandards/PHP_CodeSniffer/issues/1112
31+
if [[ "$(uname)" == "Darwin" ]]; then
32+
# Perform some magic with `& fg` to prevent the processes from turning into a background job.
33+
assert_successful_code "$(bash -ic 'bash --init-file <(echo "echo \"Subprocess\"") -c "bin/phpcbf --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc" & fg')"
34+
else
35+
# This is not needed on Linux / GitHub Actions
36+
assert_successful_code "$(bash -ic 'bash --init-file <(echo "echo \"Subprocess\"") -c "bin/phpcbf --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc"')"
37+
fi
38+
}

‎tests/EndToEnd/phpcs_test.sh

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env bash
2+
3+
function test_phpcs_is_working() {
4+
assert_successful_code "$(bin/phpcs --no-cache --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc)"
5+
}
6+
7+
function test_phpcs_is_working_in_parallel() {
8+
assert_successful_code "$(bin/phpcs --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc)"
9+
}
10+
11+
function test_phpcs_returns_error_on_issues() {
12+
OUTPUT="$(bin/phpcs --no-colors --no-cache --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassWithStyleError.inc)"
13+
assert_exit_code 2
14+
15+
assert_contains "E 1 / 1 (100%)" "$OUTPUT"
16+
assert_contains "FOUND 1 ERROR AFFECTING 1 LINE" "$OUTPUT"
17+
}
18+
19+
function test_phpcs_bug_1112() {
20+
# See https://github.com/PHPCSStandards/PHP_CodeSniffer/issues/1112
21+
if [[ "$(uname)" == "Darwin" ]]; then
22+
# Perform some magic with `& fg` to prevent the processes from turning into a background job.
23+
assert_successful_code "$(bash -ic 'bash --init-file <(echo "echo \"Subprocess\"") -c "bin/phpcs --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc" & fg')"
24+
else
25+
# This is not needed on Linux / GitHub Actions
26+
assert_successful_code "$(bash -ic 'bash --init-file <(echo "echo \"Subprocess\"") -c "bin/phpcs --no-cache --parallel=2 --standard=tests/EndToEnd/Fixtures/endtoend.xml.dist tests/EndToEnd/Fixtures/ClassOneWithoutStyleError.inc tests/EndToEnd/Fixtures/ClassTwoWithoutStyleError.inc"')"
27+
fi
28+
}

0 commit comments

Comments
 (0)