Update how we run PHPUnit by dkotter · Pull Request #1053 · 10up/classifai

dkotter · 2026-02-12T22:48:31Z

Description of the Change

Our PHPUnit workflow has started to fail and in investigating that, seems GitHub has updated their default version of Docker that is installed in their runners and that is conflicting with the tool we use to run our tests. That tool hasn't been updated since June 2024 so I'm not confident they'll push out a fix.

Also, there's a better way to run PHPUnit tests now that didn't exist/we didn't know about when we set this up (using wp-env) so this is a good opportunity to clean up that setup.

The following updates are made:

We update our GitHub Action workflow to use the wp-env environment to run our PHPUnit tests (and run those on a wider matrix)
Update our various config and bootstrap files as needed
Fixed a few tests that started failing now that we're running tests (both unit and E2E) with debug turned on
Rename our test directory to Integration (previously was Classifai)

Note most of this was copied from the approach we take on the AI Experiments plugin.

How to test the Change

Ensure the PHPUnit GitHub Action runs and passes on this PR

Changelog Entry

Developer - Update our PHPUnit workflow

Credits

Props @dkotter

Checklist:

I agree to follow this project's Code of Conduct.
I have updated the documentation accordingly.
I have added Critical Flows, Test Cases, and/or End-to-End Tests to cover my change.
All new and existing tests pass.

github-actions · 2026-02-12T22:52:15Z

✅ WordPress Plugin Check Report

✅ Status: Passed

📊 Report

All checks passed! No errors or warnings found.

🤖 Generated by WordPress Plugin Check Action • Learn more about Plugin Check

…that were causing tests to fail

peterwilsoncc

A couple of questions and a minor quibble inline.

PHP Unit is generating the files and folders below that will need to be added the ignore file.

tests/_output/html/
tests/_output/php-coverage.xml

I'm able to get the tests running using both Chassis and WP-Env. On WP-Env I'm getting the message Warning: No code coverage driver available so you may need to add something to the config file.

peterwilsoncc · 2026-02-13T01:05:04Z

.github/workflows/phpunit.yml

+            include:
+                - php: '8.4'
+                  wp: latest
+                  coverage: false


This appears to be covered by the matrix, why is it needed?

Originally I had coverage: true here, so the coverage report would be generated only on PHP 8.4 and WP latest. But our test coverage is currently so low that the report isn't super useful so I changed this to false and yeah, no need to have this now

Removed in f18f9a6

peterwilsoncc · 2026-02-13T01:21:22Z

tests/test-plugin/e2e-test-plugin.php

 		$response = file_get_contents( __DIR__ . '/image_analyze.json' );
 	} elseif ( strpos( $url, 'http://e2e-test-image-processing.test/vision/v3.2/generateThumbnail' ) !== false ) {
-		$response = file_get_contents( __DIR__ . '../classifai/assets/img/icon256x256.png' );
+		$response = file_get_contents( '/var/www/html/wp-content/plugins/classifai/assets/img/icon-256x256.png' );


As an open source project, I think it would be good to keep this relative to __DIR__ to account for people using different environments to run the tests on.

It's not a hill I will die on though.

The issue here is it seems this code never worked and we just never realized it. Now that our test environment runs with debug on, this started causing failures and I realized it was broken (and thus fixed it here). It seems the path traversal doesn't work (or I couldn't get it to work) so easy answer was this. But I can also just copy this image into our test directory and load it from there

Updated in 2ddd7cb

I didn't notice it yesterday but I now see the typo in the original code, it was __DIR__' . '../' rather than __DIR__ ' . '/../'

So there were two typos, a missing slash (__DIR__ . '../classifai vs __DIR__ . '/../classifai) and a missing dash in the filename (icon256x256.png vs icon-256x256.png). I swear I fixed both of those yesterday and tests were still failing locally but trying again today things work so 🤷🏻

In any case, this has been updated now to use the correct image path: 20c50a1

(p.s. thanks for sticking with this one)

composer.json

dkotter · 2026-02-13T14:49:36Z

A couple of questions and a minor quibble inline.

PHP Unit is generating the files and folders below that will need to be added the ignore file.

Added in 45bd2ce

I'm able to get the tests running using both Chassis and WP-Env. On WP-Env I'm getting the message Warning: No code coverage driver available so you may need to add something to the config file.

So this can be fixed by running npm run env start -- --xdebug=coverage when we start our test environment. I'm currently not doing that as we're not using the coverage reports for anything (and this causes things to be slightly slower). But if you think having those coverage reports is useful, I can change this.

peterwilsoncc

So this can be fixed by running npm run env start -- --xdebug=coverage when we start our test environment. I'm currently not doing that as we're not using the coverage reports for anything (and this causes things to be slightly slower). But if you think having those coverage reports is useful, I can change this.

That makes sense to me.

This looks good to merge.

dkotter added 7 commits February 12, 2026 15:39

Update our PHPUnit workflow to run tests in wp-env

9fa987f

Simplify our test bootstrap file

fd2634a

Add npm command to run tests

b04146d

Rename test directory from Classifai to Integration

2e260ee

Remove unneeded script

9f16ccd

Update our PHPUnit config and wp-evn config files

98bb679

Update composer dependencies

30efb47

dkotter added this to the 3.8.0 milestone Feb 12, 2026

dkotter self-assigned this Feb 12, 2026

dkotter requested review from a team and jeffpaul as code owners February 12, 2026 22:48

github-actions bot added the needs:code-review This requires code review. label Feb 12, 2026

Now that our test environment has debug on, fix a few debug warnings …

d6696ee

…that were causing tests to fail

dkotter requested review from peterwilsoncc and removed request for a team and jeffpaul February 13, 2026 00:16

peterwilsoncc reviewed Feb 13, 2026

View reviewed changes

dkotter added 4 commits February 13, 2026 07:33

Remove the coverage report from our workflow

f18f9a6

Move our test image into the test plugin

2ddd7cb

Remove unneeded dependency

bb61e3a

Ensure test output files are ignored

45bd2ce

Remove another unneeded line and fix spacing in our workflow

c04a1f9

dkotter requested a review from peterwilsoncc February 13, 2026 15:53

Fix file path for image. Remove image from test plugin

20c50a1

peterwilsoncc approved these changes Feb 16, 2026

View reviewed changes

peterwilsoncc merged commit 61aa6cb into develop Feb 16, 2026
28 checks passed

peterwilsoncc deleted the update/phpunit-workflow branch February 16, 2026 04:00

dkotter mentioned this pull request Feb 18, 2026

Update how we run PHPUnit 10up/safe-redirect-manager#440

Merged

4 tasks

This was referenced Feb 18, 2026

Update how we run PHPUnit 10up/restricted-site-access#388

Merged

Update how we run PHPUnit 10up/convert-to-blocks#224

Merged

Update how we run PHPUnit 10up/Ad-Refresh-Control#192

Merged

Conversation

dkotter commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of the Change

How to test the Change

Changelog Entry

Credits

Checklist:

Uh oh!

github-actions bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ WordPress Plugin Check Report

📊 Report

Uh oh!

peterwilsoncc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dkotter commented Feb 13, 2026

Uh oh!

peterwilsoncc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dkotter commented Feb 12, 2026 •

edited

Loading

github-actions bot commented Feb 12, 2026 •

edited

Loading

peterwilsoncc left a comment •

edited

Loading