Skip to content

Add PHP language support#233

Merged
acrmp merged 18 commits intoSWE-bench:mainfrom
akhatua2:add_php
Mar 21, 2026
Merged

Add PHP language support#233
acrmp merged 18 commits intoSWE-bench:mainfrom
akhatua2:add_php

Conversation

@akhatua2
Copy link
Contributor

@akhatua2 akhatua2 commented Mar 9, 2026

Summary

  • Add PHP language support with procedural bug generation using PHP-specific tree-sitter modifiers
  • Implement 14 procedural modifiers (8 operation, 2 control flow, 4 remove) using PHP grammar
  • Add get_test_files() for PhpProfile with PHPUnit testdox name-to-file mapping
  • Include 3 PHP repo profiles: doctrine/dbal, guzzle/guzzle, Seldaek/monolog
  • Add 59 tests (33 procedural modifier + 26 profile/log parser)

Validation

Tested on 2 PHP repos and reached a pass% of 35.4% on average:

Repository          |   Gen |   Val |  Valid |  Pass%
----------------------------------------------------
doctrine/dbal       |  1099 |  1099 |    392 |  35.7%
Seldaek/monolog     |   916 |   916 |    322 |  35.2%
----------------------------------------------------
TOTAL               |  2015 |  2015 |    714 |  35.4%
----------------------------------------------------
MEAN                | 1007.5| 1007.5|  357.0 |  35.4%
STD                 |  129.4|  129.4|   49.5 |   0.4%
MIN                 |   916 |   916 |    322 |  35.2%
MEDIAN              | 1007.5| 1007.5|  357.0 |  35.4%
MAX                 |  1099 |  1099 |    392 |  35.7%

func_pm_ternary_swap and func_pm_ctrl_invert_if are the most productive modifiers in PHP. Operator and control flow mutations have higher pass rates since they introduce subtle behavioral bugs without breaking syntax.

Modifier                |   Gen |   Val |  Valid |  Pass%
---------------------------------------------------------
func_pm_arg_swap        |   187 |   187 |     71 |  38.0%
func_pm_aug_assign_swap |    39 |    39 |     19 |  48.7%
func_pm_ctrl_invert_if  |    61 |    61 |     30 |  49.2%
func_pm_ctrl_shuffle    |    73 |    73 |     21 |  28.8%
func_pm_op_break_chains |   137 |   137 |     43 |  31.4%
func_pm_op_change       |   100 |   100 |     44 |  44.0%
func_pm_op_change_const |   112 |   112 |     30 |  26.8%
func_pm_op_flip         |   299 |   299 |    107 |  35.8%
func_pm_op_swap         |   306 |   306 |     49 |  16.0%
func_pm_remove_assign   |   290 |   290 |    123 |  42.4%
func_pm_remove_cond     |   255 |   255 |    103 |  40.4%
func_pm_remove_loop     |   104 |   104 |     46 |  44.2%
func_pm_remove_ternary  |    26 |    26 |     12 |  46.2%
func_pm_ternary_swap    |    26 |    26 |     16 |  61.5%
---------------------------------------------------------
TOTAL                   |  2015 |  2015 |    714 |  35.4%

Mirror & Task Branches

Task instance branches have been pushed to the mirror repos with the standard 2-commit structure (Initial commit + Bug Patch + Remove F2P Tests).

Example: swesmith/Seldaek__monolog.6db20ca0 — func_pm_arg_swap__zwytedjj

akhatua2 and others added 2 commits March 8, 2026 22:40
- Add _analyze_properties() and complexity() to PhpEntity for AST-based
  code property tagging (control flow, operations, expressions)
- Create PHP procedural modifiers module (reuses JavaScript modifiers
  since PHP shares tree-sitter node types)
- Register .php extension in MAP_EXT_TO_MODIFIERS
- Add 5 new PHP repo profiles: Guzzle, Monolog, PHP-Parser, Carbon, PHPMailer
- Extract reusable parse_log_phpunit_testdox() and parse_log_phpunit_verbose()
  log parser functions
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9ce293b7f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codecov
Copy link

codecov bot commented Mar 9, 2026

Codecov Report

❌ Patch coverage is 98.52528% with 21 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
swesmith/bug_gen/procedural/php/operations.py 96.57% 11 Missing ⚠️
swesmith/profiles/php.py 96.15% 4 Missing ⚠️
swesmith/bug_gen/procedural/php/remove.py 98.47% 2 Missing ⚠️
tests/profiles/test_profiles_php.py 98.70% 2 Missing ⚠️
swesmith/bug_gen/adapters/php.py 98.64% 1 Missing ⚠️
swesmith/bug_gen/procedural/php/control_flow.py 99.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
swesmith/bug_gen/procedural/__init__.py 100.00% <100.00%> (ø)
swesmith/bug_gen/procedural/php/__init__.py 100.00% <100.00%> (ø)
swesmith/bug_gen/procedural/php/base.py 100.00% <100.00%> (ø)
tests/bug_gen/adapters/test_php.py 100.00% <100.00%> (ø)
tests/bug_gen/procedural/php/test_php_base.py 100.00% <100.00%> (ø)
...ts/bug_gen/procedural/php/test_php_control_flow.py 100.00% <100.00%> (ø)
...ests/bug_gen/procedural/php/test_php_operations.py 100.00% <100.00%> (ø)
tests/bug_gen/procedural/php/test_php_remove.py 100.00% <100.00%> (ø)
swesmith/bug_gen/adapters/php.py 96.47% <98.64%> (+3.72%) ⬆️
swesmith/bug_gen/procedural/php/control_flow.py 99.00% <99.00%> (ø)
... and 4 more

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

akhatua2 and others added 3 commits March 9, 2026 16:28
- Replace JS modifier reuse with 14 PHP-specific tree-sitter modifiers
  (8 operation, 2 control flow, 4 remove) using PHP grammar
- Add php_parse() helper to handle PHP's <?php tag requirement
- Implement get_test_files() for PhpProfile with testdox name mapping
- Remove PHPMailer, Carbon, and PHP-Parser profiles (keep dbal, guzzle, monolog)
- Add 59 tests (33 procedural modifier + 26 profile/log parser)
@acrmp acrmp self-assigned this Mar 10, 2026
Copy link
Collaborator

@acrmp acrmp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @akhatua2. Thanks for opening this PR!

I haven't had a chance to look over the whole PR yet but here's some initial feedback.

akhatua2 and others added 2 commits March 10, 2026 16:49
- Fix anonymous_function node type (renamed in tree-sitter-php)
- Replace invalid ||= swap with .= for ??= operator
- Skip if/elseif/else chains to avoid dropping elseif branches
- Remove arrow_function from shuffle targets (single expression)
- Add nullsafe_member_call_expression to argument swap detection
- Improve test assertions for if-else invert and shuffle modifiers
- Add parametrized examples for member and scoped call expressions
- Add method shuffle test case
Copy link
Collaborator

@acrmp acrmp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akhatua2 Thanks! Here's some comments on the operations modifiers.

Copy link
Collaborator

@acrmp acrmp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @akhatua2. I think that's everything for my comments, sorry if it's a lot. Let me know if you have any questions.

@acrmp acrmp merged commit 9b74ac0 into SWE-bench:main Mar 21, 2026
6 checks passed
@acrmp
Copy link
Collaborator

acrmp commented Mar 21, 2026

Hi @akhatua2,

Merged. Thanks!

A heads-up that there may be a potential issue with testdox class header output matching if a test is annotated with a testdox description containing parentheses:

/**
 * @testdox Returns foo (when bar is set)
 */

This doesn't seem to be a common pattern on GitHub though so I thought it better to merge the current changes as-is. If you want to re-visit we can have a separate follow-up PR.

Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants