Support Ruby procedural modifications#231
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 999703407c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
cosgroveb
force-pushed
the
ruby-procedural-modifiers
branch
from
9997034 to
6c1b18d
Compare
|
021fbac addresses codecov |
acrmp
left a comment
acrmp
left a comment
There was a problem hiding this comment.
Hi @cosgroveb,
This is awesome! Thanks for opening this PR! It's great to see support for Ruby-specific language features too.
I haven't had a chance to look through the whole PR yet but here's some initial feedback.
|
@acrmp thanks for the review. LLMs have such a frustrating time with ruby. Hopefully this helps make the world a better place in a small way :). Let me know if you need any further changes. |
acrmp
left a comment
acrmp
left a comment
There was a problem hiding this comment.
@cosgroveb Thanks for making those changes. Looks great!
Here's some more feedback/questions I had reviewing the remainder of the modifiers.
|
@acrmp let me know if I missed anything from your second round. Cheers! |
acrmp
left a comment
acrmp
left a comment
There was a problem hiding this comment.
@cosgroveb Thanks for making those changes! Looks good.
I set Claude to work looking for correctness issues and the main one found was treatment of nested binary expressions by the OperationSwapOperandsModifier. There's also a general issue with removal of inline conditionals/loops/rescues.
cosgroveb
force-pushed
the
ruby-procedural-modifiers
branch
3 times, most recently
from
f0926e1 to
a9ee66f
Compare
|
@acrmp I rebased and addressed today's feedback |
tree-sitter-based modifiers across 5 modules: control_flow (if/else invert, shuffle lines, guard clause invert), nil_introduction (safe navigation, or-default, presence, bang method, or-equals, nil guard), operations (change, flip, swap operands, break chains, change constants), remove (loops, conditionals, assignments, rescue/ensure), and ruby_specific (symbol/string swap, block mutation). Includes base class with _remove_matching_nodes helper and Ruby adapter property analysis.
Profiles for Faker, RuboCop, Jekyll, and 19 new repos including 3 SWE-bench_Multilingual eval_set repos (fluentd, fastlane, fpm). RSpec JSON and Minitest/test-unit log parsers with auto-detection. RSpec test path detection (_is_test_path override for spec/ and _spec.rb).
40 tests: 29 modifier tests (parametrized remove and guard clause tests, deterministic seeds, specific content assertions), 4 property analysis tests via conftest fixture, 11 profile tests covering both log parsers, parser delegation, eval_sets, and RSpec test path detection.
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
cosgroveb
force-pushed
the
ruby-procedural-modifiers
branch
from
c32d1fd to
7805146
Compare
|
Thanks for your attention in getting this PR in shape @acrmp. Replied/rebased again this morning. |
acrmp
left a comment
acrmp
left a comment
There was a problem hiding this comment.
@cosgroveb I think we're just about there, thanks for your patience!
I just left a couple of questions about the test parsing.
|
Cool. Thanks again @acrmp for the review. |
|
@cosgroveb Thanks! I left a comment on an edge case with |
|
@cosgroveb Merged! Thank you! 💎 |
2 participants
Adds 20 procedural bug modifiers, 22 repo profiles, and 40 tests for Ruby.
Builds on the Ruby tree-sitter adapter (#34, @acrmp). With these changes, SWE-smith can generate bugs for Ruby repos end-to-end.
Modifiers
20 tree-sitter-based modifiers across 5 modules:
ctrl_invert_ifctrl_shuffleguard_clause_invertif↔unlesson guard clausessafe_nav_removal&.with.or_default_removal|| defaultfallbackspresence_strip.presencecallsbang_method_stripfind!→find(allowlisted methods only)or_equals_removal||=→=nil_guard_removalreturn if x.nil?guardsop_change+→*)flip_operators==→!=,>→<)op_swapa > b→b > a)op_break_chainsa + b + c→a)op_change_constremove_loopwhile/until/forblocksremove_condif/unlessblocksremove_assignremove_rescue_ensurerescue/ensureblockssymbol_string_swap:foo↔"foo"block_mutationnil_introductionandruby_specifictarget idioms that don't exist in other languages: safe navigation, memoization via||=, symbols, blocks.Validated on RuboCop: 75.4% pass rate
Tested on rubocop/rubocop (519206df) with 175 generated patches:
ctrl_invert_ifor_default_removalguard_clause_invertop_changeflip_operatorsop_change_constop_swapsafe_nav_removalsymbol_string_swapremove_assignop_break_chainsctrl_shufflenil_guard_removalremove_loopremove_condOperator and control-flow modifiers have the highest pass rates. Removal modifiers sit lower because deleting code tends to break compilation or cause cascading failures.
Repo profiles
22 profiles (3 existing + 19 new), including 3 SWE-bench_Multilingual
eval_setsrepos:Log parsers handle RSpec JSON (
--format json) and Minitest/test-unit verbose output with auto-detection._is_test_pathoverridden to recognizespec/directories and_spec.rbfiles.Tests
40 tests across 7 files: