Skip to content

chore: isolate test runs with forking_test_runner#762

Draft
joshuap wants to merge 12 commits intomasterfrom
isolate-tests
Draft

chore: isolate test runs with forking_test_runner#762
joshuap wants to merge 12 commits intomasterfrom
isolate-tests

Conversation

@joshuap
Copy link
Member

@joshuap joshuap commented Dec 10, 2025

This PR uses forking_test_runner gem to run the tests, which isolates each spec file when running multiple specs together. This solves two problems:

  1. It should result in fewer race conditions due to global state
  2. It allows us to test different Rails initialization contexts in the integration tests, which would be a really big improvement over the current integration tests, which share the global Rails initialization (so you can't test different plugin initialization configs, for instance).
  3. As a bonus, it also parallelizes the tests, which makes them run faster. :)

I also cleaned up a bunch of things and tried to make the test setup more explicit. We should follow these rules in the future:

  1. The unit tests should run without external dependencies (you can mock dependencies if necessary)
  2. Use the integration tests to test external dependencies (i.e. Rails, Rack, ActiveSupport::Notifications, etc.)
  3. Use the cli tests for anything that interacts with the command line

Copilot AI review requested due to automatic review settings December 10, 2025 23:15
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes the test infrastructure by adopting the forking_test_runner gem to isolate test execution and enable parallel test runs. This approach addresses race conditions from shared global state and allows testing different Rails initialization contexts in integration tests.

Key changes include:

  • Integration of forking_test_runner with parallel execution (4 workers)
  • Refactored test organization with explicit configuration per integration spec file
  • Standardized RSpec syntax throughout the test suite
  • Cleaned up test setup to separate unit tests (no external dependencies), integration tests (framework dependencies), and CLI tests

Reviewed changes

Copilot reviewed 99 out of 99 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Rakefile Replaced RSpec task definitions with forking-test-runner invocation; added flexible spec task with suite selection
spec/spec_helper.rb Reorganized to support parallel test runners with unique Aruba directories; moved framework-specific hooks to appropriate contexts
spec/integration/rails_helper.rb Simplified Rails initialization to run once per spec file instead of using shared hooks
spec/integration/rails/*.rb Added file-level Honeybadger configuration for isolated test execution
spec/cli/*.rb Converted from feature/scenario to RSpec.describe/context; updated to use type: :aruba
spec/unit/**/*.rb Standardized to use RSpec.describe instead of bare describe
spec/support/aruba_helpers.rb Renamed module from FeatureHelpers to ArubaHelpers
gemfiles/*.gemfile Added forking_test_runner dependency; removed bump gem; added reline and standard
lib/honeybadger/rack/error_notifier.rb Removed unused require for rack/request
tools/release.rb Removed deprecated release tooling
.github/workflows/ruby.yml Updated to use new spec task syntax

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants