Structured CI with bin/ci (rails#54693)

jeremy · David Heinemeier Hansson · dhh · web-flow · commit bcd5c93609c5 · 2025-03-08T10:37:23.000+01:00
* Local CI with bin/ci Introduce `bin/ci`, a low-rent local CI for new apps. `bin/ci` runs your all tests, linters, and security scanners. And it optionally signs off on your work by giving your PR a green status. This is your final check after completing a major chunk of work, before you merge your PR, etc. For everyday dev/test, use targeted `rails t …`. Use in tandem with [gh-signoff](https://github.com/basecamp/gh-signoff) to set a green PR status when `bin/ci` completes successfully. * Extract CI DSL * Beef up DSL and speed up the binstub * Less spacing * Clarify purpose * Allow steps to have multi-arg command definitions to escape nicely for system * I hate these warts so much * Include bundler-audit as default step * Fix excess whitespace in generation * No need for instance_eval * Fix that CI DSL now lives in config * Expand scope of bin/ci * Add documentation * Not just local * CI.run * Eliminates the inner report call * Ensures we exit with non-zero status on failure * Introduces success? so we can do conditional steps like signoff * Introduces heading to simplify title/subtitle output * Clarify doc * Include heading setting in the factory method * New call signature * More cleanup * Add test for ContinuousIntegration class * Give a hint as to how to run this CI flow using the binstub * Be brief * Appease Rubocop Silly rule * Show DSL * Absolutely not on this rule --------- Co-authored-by: David Heinemeier Hansson <david@37signals.com> Co-authored-by: David Heinemeier Hansson <david@hey.com>
diff --git a/.rubocop.yml b/.rubocop.yml
@@ -387,9 +387,6 @@ Performance/StringInclude:
 Minitest/AssertNil:
   Enabled: true
 
-Minitest/AssertPredicate:
-  Enabled: true
-
 Minitest/AssertRaisesWithRegexpArgument:
   Enabled: true
 
diff --git a/activesupport/lib/active_support.rb b/activesupport/lib/active_support.rb
@@ -40,6 +40,7 @@ module ActiveSupport
   autoload :CodeGenerator
   autoload :ActionableError
   autoload :ConfigurationFile
+  autoload :ContinuousIntegration
   autoload :CurrentAttributes
   autoload :Dependencies
   autoload :DescendantsTracker
diff --git a/activesupport/lib/active_support/continuous_integration.rb b/activesupport/lib/active_support/continuous_integration.rb
@@ -0,0 +1,145 @@
+# frozen_string_literal: true
+
+module ActiveSupport
+  # Provides a DSL for declaring a continuous integration workflow that can be run either locally or in the cloud.
+  # Each step is timed, reports success/error, and is aggregated into a collective report that reports total runtime,
+  # as well as whether the entire run was successful or not.
+  #
+  # Example:
+  #
+  #   ActiveSupport::ContinuousIntegration.run do
+  #     step "Setup", "bin/setup --skip-server"
+  #     step "Style: Ruby", "bin/rubocop"
+  #     step "Security: Gem audit", "bin/bundler-audit"
+  #     step "Tests: Rails", "bin/rails test test:system"
+  #
+  #     if success?
+  #       step "Signoff: Ready for merge and deploy", "gh signoff"
+  #     else
+  #       failure "Skipping signoff; CI failed.", "Fix the issues and try again."
+  #     end
+  #   end
+  #
+  # Starting with Rails 8.1, a default `bin/ci` and `config/ci.rb` file are created to provide out-of-the-box CI.
+  class ContinuousIntegration
+    COLORS = {
+      banner: "\033[1;32m",   # Green
+      title: "\033[1;35m",    # Purple
+      subtitle: "\033[1;90m", # Medium Gray
+      error: "\033[1;31m",    # Red
+      success: "\033[1;32m"   # Green
+    }
+
+    attr_reader :results
+
+    # Perform a CI run. Execute each step, show their results and runtime, and exit with a non-zero status if there are any failures.
+    #
+    # Pass an optional title, subtitle, and a block that declares the steps to be executed.
+    #
+    # Sets the CI environment variable to "true" to allow for conditional behavior in the app, like enabling eager loading and disabling logging.
+    #
+    # Example:
+    #
+    #   ActiveSupport::ContinuousIntegration.run do
+    #     step "Setup", "bin/setup --skip-server"
+    #     step "Style: Ruby", "bin/rubocop"
+    #     step "Security: Gem audit", "bin/bundler-audit"
+    #     step "Tests: Rails", "bin/rails test test:system"
+    #
+    #     if success?
+    #       step "Signoff: Ready for merge and deploy", "gh signoff"
+    #     else
+    #       failure "Skipping signoff; CI failed.", "Fix the issues and try again."
+    #     end
+    #   end
+    def self.run(title = "Continuous Integration", subtitle = "Running tests, style checks, and security audits", &block)
+      new.tap do |ci|
+        ENV["CI"] = "true"
+        ci.heading title, subtitle, padding: false
+        ci.report(title, &block)
+        abort unless ci.success?
+      end
+    end
+
+    def initialize(&block)
+      @results = []
+    end
+
+    # Declare a step with a title and a command. The command can either be given as a single string or as multiple
+    # strings that will be passed to `system` as individual arguments (and therefore correctly escaped for paths etc).
+    #
+    # Examples:
+    #
+    #   step "Setup", "bin/setup"
+    #   step "Single test", "bin/rails", "test", "--name", "test_that_is_one"
+    def step(title, *command)
+      heading title, command.join(" "), type: :title
+      report(title) { results << system(*command) }
+    end
+
+    # Returns true if all steps were successful.
+    def success?
+      results.all?(&:itself)
+    end
+
+    # Display an error heading with the title and optional subtitle to reflect that the run failed.
+    def failure(title, subtitle = nil)
+      heading title, subtitle, type: :error
+    end
+
+    # Display a colorized heading followed by an optional subtitle.
+    #
+    # Examples:
+    #
+    #   heading "Smoke Testing", "End-to-end tests verifying key functionality", padding: false
+    #   heading "Skipping video encoding tests", "Install FFmpeg to run these tests", type: :error
+    #
+    # See ActiveSupport::ContinuousIntegration::COLORS for a complete list of options.
+    def heading(heading, subtitle = nil, type: :banner, padding: true)
+      echo "#{padding ? "\n\n" : ""}#{heading}", type: type
+      echo "#{subtitle}#{padding ? "\n" : ""}", type: :subtitle if subtitle
+    end
+
+    # Echo text to the terminal in the color corresponding to the type of the text.
+    #
+    # Examples:
+    #
+    #   echo "This is going to be green!", type: :success
+    #   echo "This is going to be red!", type: :error
+    #
+    # See ActiveSupport::ContinuousIntegration::COLORS for a complete list of options.
+    def echo(text, type:)
+      puts colorize(text, type)
+    end
+
+    # :nodoc:
+    def report(title, &block)
+      Signal.trap("INT") { abort colorize(:error, "\n❌ #{title} interrupted") }
+
+      ci = self.class.new
+      elapsed = timing { ci.instance_eval(&block) }
+
+      if ci.success?
+        echo "\n✅ #{title} passed in #{elapsed}", type: :success
+      else
+        echo "\n❌ #{title} failed in #{elapsed}", type: :error
+      end
+
+      results.concat ci.results
+    ensure
+      Signal.trap("INT", "-")
+    end
+
+    private
+      def timing
+        started_at = Time.now.to_f
+        yield
+        min, sec = (Time.now.to_f - started_at).divmod(60)
+        "#{"#{min}m" if min > 0}%.2fs" % sec
+      end
+
+      def colorize(text, type)
+        "#{COLORS.fetch(type)}#{text}\033[0m"
+      end
+  end
+end
diff --git a/activesupport/test/continuous_integration_test.rb b/activesupport/test/continuous_integration_test.rb
@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+require_relative "abstract_unit"
+require "active_support/continuous_integration"
+
+class ContinuousIntegrationTest < ActiveSupport::TestCase
+  setup { @CI = ActiveSupport::ContinuousIntegration.new }
+
+  test "successful step" do
+    output = capture_io { @CI.step "Success!", "which ruby > /dev/null" }.to_s
+    assert_match(/Success! passed/, output)
+    assert @CI.success?
+  end
+
+  test "failed step" do
+    output = capture_io { @CI.step "Failed!", "which rubyxx > /dev/null" }.to_s
+    assert_match(/Failed! failed/, output)
+    assert_not @CI.success?
+  end
+
+  test "report with only successful steps combined gives success" do
+    output = capture_io do
+      @CI.report("CI") do
+        step "Success!", "which ruby > /dev/null"
+        step "Success again!", "which ruby > /dev/null"
+      end
+    end.to_s
+
+    assert_match(/CI passed/, output)
+    assert @CI.success?
+  end
+
+  test "report with successful and failed steps combined gives failure" do
+    output = capture_io do
+      @CI.report("CI") do
+        step "Success!", "which ruby > /dev/null"
+        step "Failed!", "which rubyxx > /dev/null"
+      end
+    end.to_s
+
+    assert_match(/CI failed/, output)
+    assert_not @CI.success?
+  end
+
+  test "echo uses terminal coloring" do
+    output = capture_io { @CI.echo "Hello", type: :success }.first.to_s
+    assert_equal "\e[1;32mHello\e[0m\n", output
+  end
+
+  test "heading" do
+    output = capture_io { @CI.heading "Hello", "To all of you" }.first.to_s
+    assert_match(/Hello[\s\S]*To all of you/, output)
+  end
+
+  test "failure output" do
+    output = capture_io { @CI.failure "This sucks", "But such is the life of programming sometimes" }.first.to_s
+    assert_equal "\e[1;31m\n\nThis sucks\e[0m\n\e[1;90mBut such is the life of programming sometimes\n\e[0m\n", output
+  end
+end
diff --git a/railties/CHANGELOG.md b/railties/CHANGELOG.md
@@ -1,3 +1,21 @@
+*   Introduce `bin/ci` for running your tests, style checks, and security audits locally or in the cloud.
+
+    The specific steps are defined by a new DSL in `config/ci.rb`.
+
+    ```ruby
+    ActiveSupport::ContinuousIntegration.run do
+      step "Setup", "bin/setup --skip-server"
+      step "Style: Ruby", "bin/rubocop"
+      step "Security: Gem audit", "bin/bundler-audit"
+      step "Tests: Rails", "bin/rails test test:system"
+    end
+    ```
+
+    Optionally use [gh-signoff](https://github.com/basecamp/gh-signoff) to
+    set a green PR status - ready for merge.
+
+    *Jeremy Daer*, *DHH*
+
 *   Generate session controller tests when running the authentication generator.
 
     *Jerome Dalbert*
diff --git a/railties/lib/rails/generators/rails/app/app_generator.rb b/railties/lib/rails/generators/rails/app/app_generator.rb
@@ -129,6 +129,7 @@ def config
         template "environment.rb"
         template "bundler-audit.yml"
         template "cable.yml" unless options[:update] || options[:skip_action_cable]
+        template "ci.rb"
         template "puma.rb"
         template "storage.yml" unless options[:update] || skip_active_storage?
 
@@ -141,6 +142,7 @@ def config
     def config_when_updating
       action_cable_config_exist       = File.exist?("config/cable.yml")
       active_storage_config_exist     = File.exist?("config/storage.yml")
+      ci_config_exist                 = File.exist?("config/ci.rb")
       bundle_audit_config_exist       = File.exist?("config/bundler-audit.yml")
       rack_cors_config_exist          = File.exist?("config/initializers/cors.rb")
       assets_config_exist             = File.exist?("config/initializers/assets.rb")
@@ -159,6 +161,10 @@ def config_when_updating
         template "config/storage.yml"
       end
 
+      if !ci_config_exist
+        template "config/ci.rb"
+      end
+
       if skip_asset_pipeline? && !assets_config_exist
         remove_file "config/initializers/assets.rb"
       end
diff --git a/railties/lib/rails/generators/rails/app/templates/bin/ci.tt b/railties/lib/rails/generators/rails/app/templates/bin/ci.tt
@@ -0,0 +1,5 @@
+require_relative "../config/boot"
+require "active_support/continuous_integration"
+
+CI = ActiveSupport::ContinuousIntegration
+require_relative "../config/ci.rb"
diff --git a/railties/lib/rails/generators/rails/app/templates/config/ci.rb.tt b/railties/lib/rails/generators/rails/app/templates/config/ci.rb.tt
@@ -0,0 +1,33 @@
+# Run using bin/ci
+
+CI.run do
+  step "Setup", "bin/setup --skip-server"
+<% unless options.skip_rubocop? %>
+  step "Style: Ruby", "bin/rubocop"
+<% end -%>
+
+  step "Security: Gem audit", "bin/bundler-audit"
+<% if using_node? -%>
+  step "Security: Yarn vulnerability audit", "yarn audit"
+<% end -%>
+<% if options[:javascript] == "importmap" && !options[:api] && !options[:skip_javascript] -%>
+  step "Security: Importmap vulnerability audit", "bin/importmap audit"
+<% end -%>
+<% unless options.skip_brakeman? -%>
+  step "Security: Brakeman code analysis", "bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error"
+<% end -%>
+<% if options[:api] || options[:skip_system_test] -%>
+  step "Tests: Rails", "bin/rails test"
+<% else %>
+  step "Tests: Rails", "bin/rails test test:system"
+<% end -%>
+  step "Tests: Seeds", "env RAILS_ENV=test bin/rails db:seed:replant"
+
+  # Optional: set a green GitHub commit status to unblock PR merge.
+  # Requires the `gh` CLI and and `gh extension install basecamp/gh-signoff`.
+  # if success?
+  #   step "Signoff: All systems go. Ready for merge and deploy.", "gh signoff"
+  # else
+  #   failure "Signoff: CI failed. Do not merge or deploy.", "Fix the issues and try again."
+  # end
+end
diff --git a/railties/test/application/bin_ci_test.rb b/railties/test/application/bin_ci_test.rb
@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require "isolation/abstract_unit"
+
+module ApplicationTests
+  class BinCiTest < ActiveSupport::TestCase
+    include ActiveSupport::Testing::Isolation
+
+    setup :build_app
+    teardown :teardown_app
+
+    test "bin/ci exists and is executable with default content" do
+      Dir.chdir(app_path) do
+        assert File.exist?("bin/ci"), "bin/ci does not exist"
+        assert File.executable?("bin/ci"), "bin/ci is not executable"
+
+        content = File.read("config/ci.rb")
+
+        # Default steps
+        assert_match(/bin\/rubocop/, content)
+        assert_match(/bin\/brakeman/, content)
+        assert_match(/bin\/rails test/, content)
+        assert_match(/bin\/rails db:seed:replant/, content)
+
+        # Node-specific steps excluded by default
+        assert_no_match(/yarn audit/, content)
+
+        # GitHub signoff is commented
+        assert_match(/# .*gh signoff/, content)
+      end
+    end
+  end
+end
diff --git a/railties/test/generators/app_generator_test.rb b/railties/test/generators/app_generator_test.rb
@@ -33,6 +33,7 @@
   app/views/pwa/service-worker.js
   bin/brakeman
   bin/bundler-audit
+  bin/ci
   bin/dev
   bin/docker-entrypoint
   bin/rails
@@ -45,6 +46,7 @@
   config/boot.rb
   config/bundler-audit.yml
   config/cable.yml
+  config/ci.rb
   config/credentials.yml.enc
   config/database.yml
   config/environment.rb
diff --git a/railties/test/generators/plugin_generator_test.rb b/railties/test/generators/plugin_generator_test.rb
@@ -40,6 +40,7 @@
   test/dummy/app/views/pwa/manifest.json.erb
   test/dummy/app/views/pwa/service-worker.js
   test/dummy/bin/bundler-audit
+  test/dummy/bin/ci
   test/dummy/bin/dev
   test/dummy/bin/rails
   test/dummy/bin/rake
@@ -49,6 +50,7 @@
   test/dummy/config/boot.rb
   test/dummy/config/bundler-audit.yml
   test/dummy/config/cable.yml
+  test/dummy/config/ci.rb
   test/dummy/config/database.yml
   test/dummy/config/environment.rb
   test/dummy/config/environments/development.rb
