Add bin/jobs binstub that gets copied over on installation

And a nicer Cli class to use over in the binstub with better args
instead of environment variables.

Author: rosa

README.md

@@ -2,7 +2,7 @@
 
 Solid Queue is a DB-based queuing backend for [Active Job](https://edgeguides.rubyonrails.org/active_job_basics.html), designed with simplicity and performance in mind.
 
-Besides regular job enqueuing and processing, Solid Queue supports delayed jobs, concurrency controls, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (`enqueue_all` for Active Job's `perform_all_later`). _Improvements to logging and instrumentation, a better CLI tool, a way to run within an existing process in "async" mode, and some way of specifying unique jobs are coming very soon._
+Besides regular job enqueuing and processing, Solid Queue supports delayed jobs, concurrency controls, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (`enqueue_all` for Active Job's `perform_all_later`).
 
 Solid Queue can be used with SQL databases such as MySQL, PostgreSQL or SQLite, and it leverages the `FOR UPDATE SKIP LOCKED` clause, if available, to avoid blocking and waiting on locks when polling jobs. It relies on Active Job for retries, discarding, error handling, serialization, or delays, and it's compatible with Ruby on Rails multi-threading.
 
@@ -59,14 +59,14 @@ Finally, you need to run the migrations:
 $ bin/rails db:migrate
 ```
 
-After this, you'll be ready to enqueue jobs using Solid Queue, but you need to start Solid Queue's supervisor to run them.
+After this, you'll be ready to enqueue jobs using Solid Queue, but you need to start Solid Queue's supervisor to run them. You can use the provided binstub:`
 ```bash
-$ bundle exec rake solid_queue:start
+$ bin/jobs
 ```
 
 This will start processing jobs in all queues using the default configuration. See [below](#configuration) to learn more about configuring Solid Queue.
 
-For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bundle exec rake solid_queue:start` on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the [configuration](#configuration) section for more details on this.
+For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bin/jobs` on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the [configuration](#configuration) section for more details on this.
 
 ## Requirements
 Besides Rails 7.1, Solid Queue works best with MySQL 8+ or PostgreSQL 9.5+, as they support `FOR UPDATE SKIP LOCKED`. You can use it with older versions, but in that case, you might run into lock waits if you run multiple workers for the same queue.
@@ -159,6 +159,16 @@ When receiving a `QUIT` signal, if workers still have jobs in-flight, these will
 If processes have no chance of cleaning up before exiting (e.g. if someone pulls a cable somewhere), in-flight jobs might remain claimed by the processes executing them. Processes send heartbeats, and the supervisor checks and prunes processes with expired heartbeats, which will release any claimed jobs back to their queues. You can configure both the frequency of heartbeats and the threshold to consider a process dead. See the section below for this.
 
 
+### Fork vs. async mode
+
+By default, the supervisor will fork additional processes for each worker and dispatcher so that they run in different processes. This provides the best isolation and performance, but can have additional memory usage. Alternatively, you can run all workers and dispatchers in the same process as the supervisor by passing `--mode async` when starting it:
+```
+bin/jobs --mode async
+```
+
+If you use the `async` mode, the `processes` option in the configuration will be ignored.
+
+
 ### Dedicated database configuration
 
 Solid Queue can be configured to run on a different database than the main application.

lib/generators/solid_queue/install/USAGE

@@ -7,3 +7,4 @@ Example:
     This will perform the following:
         Installs solid_queue migrations
         Replaces Active Job's adapter in environment configuration
+        Installs bin/jobs binstub to start the supervisor

lib/generators/solid_queue/install/install_generator.rb

@@ -14,8 +14,16 @@ def add_solid_queue
       end
     end
 
-    say "Copying sample configuration"
-    copy_file "config.yml", "config/solid_queue.yml"
+    if File.exist?("config/solid_queue.yml")
+      say "Skipping sample configuration as config/solid_queue.yml exists"
+    else
+      say "Copying sample configuration"
+      copy_file "config.yml", "config/solid_queue.yml"
+    end
+
+    say "Copying binstub"
+    copy_file "jobs", "bin/jobs"
+    chmod "bin/jobs", 0755 & ~File.umask, verbose: false
   end
 
   def create_migrations

lib/generators/solid_queue/install/templates/jobs

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require_relative "../config/environment"
+require "solid_queue/cli"
+
+SolidQueue::Cli.start(ARGV)

lib/solid_queue/cli.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require "thor"
+
+module SolidQueue
+  class Cli < Thor
+    class_option :config_file, type: :string, aliases: "-c", default: Configuration::DEFAULT_CONFIG_FILE_PATH, desc: "Path to config file"
+    class_option :mode, type: :string, default: "fork", enum: %w[ fork async ], desc: "Whether to fork processes for workers and dispatchers (fork) or to run these in the same process as the supervisor (async)"
+
+    def self.exit_on_failure?
+      true
+    end
+
+    desc :start, "Starts Solid Queue supervisor to dispatch and perform enqueued jobs. Default command."
+    default_command :start
+
+    def start
+      SolidQueue::Supervisor.start(mode: options["mode"], load_configuration_from: options["config_file"])
+    end
+  end
+end

test/unit/configuration_test.rb

@@ -19,6 +19,14 @@ class ConfigurationTest < ActiveSupport::TestCase
     assert_processes configuration, :dispatcher, 1, batch_size: SolidQueue::Configuration::DISPATCHER_DEFAULTS[:batch_size]
   end
 
+  test "default configuration when config given is empty" do
+    configuration = SolidQueue::Configuration.new(load_from: {})
+
+    assert_equal 2, configuration.configured_processes.count
+    assert_processes configuration, :worker, 1, queues: [ "*" ]
+    assert_processes configuration, :dispatcher, 1, batch_size: SolidQueue::Configuration::DISPATCHER_DEFAULTS[:batch_size]
+  end
+
   test "read configuration from default file" do
     configuration = SolidQueue::Configuration.new
     assert 3, configuration.configured_processes.count