mission_control-jobs fails with new Rails 8.0.0.beta1

[kbachl]

1. Create new rails app with 8.0.0.beta1, add dummy job;
2. Add gem "solid_queue", ">= 1.0.0"
3. Mount under e.g. /jobs -> mount MissionControl::Jobs::Engine, at: "/jobs"
4. Open up /jobs

Error comes:

MissionControl::Jobs::Errors::IncompatibleAdapter in MissionControl::Jobs::QueuesController#index
Adapter Async must implement queue_names

private
def raise_incompatible_adapter_error_from(method_name)
raise MissionControl::Jobs::Errors::IncompatibleAdapter, "Adapter #{ActiveJob.adapter_name(self)} must implement #{method_name}"
end
end

mission_control-jobs (0.3.2) lib/mission_control/jobs/adapter.rb:154:in raise_incompatible_adapter_error_from' mission_control-jobs (0.3.2) lib/mission_control/jobs/adapter.rb:91:in queues'
mission_control-jobs (0.3.2) lib/active_job/querying.rb:24:in `fetch_queues'

[Roupiye]

having the same problem

[Roupiye]

but i am using rails 7.2.1

[Funcke]

In config/envornments/development.rb what's your config.active_job.queue_adapter value? If it's not set it will use :async.
If you are setting it manually like this
config.active_job.queue_adapter = :solid_queue
It should work.
Additionally, make sure you have configured the database connection you want to use for your queue. In rails 8.0.0.beta1 it could look something like this:

# database.yml
default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  cache:
    <<: *default
    database: storage/development_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/development_cable.sqlite3
    migrations_paths: db/cable_migrate

And for defining the correct connection you would need to adapt your config/environments/development.rb by adding:
config.solid_queue.connects_to = { database: { writing: :queue, reading: :queue } }

[Funcke]

Also, quick edit:
in rails 8.0.0.beta1 you have to migrate the solid_queue migrations on the connection you have dedicated it for manually by running bin/rails db:migrate:<connection_name> in the case above it would be bin/rails db:migrate:queue.

[kbachl]

Hi,

I'm a bit puzzled at the moment. I indeed did forget to add
config.active_job.queue_adapter = :solid_queue
to development.rb.

The database.yml itself looks plain, as it was created by rails 8 beta 1:

default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: storage/development.sqlite3

My jobs also work and run when I call them. However, even I did a bin/rails db:migrate command, I now get an error saying:

ActiveRecord::StatementInvalid in MissionControl::Jobs::QueuesController#index
Could not find table 'solid_queue_jobs'

[kbachl]

PS: after adding config.active_job.queue_adapter = :solid_queue to development.rb my jobs also fail with same error

It seems that rails beta1 somehow makes solid-queue a first citizen but doesnt configure it enough to be ready to use even it seems so partially... need to look deeper into it

[Funcke]

Hi,
the second error indicates that the solid queue tables have not been created in the database.

Make sure you have set up a solid queue database for development that uses the data structure provided in db/queue_migrations.rb as mentioned in the example above.

Since the generated development configuration in a new rails 8.0.0.beta1 app uses the :async adapter for solid_queue by default, a corresponding database connection supporting the solid_queue schema has to be provided manually, as well as running the migrations.

[mabras]

My jobs also work and run when I call them. However, even I did a bin/rails db:migrate command, I now get an error saying:

ActiveRecord::StatementInvalid in MissionControl::Jobs::QueuesController#index
Could not find table 'solid_queue_jobs'

# config/environments/development.rb
config.solid_queue.connects_to = { database: { writing: :queue } }

[Funcke]

Hi @mabras,
What‘s your corresponding database configuration and how did you executw the expected job?

[kbachl]

Ok, so I did a "rails solid_queue:install" and it creates a "queu_schema.rb" that has much in it, e.g.:

ActiveRecord::Schema[7.1].define(version: 1) do
  create_table "solid_queue_blocked_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "queue_name", null: false
    t.integer "priority", default: 0, null: false
    t.string "concurrency_key", null: false
    t.datetime "expires_at", null: false
    t.datetime "created_at", null: false
    t.index [ "concurrency_key", "priority", "job_id" ], name: "index_solid_queue_blocked_executions_for_release"
    t.index [ "expires_at", "concurrency_key" ], name: "index_solid_queue_blocked_executions_for_maintenance"
    t.index [ "job_id" ], name: "index_solid_queue_blocked_executions_on_job_id", unique: true
  end

  create_table "solid_queue_claimed_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.bigint "process_id"
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_claimed_executions_on_job_id", unique: true
    t.index [ "process_id", "job_id" ], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id"
  end

...
  add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
end

However, when I then run rails db:migrate:queue it gets replaced by this:

# This file is auto-generated from the current state of the database. Instead
# of editing this file, please use the migrations feature of Active Record to
# incrementally modify your database, and then regenerate this schema definition.
#
# This file is the source Rails uses to define your schema when running `bin/rails
# db:schema:load`. When creating a new database, `bin/rails db:schema:load` tends to
# be faster and is potentially less error prone than running all of your
# migrations from scratch. Old migrations may fail to apply correctly if those
# migrations use external dependencies or application code.
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema[8.0].define(version: 0) do
end

And nothing in the queue database changes at all. What exactly goes wrong here? Am I supposed to copy that over to a migration file?

[kbachl]

PS: i also tried a

rails db:schema:load:queue --trace

** Invoke db:schema:load:queue (first_time)
** Invoke db:test:purge:queue (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:load_config
** Invoke db:check_protected_environments (first_time)
** Invoke db:load_config 
** Execute db:check_protected_environments
** Execute db:test:purge:queue
bin/rails aborted!
TypeError: Invalid type for configuration. Expected Symbol, String, or Hash. Got nil (TypeError)

        raise TypeError, "Invalid type for configuration. Expected Symbol, String, or Hash. Got #{config.inspect}"
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/home/user/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-8.0.0.beta1/lib/active_record/database_configurations.rb:183:in `resolve'

[kbachl]

OK! I got it working on rails 8 Beta1!

The step is simply: DONT USE rails solid_queue:install - but instead add the part of the production.rb to your development.rb:

config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }

Then delete all previous rails queue DBs and config it in your database.yml that way:

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate

after this make sure you got the original "queue_schema.rb" file from rails 8!

Then do a rails db:prepare , NOT A rails db:migrate:queue as this will create an empty schema.rb file instead!
Then after the db prepare has run you can use the db:migrate as it then will recreate it from the datase itself.

Quite an annoying path to take to have this working in dev...

[rosa]

@kbachl, this behaviour is caused by a bug in Rails that has already been fixed: rails/rails#52829. In general you don't need to go through all that trouble to get Solid Queue working in dev.

As for the error with the async adapter, yes, that's pending: #32

[kbachl]

@rosa thanks for the info!
I cant believe I hit that bug...

[igrigorik]

Hit the same issue. @kbachl's fix worked — ty.

@rosa what's the best to unwind and reset? Would prefer not to carry this forward in existing project.