Skip to content

Boot the Rails environment in the Resque worker so background jobs run - #490

Merged
rosa merged 1 commit into
mainfrom
writebook-resque-env
Sep 14, 2026
Merged

rosa merged 1 commit into
mainfrom
writebook-resque-env

Conversation

@jeremy

@jeremy jeremy commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

The bug

The worker process is started by the workers: line in the Procfile:

workers: FORK_PER_JOB=false INTERVAL=0.1 bundle exec resque-pool

resque-pool's CLI requires resque (via resque/pool/tasks) before it loads the Rakefile, which is what loads config/application. So at the moment resque is required, Rails::Railtie is not defined yet, and resque's own railtie — the code that would wire resque:setup => :environment — is skipped:

# resque/lib/resque.rb
require 'resque/railtie' if defined?(Rails::Railtie)

By the time rails/all finally loads, require "resque" is already a no-op, so the railtie never loads. The pool master therefore invokes resque:pool → resque:preload → resque:setup with resque:setup still a no-op, and never boots the Rails environment. resque:preload only eager-loads if Rails.application.config.eager_load, which is only true once the production environment has actually been loaded — which never happens. No ActiveJob class is ever defined in the worker, and every enqueued job dies with:

Failed to instantiate job, class `ActiveStorage::PurgeJob` doesn't exist

The most serious consequence: ActiveStorage::PurgeJob never runs, so destroying a book never purges its uploaded files. They stay on disk and remain anonymously retrievable via the public Disk URL for the life of the instance, and storage grows without bound. Turbo::Streams::BroadcastJob never runs either (collaborative-editing indicators break).

Note this only affects the resque-pool boot path used in production; a bare bundle exec rake resque:work happens to work because rake loads config/application (and thus rails/all) before requiring resque, so the railtie loads there.

The fix

Add lib/tasks/resque.rake that wires resque:setup => :environment unconditionally (independent of whether resque's railtie loaded), so the pool master boots the full Rails environment and eager-loads every job class before forking workers. Because the workers are forked from the master with FORK_PER_JOB=false, the resque:pool:setup hook hands each forked child its own Redis and database connections instead of sharing the master's.

require "resque/pool/tasks"

task "resque:setup" => :environment

task "resque:pool:setup" do
  ActiveRecord::Base.connection_handler.clear_all_connections!

  Resque::Pool.after_prefork do
    Resque.redis.reconnect
    ActiveRecord::Base.establish_connection
  end
end

Verification

Reproduced end-to-end against main in the production environment (SQLite, local Disk service, Redis), using the exact Procfile command FORK_PER_JOB=false INTERVAL=0.1 bundle exec resque-pool. In each run a throwaway ActiveStorage::Blob was uploaded and ActiveStorage::PurgeJob.perform_later(blob) enqueued, then the pool ran briefly and Resque stats + the blob's file were inspected.

Before the fix:

  • processed=1, failed=1
  • failure: Failed to instantiate job, class ActiveStorage::PurgeJob doesn't exist
  • the uploaded file was not purged (still on disk)

After the fix:

  • processed=1, failed=0
  • worker log: Performed ActiveStorage::PurgeJob ... in 52.27ms, Deleted file from key: ...
  • the uploaded file was purged (removed from disk and the blob record deleted), and the job ran in a forked pool worker with working database + Redis + disk access (9 concurrent workers)

CI gates run locally and green: bin/rubocop, bin/brakeman --no-pager (0 warnings), bin/importmap audit, bin/rails test (277 runs, 0 failures/errors), bin/rails test:system (0 failures).

Copilot AI balanced review requested due to automatic review settings September 12, 2026 19:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-12T19:44:15.182339Z d1297d3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approved

The change correctly restores Rails initialization and safely reconnects inherited resources after forking.

Pull request overview

Boots Rails for production Resque pool workers and establishes fork-safe connections.

Changes:

  • Makes resque:setup depend on Rails’ environment.
  • Reconnects Redis and Active Record in forked workers.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File summaries
File Description
lib/tasks/resque.rake Configures Rails boot and per-worker connections.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

The Procfile runs the worker with `resque-pool`, whose CLI requires
`resque` before the Rakefile loads `config/application`. At that point
`Rails::Railtie` isn't defined yet, so resque's own railtie -- the code
that would wire `resque:setup => :environment` -- is skipped, and by the
time `rails/all` loads, `require "resque"` is already a no-op. The pool
master therefore never boots the app: `resque:preload` eager-loads
nothing (eager_load is only true once the environment is loaded), so no
ActiveJob class is ever defined and every enqueued job dies with
"Failed to instantiate job, class ... doesn't exist."

Concretely, ActiveStorage::PurgeJob never runs, so destroying a book
never purges its uploaded files -- they stay on disk and remain
retrievable via the public Disk URL for the life of the instance --
and Turbo::Streams::BroadcastJob never runs either.

Wire `resque:setup => :environment` in lib/tasks/resque.rake so the pool
master boots the full Rails environment and eager-loads every job class
before forking workers, and reconnect each forked worker to its own
Redis and database connections.
@rosa
rosa force-pushed the writebook-resque-env branch from d1297d3 to c599726 Compare September 12, 2026 20:58
@jeremy

jeremy commented Sep 13, 2026

Copy link
Copy Markdown
Member Author

Review threads: 0 — Copilot approved the head and raised no threads. CI green, mergeable. Converged; merge held for the operator (Writebook is source-available, kudos-only).

@rosa
rosa merged commit 1611741 into main Sep 14, 2026
7 checks passed
@rosa
rosa deleted the writebook-resque-env branch September 14, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants