Skip to content

Let bin/jobs check validate another env's config section without a database - #790

Open
wintan1418 wants to merge 2 commits into
rails:mainfrom
wintan1418:validate-config-for-target-env-without-database
Open

Let bin/jobs check validate another env's config section without a database#790
wintan1418 wants to merge 2 commits into
rails:mainfrom
wintan1418:validate-config-for-target-env-without-database

Conversation

@wintan1418

Copy link
Copy Markdown
Contributor

Fixes #780

Follows the analysis in #780 (thanks @kylekeesling for the detailed report — this implements both fixes you outlined).

1. --env option for bin/jobs check

check always read the config files through the current Rails env, so a CI run in the test env couldn't inspect the production: section of config/recurring.yml — and config_from's whole-file fallback made it worse: with no test: key it treated production as a task with no :schedule, dropped it, and reported an empty configuration valid.

$ RAILS_ENV=test bin/jobs check --env production --recurring_schedule_file config/recurring.yml
Solid Queue configuration is invalid:
  Invalid recurring tasks:
  - my_task: Schedule is not a supported recurring schedule
$ echo $?
1

The option only exists on checkstart keeps running strictly under the current env.

2. check no longer requires a database

Configuration used to instantiate RecurringTask Active Record models just to list the configured processes, which needs the solid_queue_recurring_tasks schema and blew up with a raw StatementInvalid backtrace in environments with no queue database. Now it passes the scheduler raw [ key, options ] pairs — RecurringSchedule already wraps whatever it receives via RecurringTask.wrap at boot, in the running process where a database is guaranteed — so process listing is database-free.

Task-level validation (schedule syntax, job class existence) still requires the schema to build the models. When there's no usable connection, check now degrades to an explicit warning instead of crashing, mirroring the rescue the pool-size warning already had, and still validates everything else:

$ RAILS_ENV=test bin/jobs check --env production   # no queue database configured for test
Warning: recurring tasks couldn't be validated because there's no usable database connection. Run `bin/jobs check` with the Solid Queue database reachable to validate them.
Solid Queue configuration is valid.

Making task-level validation itself database-free would mean decoupling RecurringTask's validations from Active Record; that felt like a separate discussion, and the warning keeps check honest about what it did and didn't verify in the meantime.

Tests

  • --env targeting validates the production:-only recurring fixture from the test env, including instantiating the scheduler from the raw definitions.
  • With RecurringTask.from_configuration raising StatementInvalid (no schema), the scheduler is still detected from the raw configuration, check stays valid, and the warning is reported.

Active Job's retry_on and rescue_from only hook into exceptions raised
while perform runs. ProcessPrunedError, ProcessExitError and
ProcessMissingError are never raised inside the job: the process running
it is already gone, and a different process records the error directly as
a failed execution after the fact. People coming from other backends
expect retry_on to cover this case and are surprised when it silently
doesn't, so spell out why it can't work and point to the mechanisms that
do: Mission Control and the fail_many_claimed subscription.

Fixes rails#786
…tabase

bin/jobs check always read the config files through the current Rails
env, so a CI run in the test env couldn't validate the production:
section of config/recurring.yml — the one file whose env scoping is
load-bearing. Worse, config_from's fallback made it silently validate
nothing: with no test: key it took the whole file, treated production as
a task with no schedule, dropped it, and reported the configuration
valid. A new --env option threads the target env through to both config
readers, so `bin/jobs check --env production` validates the section
that will actually be deployed.

Validation also crashed with a raw StatementInvalid backtrace when the
solid_queue tables weren't reachable — instantiating RecurringTask needs
its schema, and check already anticipates a missing database a few lines
down, in the pool-size warning. Configuration now passes the scheduler
raw [key, options] pairs instead of Active Record objects, which
RecurringSchedule already knows how to wrap when the scheduler boots, so
listing processes needs no database at all. Task-level validation still
needs the schema; when it isn't available, check degrades to an explicit
warning instead of failing, and still validates everything else.

Fixes rails#780
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Let bin/jobs check validate a config for an env it isn't running in, and without a database

1 participant