Skip to content

feat(orchestrator): Refresh each running container execution at most once a minute unless it changed - #473

Open
morgan-wowk wants to merge 1 commit into
orchestrator-sweep-when-overduefrom
orchestrator-refresh-at-most-once-a-minute
Open

morgan-wowk wants to merge 1 commit into
orchestrator-sweep-when-overduefrom
orchestrator-refresh-at-most-once-a-minute

Conversation

@morgan-wowk

Copy link
Copy Markdown
Collaborator

Problem

The running queue visits the oldest active row on every sweep, even one refreshed a moment ago. Visits, DB queries and launcher API calls grow with the sweep rate instead of with what changed. This is the per-execution throttle suggested in #426 (comment).

Changes

Impact

  • Each active container execution is refreshed at most once a minute unless something changed.
  • Changes nobody reports (launchers without change notifications, refresh retries) wait up to 60s.
  • Deployments that set _running_execution_refresh_interval keep their value.

Merge gates

…once a minute unless it changed

The running queue visited the oldest active row on every sweep, even when it
had just been refreshed, so visits, DB queries and launcher API calls grew
with sweep rate rather than with what changed.

The running-queue select now skips rows refreshed within
_running_execution_refresh_interval, whose default goes from 30s to 60s.
Rows that need attention still go first:
- rows a launcher reported as changed are moved further back than 60s;
- finished completion prefetches are moved to 1970-01-01;
- cancelling a run now moves its active container executions to 1970-01-01,
  so termination does not wait for the interval.

Because every selected row is now overdue, a sweep that refreshed anything
starts the next one without waiting, and the separate overdue check is gone.
Changes nobody reports (launchers without change notifications, failed
refresh retries) wait up to the interval.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant