Repository navigation
feat(orchestrator): Refresh each running container execution at most once a minute unless it changed - #473
Open
morgan-wowk wants to merge 1 commit into
Conversation
…once a minute unless it changed The running queue visited the oldest active row on every sweep, even when it had just been refreshed, so visits, DB queries and launcher API calls grew with sweep rate rather than with what changed. The running-queue select now skips rows refreshed within _running_execution_refresh_interval, whose default goes from 30s to 60s. Rows that need attention still go first: - rows a launcher reported as changed are moved further back than 60s; - finished completion prefetches are moved to 1970-01-01; - cancelling a run now moves its active container executions to 1970-01-01, so termination does not wait for the interval. Because every selected row is now overdue, a sweep that refreshed anything starts the next one without waiting, and the separate overdue check is gone. Changes nobody reports (launchers without change notifications, failed refresh retries) wait up to the interval.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The running queue visits the oldest active row on every sweep, even one refreshed a moment ago. Visits, DB queries and launcher API calls grow with the sweep rate instead of with what changed. This is the per-execution throttle suggested in #426 (comment).
Changes
_running_execution_refresh_interval. Its default goes from 30s to 60s.Impact
_running_execution_refresh_intervalkeep their value.Merge gates
_running_execution_refresh_interval=timedelta(0); all five PRs together pass the suite.