CHORE: Flaky Setup - Gate benchmark and baseline steps on job success - #724
Draft
Gaurav Sharma (bewithgaurav) wants to merge 1 commit into
Draft
CHORE: Flaky Setup - Gate benchmark and baseline steps on job success#724Gaurav Sharma (bewithgaurav) wants to merge 1 commit into
Gaurav Sharma (bewithgaurav) wants to merge 1 commit into
Conversation
the AdventureWorks restore, baseline download, baseline prep and benchmark steps only checked which SQL version the leg was running, never whether anything upstream still worked. an explicit condition replaces the implicit succeeded() rather than adding to it, so these kept running after SQL Server setup had already failed. build 167929 spent 20.2 minutes benchmarking against a database that did not exist on a job that was already dead. each condition now starts with succeeded(), matching what the publish-baseline step already did. applies to the windows and macos legs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work Item / Issue Reference
Summary
the benchmark and baseline steps keep running after SQL Server setup has already failed, so a dead job still burns 20 minutes benchmarking against a database that does not exist.
build 167929,
macOS x86_64 SQL2025:pytest was correctly skipped because it relies on the default step condition. the four that ran override it:
that only asks which SQL version the leg is. writing an explicit condition replaces the implicit
succeeded()rather than adding to it, so nothing upstream is checked.Publish macOS benchmark baselinealready had this right withand(succeeded(), ...).each condition now starts with
succeeded(). the windows leg has the same four steps with the same pattern, and its SQL setup steps carry nocontinueOnError, so a failure there propagates the same way. eight sites in total acrosspytestonwindowsandPytestOnMacOS.the benchmark step also carries
timeoutInMinutes: 20andcontinueOnError: true, so when it times out the job reportssucceededWithIssuesinstead of failing. the macOS SQL2022 leg in that same build finishedsucceededWithIssuesfor exactly that reason. this does not change that behaviour, it just stops the step running when there is nothing to benchmark against.related to AB#47311. that one raised the macOS job timeout to 90 minutes, this one stops a failed setup from eating 20 minutes of the budget.