Repository navigation
Stop the broker schema monitor spinning on "starting" when the agent is unreachable - #9001
Conversation
…is unreachable If the schema agent couldn't be reached, its status kept coming back as "still getting status", which the broker page showed as "starting" with no end and no explanation. Now that only holds for a bounded window: past it the page stops waiting, drops the broker back to an idle state so the toggle can be used again, and shows a single toast. The timed-out state is UI-only and clears on a refresh, a successful status, or another attempt to start. The schema monitor toggle and the connect toggle also awaited nothing on start, so a failed start went unnoticed. Both now wait on the request and surface a failure the same way.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9001 +/- ##
=======================================
Coverage 79.46% 79.46%
=======================================
Files 475 475
Lines 26018 26018
Branches 6934 6934
=======================================
Hits 20674 20674
Misses 5344 5344
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
andypalmi
left a comment
There was a problem hiding this comment.
Nice fix for the endless spinner, and good call awaiting the start requests. A few questions:
- After the timeout the page shows the broker as idle, but the backend row is still
running.toggleAgentdecides from the backend state, so the first click after a timeout goes down therunningbranch and callssuspendBroker, notstartBroker. I checked this with the page's methods and mocked responses: after the timeout, one click makes 0 start calls and 1 suspend call, and the UI goes tosuspending. So "retry" needs two clicks. CouldtoggleAgentstart when the page has timed out, since the toggle is showing as off?
async toggleAgent () {
this.loadingState = true
try {
const state = await brokerAPI.getBrokerStatus(this.team.id, this.$route.params.brokerId)
if (state.state === 'running' && !this.startingTimedOut) {
await brokerAPI.suspendBroker(this.team.id, this.$route.params.brokerId)
this.brokerState = 'suspending'
} else {
// Fresh attempt to start, so give the agent the full window again
this.resetStartingTimeout()
await brokerAPI.startBroker(this.team.id, this.$route.params.brokerId)
this.brokerState = 'starting'
}
} catch (err) {
...
}
}For a third-party broker whose row is running, the start route sends the agent a start command, so this is a real retry. A test that clicks the toggle after a timeout and asserts startBroker was called (and suspendBroker was not) would lock it in.
-
Is 30s enough for
STARTING_TIMEOUTon Kubernetes, where a cold agent pod (image pull, scheduling) can take longer? The user would see "Could not reach the broker agent" and then watch it flip to connected a moment later. A longer window might avoid the false alarm. -
Nit: the new
constsits between two imports, andBrokerStatusBadgeis imported below it. Moving the const under all the imports would read better.
After a timeout the toggle shows as off but the backend row is still running, so a click went down the suspend branch and the user needed two clicks to retry. The toggle now starts when the page has timed out, which sends the agent a start command. Two tests cover the retry and the normal suspend. The wait window is 60s instead of 30s. On Kubernetes the driver returns as soon as the Deployment exists and doesn't wait for the pod, so a cold image pull sat entirely inside the old window and produced a false alarm.
|
If the schema agent can't be reached, its status keeps coming back as "still getting status", which the broker page showed as "starting" forever, with no end and no explanation.
Now that only holds for a short window. Past it the page stops waiting, drops the broker back to an idle state so the toggle can be used again, and shows a single toast saying the agent couldn't be reached. The timed-out state is UI-only and clears on a refresh, a successful status, or another attempt to start, so nothing is persisted and the backend is untouched.
While here, the schema monitor toggle and the connect toggle both fired the start request without waiting on it, so a failed start went unnoticed. Both now wait on the request and surface a failure the same way.
One thing worth knowing for review: the team broker status badge is hardcoded to connected and its toggle reads on/off from the agent's connected flag, so the visible "stuck on starting" is really on third-party brokers. The fix sits on the shared status poll, so it covers both: the third-party spinner stops, and the team-broker path times out cleanly too.
Closes #8996