fix(live): hold live triggers during stage motion; survive a busy MCU at acquisition start - #639
Open
Alpaca233 wants to merge 2 commits into
Open
fix(live): hold live triggers during stage motion; survive a busy MCU at acquisition start#639Alpaca233 wants to merge 2 commits into
Alpaca233 wants to merge 2 commits into
Conversation
… at acquisition start Sending SEND_HARDWARE_TRIGGER while a MOVETO_* is in flight can wedge the firmware's single command-status slot: the MCU keeps executing commands but reports IN_PROGRESS for everything, and every wait_till_operation_is_completed call times out until the trigger stream stops (seen on an instrument on 2026-09-02 during focus-map point navigation with live view running). - LiveController.trigger_acquisition() returns False while the microcontroller is busy; the trigger timer re-checks within 10 ms, so live holds during stage moves and resumes automatically. - toggle_live handlers (LiveControlWidget, LaserAutofocusSettingWidget, NapariLiveWidget) update the button in a finally block from liveController.is_live, so a failed stop can't leave "Stop" showing while live is off. - run_acquisition() survives a stop_live() TimeoutError and waits for the MCU to go idle (3 x 5 s, logged) before the worker moves the stage; if the MCU stays busy it aborts through the normal failed_to_start path with an actionable error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Command dispatch remains race-prone, and failed preflight can leave UI and API callers reporting incorrect state.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Improves live-view coordination with MCU-controlled stage motion and acquisition startup recovery.
Changes:
- Pauses live triggers while the MCU reports busy.
- Synchronizes live-button labels with controller state.
- Adds MCU-idle acquisition preflight and regression tests.
File summaries
| File | Description |
|---|---|
software/control/core/live_controller.py |
Adds the MCU-busy trigger gate. |
software/control/core/multi_point_controller.py |
Adds acquisition-start recovery and idle waiting. |
software/control/widgets.py |
Updates live controls in finally blocks. |
software/tests/control/core/test_live_controller_motion_hold.py |
Tests trigger gating. |
software/tests/control/test_MultiPointController.py |
Tests MCU preflight recovery. |
software/tests/control/test_widget_toggle_live_truthful.py |
Tests live-control state updates. |
Review details
Suppressed comments (1)
software/control/core/multi_point_controller.py:837
- This abort leaves the live controls untruthful in the acquisition-start recovery path. If live was on,
stop_live()has already setis_live=False, but this return occurs beforehardware_prepared=True, so thefinallyblock neither resumes live nor notifies any live widget; the three newtoggle_livefinally blocks are not invoked because the controller stopped live directly. The checked button therefore still reads “Stop” after the failed start—the same operator-facing state this PR is intended to eliminate. Publish/synchronize the controller's live-state change before returning.
if not self._wait_for_microcontroller_idle():
self._log.error(
f"Microcontroller still busy after "
f"{self._MCU_IDLE_WAIT_ATTEMPTS * self._MCU_IDLE_WAIT_TIMEOUT_S} s - home the stage or "
f"power-cycle the controller. Aborting the acquisition start."
)
return
- Files reviewed: 6/6 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+499
to
+505
| if self.microscope.low_level_drivers.microcontroller.is_busy(): | ||
| # Hold live while the MCU executes a command (usually a stage move): | ||
| # triggering mid-move corrupts the firmware's single command-status | ||
| # slot and wedges it - every later wait times out until the trigger | ||
| # stream stops. The trigger timer re-checks within 10 ms when we | ||
| # return False, so live resumes as soon as the move completes. | ||
| return False |
Comment on lines
+831
to
+837
| if not self._wait_for_microcontroller_idle(): | ||
| self._log.error( | ||
| f"Microcontroller still busy after " | ||
| f"{self._MCU_IDLE_WAIT_ATTEMPTS * self._MCU_IDLE_WAIT_TIMEOUT_S} s - home the stage or " | ||
| f"power-cycle the controller. Aborting the acquisition start." | ||
| ) | ||
| return |
- Re-check a busy-MCU trigger hold at the normal frame cadence instead of every 10 ms - each re-check builds a fresh Timer thread, and a stage move lasts hundreds of 10 ms ticks. - Guard snap() with the same wedge-avoidance as trigger_acquisition(): wait for the MCU, refuse to trigger if it stays busy. - Deduplicate the three toggle_live handlers into one _sync_live_button helper, which now also syncs the checked state so the next click sends the right `pressed`. - Drop the _PreflightStub shadow class and redundant unit tests; the retry behavior is covered by one real-controller test plus the existing integration tests. - Trim repeated rationale comments to point at the trigger_acquisition guard, and state plainly that the pre-flight loop exists for progress logging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On an instrument on 2026-09-02, clicking "go to point" in the focus-map navigator while live view was streaming hardware triggers (~9.3 fps) wedged the MCU: a
MOVETO_Zraced theSEND_HARDWARE_TRIGGERstream and the firmware's single command-status slot got stuck reportingIN_PROGRESSfor every subsequent command — while still executing them (frames kept arriving). Everywait_till_operation_is_completed()then timed out for ~10 minutes:TimeoutError,stop_live()(illumination-off wait timed out),setText), inviting more toggling that restarted the trigger stream and re-wedged the MCU.Log evidence shows the wedge clears a few seconds after the trigger stream stops — the one acquisition attempt that worked ran
stop_liveas a no-op (live already stopped by a failed toggle), reached its firstMOVETO_Z~6 s after the last illuminated trigger, and completed it in 17.5 ms.Fix (host-side; the firmware's single command-status slot remains the underlying defect)
LiveController.trigger_acquisition()returnsFalsewhilemicrocontroller.is_busy(). The trigger timer already re-checks within 10 ms onFalse, so live pauses for the duration of a move and resumes by itself. This prevents the trigger/move race on every motion path (focus map, click-to-move, autofocus).toggle_liveinLiveControlWidget,LaserAutofocusSettingWidget, andNapariLiveWidgetnow sets button state in afinallyblock fromliveController.is_live, so a failed stop can no longer leave the button lying.run_acquisition()catches aTimeoutErrorfromstop_live()(live is already stopped and its trigger timer cancelled by then; the command was sent) and waits for the MCU to go idle via_wait_for_microcontroller_idle()(3 × 5 s attempts, each logged) before the worker moves the stage. If the MCU stays busy it aborts through the existingfailed_to_startpath with an actionable "home the stage or power-cycle" error instead of an uncaught traceback.With (1) the wedge shouldn't occur; (2) and (3) make the recovery path work if it ever does.
Testing
tests/control/core/test_live_controller_motion_hold.py(trigger gate),tests/control/test_widget_toggle_live_truthful.py(14 parametrized button-truthfulness tests), and 5 pre-flight tests intest_MultiPointController.py— including one proving an acquisition still runs to completion after astop_live()timeout when the MCU recovers. All written failing-first.🤖 Generated with Claude Code