fix(cli): restore the standalone rerun-bridge command by paul-nechifor · Pull Request #5191 · dimensionalOS/dimos

paul-nechifor · 2026-11-11T11:29:35Z

Why do we need this?

pyproject.toml: rerun-bridge imports a removed symbol and crashes before help.

What is this feature?

pyproject.toml: Route the command through dimos rerun-bridge. Add tests for help and argument handling.

Closes DIM-508

greptile-apps · 2026-11-11T11:31:30Z

Confidence Score: 5/5

[Low impact] The PR appears safe to merge.

Summary

Restores standalone rerun-bridge by routing it through the shared dimos CLI.

The standalone rerun-bridge command now enters through the shared CLI.

_{Reviews (1) · Last reviewed commit: "fix(cli): restore the standalone rerun-b..." · Reviewed by Greptile}

github-actions

go2 replay realtime (arm64)

Details

Benchmark suite	Current: `ac9ce10`	Previous: `442d694`	Ratio
`peak memory`	`1726.504` MB	`1745.414` MB	`0.99`
`peak threads`	`367` threads	`374` threads	`0.98`
`network (transport)`	`2733.725` MB	`2736.265` MB	`1.00`
`disk write`	`1.941` MB	`1.844` MB	`1.05`
`instructions`	`137.067` G	`138.643` G	`0.99`

This comment was automatically generated by workflow using github-action-benchmark.

codecov · 2026-11-11T11:39:01Z

❌ 26 Tests Failed:

Tests completed	Failed	Passed	Skipped
9036	26	9010	84

View the top 3 failed test(s) by shortest run time

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_bbox_properties[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_cropped_image[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_is_valid[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_ros_conversion[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_string_representation[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_image_detections2d_structure[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_multiple_detections[bbox_detector]

Stack Traces | 0s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.perception.detection.detectors.test_bbox_detectors::test_detection_basic[bbox_detector]

Stack Traces | 10s run time

detector = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

    @pytest.fixture(scope="session")
    def detections(detector, test_image, topic_image):
        """Get ImageDetections2D from any detector."""
>       topic_image.publish(test_image)

detector   = <dimos.perception.detection.detectors.yolo.Yolo2DDetector object at 0x3693e5eb0>
test_image = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
topic_image = <dimos.core.transport.LCMTransport object at 0x3691d32f0>

.../detection/detectors/test_bbox_detectors.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Image(shape=(771, 1024, 3), format=BGR, dtype=uint8, frame_id='', ts=1794398348.5273979)
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3691d32f0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691d3470>
startup_complete = <threading.Event at 0x3691d3320: set>
thread     = <Thread(Thread-1123 (_lcm_loop), stopped daemon 15764254720)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.mapping.test_voxels::test_roundtrip[0.1-7290]

Stack Traces | 10.2s run time

moment1 = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a13e90>
voxel_size = 0.1, expected_points = 7290

    @pytest.mark.parametrize(
        "voxel_size, expected_points",
        [
            (0.5, 277),
            (0.1, 7290),
            (0.05, 28199),
        ],
    )
    def test_roundtrip(moment1: Go2MapperMoment, voxel_size: float, expected_points: int) -> None:
        lidar_frame = moment1.lidar.value
        assert lidar_frame is not None
    
        grid = VoxelGrid(voxel_size=voxel_size)
        grid.add_frame(lidar_frame)
    
        global1 = grid.get_global_pointcloud2()
        assert len(global1) == expected_points
    
        # loseless roundtrip
        if voxel_size == 0.05:
            assert len(global1) == len(lidar_frame)
            # TODO: we want __eq__ on PointCloud2 - should actually compare
            # all points in both frames
    
        grid.add_frame(global1)
        # no new information, no global map change
        assert len(grid.get_global_pointcloud2()) == len(global1)
    
>       moment1.publish()

expected_points = 7290
global1    = PointCloud(points=7290, frame_id='world', ts=1758149209.1916595)
grid       = <dimos.mapping.voxels.grid.VoxelGrid object at 0x367a12bd0>
lidar_frame = PointCloud(points=28199, frame_id='world', ts=1758149209.1916595)
moment1    = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a13e90>
voxel_size = 0.1

dimos/mapping/test_voxels.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../utils/testing/test_moment.py:68: in publish
    camera_info_transport.publish(camera_info)
        __class__  = <class 'dimos.utils.testing.test_moment.Go2Moment'>
        camera_info = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398207.113628)
        camera_info_transport = <dimos.core.transport.LCMTransport object at 0x3679b4050>
        self       = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a13e90>
        t          = <dimos.protocol.tf.tf.TF object at 0x367a10620>
        tf_transport = <dimos.core.transport.ZenohTransport object at 0x3679b4650>
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398207.113628)
        self       = <dimos.core.transport.LCMTransport object at 0x3679b4050>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398207.113628)
        self       = <dimos.core.transport.LCMTransport object at 0x3679b4050>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3679b4050>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691f3b90>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3691f3b90>
startup_complete = <threading.Event at 0x3691f3f80: set>
thread     = <Thread(Thread-807 (_lcm_loop), stopped daemon 15751671808)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.mapping.test_voxels::test_roundtrip[0.05-28199]

Stack Traces | 10.3s run time

moment1 = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a9b770>
voxel_size = 0.05, expected_points = 28199

    @pytest.mark.parametrize(
        "voxel_size, expected_points",
        [
            (0.5, 277),
            (0.1, 7290),
            (0.05, 28199),
        ],
    )
    def test_roundtrip(moment1: Go2MapperMoment, voxel_size: float, expected_points: int) -> None:
        lidar_frame = moment1.lidar.value
        assert lidar_frame is not None
    
        grid = VoxelGrid(voxel_size=voxel_size)
        grid.add_frame(lidar_frame)
    
        global1 = grid.get_global_pointcloud2()
        assert len(global1) == expected_points
    
        # loseless roundtrip
        if voxel_size == 0.05:
            assert len(global1) == len(lidar_frame)
            # TODO: we want __eq__ on PointCloud2 - should actually compare
            # all points in both frames
    
        grid.add_frame(global1)
        # no new information, no global map change
        assert len(grid.get_global_pointcloud2()) == len(global1)
    
>       moment1.publish()

expected_points = 28199
global1    = PointCloud(points=28199, frame_id='world', ts=1758149209.1916595)
grid       = <dimos.mapping.voxels.grid.VoxelGrid object at 0x367a9b9b0>
lidar_frame = PointCloud(points=28199, frame_id='world', ts=1758149209.1916595)
moment1    = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a9b770>
voxel_size = 0.05

dimos/mapping/test_voxels.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../utils/testing/test_moment.py:68: in publish
    camera_info_transport.publish(camera_info)
        __class__  = <class 'dimos.utils.testing.test_moment.Go2Moment'>
        camera_info = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398217.311852)
        camera_info_transport = <dimos.core.transport.LCMTransport object at 0x367981220>
        self       = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x367a9b770>
        t          = <dimos.protocol.tf.tf.TF object at 0x367a9be00>
        tf_transport = <dimos.core.transport.ZenohTransport object at 0x367a9a030>
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398217.311852)
        self       = <dimos.core.transport.LCMTransport object at 0x367981220>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398217.311852)
        self       = <dimos.core.transport.LCMTransport object at 0x367981220>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x367981220>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x367981880>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x367981880>
startup_complete = <threading.Event at 0x36796aff0: set>
thread     = <Thread(Thread-810 (_lcm_loop), stopped daemon 15751671808)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.experimental.memory.test_rust_recorder_e2e::test_cli_recording_uses_existing_binary_for_both_formats[mcap]

Stack Traces | 10.4s run time

tmp_path = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi1')
rust_recorder_executable = PosixPath('.../result/bin/dimos-memory-recorder')
store_kind = 'mcap'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x365159340>

    @pytest.mark.parametrize(
        "store_kind",
        ["sqlite", "mcap"],
    )
    def test_cli_recording_uses_existing_binary_for_both_formats(
        tmp_path: Path,
        rust_recorder_executable: Path,
        store_kind: str,
        monkeypatch: pytest.MonkeyPatch,
    ) -> None:
        suffix = "db" if store_kind == "sqlite" else "mcap"
        artifact = tmp_path / f"memory.{suffix}"
        lcm_url = f"udpm://239.255.76.67:{_free_port()}?ttl=0"
        monkeypatch.setenv("LCM_DEFAULT_URL", lcm_url)
        monkeypatch.setattr(global_config, "record_engine", "rust")
        monkeypatch.setattr(global_config, "record", store_kind)
        monkeypatch.setattr(global_config, "record_topics", "*")
        monkeypatch.setattr(global_config, "record_encoding_threads", 2)
        monkeypatch.setattr(global_config, "transport", "lcm")
        monkeypatch.setattr(global_config, "build_native", False)
        monkeypatch.setattr(
            rust_cli_recorder, "prepare_rust_recorder", lambda: rust_recorder_executable
        )
        monkeypatch.setattr(rust_cli_recorder, "recording_dir", lambda: tmp_path)
        channel = f"/rust-recorder-{uuid.uuid4().hex[:8]}"
        publisher: LCMTransport[Imu] = LCMTransport(channel, Imu, url=lcm_url)
        session = RustRecordingSession(rust_cli_recorder.make_plan({("imu", Imu): publisher}))
        expected = Imu(ts=22.5, frame_id="imu_link", angular_velocity=Vector3(1.0, 2.0, 3.0))
        try:
            publisher.start()
            session.start()
            for _ in range(3):
                publisher.broadcast(None, expected)
                time.sleep(0.05)
        finally:
            session.stop()
            publisher.stop()
    
        memory: SqliteStore | McapStore
        if store_kind == "mcap" and not _MCAP_AVAILABLE:
            data = artifact.read_bytes()
            assert data.startswith(b"\x89MCAP0\r\n")
            assert data.endswith(b"\x89MCAP0\r\n")
            return
        if store_kind == "sqlite":
            memory = SqliteStore(path=str(artifact))
        else:
            memory = McapStore(path=str(artifact))
        with memory:
>           observation = cast("Observation[Imu]", memory.stream("imu").first())

_          = 2
artifact   = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi1/memory.mcap')
channel    = '/rust-recorder-a31f3bff'
expected   = Imu(ts=22.5, frame_id='imu_link', angular_velocity=↘ Vector Vector([          1           2           3]), linear_acce...· Vector Vector([          0           0           0]), orientation=Quaternion(0.000000, 0.000000, 0.000000, 1.000000))
lcm_url    = 'udpm://239.255.76.67:54367?ttl=0'
memory     = <dimos.memory.store.mcap.McapStore object at 0x3651590a0>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x365159340>
publisher  = <dimos.core.transport.LCMTransport object at 0x365158200>
rust_recorder_executable = PosixPath('.../result/bin/dimos-memory-recorder')
session    = <dimos.experimental.memory.rust_cli_recorder.RustRecordingSession object at 0x36515a510>
store_kind = 'mcap'
suffix     = 'mcap'
tmp_path   = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi1')

.../experimental/memory/test_rust_recorder_e2e.py:465: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.memory.stream.Stream object at 0x365159820>

    def first(self) -> O:
        """Return the first matching observation."""
        it = iter(self.limit(1))
        try:
            return next(it)
        except StopIteration:
>           raise LookupError("No matching observation") from None
E           LookupError: No matching observation

it         = <generator object Backend._iterate_snapshot at 0x365152740>
self       = <dimos.memory.stream.Stream object at 0x365159820>

dimos/memory/stream.py:484: LookupError

dimos.mapping.test_voxels::test_roundtrip[0.5-277]

Stack Traces | 10.5s run time

moment1 = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x3679bef60>
voxel_size = 0.5, expected_points = 277

    @pytest.mark.parametrize(
        "voxel_size, expected_points",
        [
            (0.5, 277),
            (0.1, 7290),
            (0.05, 28199),
        ],
    )
    def test_roundtrip(moment1: Go2MapperMoment, voxel_size: float, expected_points: int) -> None:
        lidar_frame = moment1.lidar.value
        assert lidar_frame is not None
    
        grid = VoxelGrid(voxel_size=voxel_size)
        grid.add_frame(lidar_frame)
    
        global1 = grid.get_global_pointcloud2()
        assert len(global1) == expected_points
    
        # loseless roundtrip
        if voxel_size == 0.05:
            assert len(global1) == len(lidar_frame)
            # TODO: we want __eq__ on PointCloud2 - should actually compare
            # all points in both frames
    
        grid.add_frame(global1)
        # no new information, no global map change
        assert len(grid.get_global_pointcloud2()) == len(global1)
    
>       moment1.publish()

expected_points = 277
global1    = PointCloud(points=277, frame_id='world', ts=1758149209.1916595)
grid       = <dimos.mapping.voxels.grid.VoxelGrid object at 0x3691f3f50>
lidar_frame = PointCloud(points=28199, frame_id='world', ts=1758149209.1916595)
moment1    = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x3679bef60>
voxel_size = 0.5

dimos/mapping/test_voxels.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../utils/testing/test_moment.py:68: in publish
    camera_info_transport.publish(camera_info)
        __class__  = <class 'dimos.utils.testing.test_moment.Go2Moment'>
        camera_info = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398196.7305532)
        camera_info_transport = <dimos.core.transport.LCMTransport object at 0x3678fcb60>
        self       = <dimos.mapping.test_voxels.Go2MapperMoment object at 0x3679bef60>
        t          = <dimos.protocol.tf.tf.TF object at 0x3679bc140>
        tf_transport = <dimos.core.transport.ZenohTransport object at 0x3679bfb30>
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398196.7305532)
        self       = <dimos.core.transport.LCMTransport object at 0x3678fcb60>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398196.7305532)
        self       = <dimos.core.transport.LCMTransport object at 0x3678fcb60>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3678fcb60>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3679be6c0>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3679be6c0>
startup_complete = <threading.Event at 0x3679bd580: set>
thread     = <Thread(Thread-804 (_lcm_loop), stopped daemon 15751671808)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.experimental.memory.test_rust_recorder_e2e::test_cli_recording_uses_existing_binary_for_both_formats[sqlite]

Stack Traces | 10.8s run time

tmp_path = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi0')
rust_recorder_executable = PosixPath('.../result/bin/dimos-memory-recorder')
store_kind = 'sqlite'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x365159b50>

    @pytest.mark.parametrize(
        "store_kind",
        ["sqlite", "mcap"],
    )
    def test_cli_recording_uses_existing_binary_for_both_formats(
        tmp_path: Path,
        rust_recorder_executable: Path,
        store_kind: str,
        monkeypatch: pytest.MonkeyPatch,
    ) -> None:
        suffix = "db" if store_kind == "sqlite" else "mcap"
        artifact = tmp_path / f"memory.{suffix}"
        lcm_url = f"udpm://239.255.76.67:{_free_port()}?ttl=0"
        monkeypatch.setenv("LCM_DEFAULT_URL", lcm_url)
        monkeypatch.setattr(global_config, "record_engine", "rust")
        monkeypatch.setattr(global_config, "record", store_kind)
        monkeypatch.setattr(global_config, "record_topics", "*")
        monkeypatch.setattr(global_config, "record_encoding_threads", 2)
        monkeypatch.setattr(global_config, "transport", "lcm")
        monkeypatch.setattr(global_config, "build_native", False)
        monkeypatch.setattr(
            rust_cli_recorder, "prepare_rust_recorder", lambda: rust_recorder_executable
        )
        monkeypatch.setattr(rust_cli_recorder, "recording_dir", lambda: tmp_path)
        channel = f"/rust-recorder-{uuid.uuid4().hex[:8]}"
        publisher: LCMTransport[Imu] = LCMTransport(channel, Imu, url=lcm_url)
        session = RustRecordingSession(rust_cli_recorder.make_plan({("imu", Imu): publisher}))
        expected = Imu(ts=22.5, frame_id="imu_link", angular_velocity=Vector3(1.0, 2.0, 3.0))
        try:
            publisher.start()
            session.start()
            for _ in range(3):
                publisher.broadcast(None, expected)
                time.sleep(0.05)
        finally:
            session.stop()
            publisher.stop()
    
        memory: SqliteStore | McapStore
        if store_kind == "mcap" and not _MCAP_AVAILABLE:
            data = artifact.read_bytes()
            assert data.startswith(b"\x89MCAP0\r\n")
            assert data.endswith(b"\x89MCAP0\r\n")
            return
        if store_kind == "sqlite":
            memory = SqliteStore(path=str(artifact))
        else:
            memory = McapStore(path=str(artifact))
        with memory:
>           observation = cast("Observation[Imu]", memory.stream("imu").first())

_          = 2
artifact   = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi0/memory.db')
channel    = '/rust-recorder-33e64c8a'
expected   = Imu(ts=22.5, frame_id='imu_link', angular_velocity=↘ Vector Vector([          1           2           3]), linear_acce...· Vector Vector([          0           0           0]), orientation=Quaternion(0.000000, 0.000000, 0.000000, 1.000000))
lcm_url    = 'udpm://239.255.76.67:54366?ttl=0'
memory     = <dimos.memory.store.sqlite.SqliteStore object at 0x364bf5100>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x365159b50>
publisher  = <dimos.core.transport.LCMTransport object at 0x3651589e0>
rust_recorder_executable = PosixPath('.../result/bin/dimos-memory-recorder')
session    = <dimos.experimental.memory.rust_cli_recorder.RustRecordingSession object at 0x365159dc0>
store_kind = 'sqlite'
suffix     = 'db'
tmp_path   = PosixPath('.../pytest-of-admin/pytest-13/test_cli_recording_uses_existi0')

.../experimental/memory/test_rust_recorder_e2e.py:465: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.memory.stream.Stream object at 0x364bf7800>

    def first(self) -> O:
        """Return the first matching observation."""
        it = iter(self.limit(1))
        try:
            return next(it)
        except StopIteration:
>           raise LookupError("No matching observation") from None
E           LookupError: No matching observation

it         = <generator object Backend._iterate_snapshot at 0x364a45340>
self       = <dimos.memory.stream.Stream object at 0x364bf7800>

dimos/memory/stream.py:484: LookupError

dimos.utils.testing.test_moment::test_moment_seek_and_publish

Stack Traces | 30.5s run time

def test_moment_seek_and_publish() -> None:
        moment = Go2Moment()
    
        # Seek to 5 seconds
        moment.seek(5.0)
    
        # Check that frames were loaded
        assert moment.lidar.value is not None
        assert moment.video.value is not None
        assert moment.odom.value is not None
    
        # Publish all frames
>       moment.publish()

moment     = <dimos.utils.testing.test_moment.Go2Moment object at 0x3ded81550>

.../utils/testing/test_moment.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../utils/testing/test_moment.py:71: in publish
    super().publish()
        __class__  = <class 'dimos.utils.testing.test_moment.Go2Moment'>
        camera_info = CameraInfo(height=720, width=1280, distortion_model='equidistant', frame_id='camera_optical', ts=1794398528.089029)
        camera_info_transport = <dimos.core.transport.LCMTransport object at 0x3e3967650>
        self       = <dimos.utils.testing.test_moment.Go2Moment object at 0x3ded81550>
        t          = <dimos.protocol.tf.tf.TF object at 0x3dec9bfe0>
        tf_transport = <dimos.core.transport.ZenohTransport object at 0x3de7284a0>
.../utils/testing/moment.py:96: in publish
    moment.publish()
        moment     = <dimos.utils.testing.moment.SensorMoment object at 0x3dec99610>
        self       = <dimos.utils.testing.test_moment.Go2Moment object at 0x3ded81550>
.../utils/testing/moment.py:41: in publish
    self.transport.publish(self.value)
        self       = <dimos.utils.testing.moment.SensorMoment object at 0x3dec99610>
dimos/core/stream.py:110: in publish
    self.broadcast(None, msg)
        msg        = Odom pos(→ Vector Vector([      1.955     0.12131      0.3083])), rot(Quaternion(0.000003, -0.029749, 0.016729, -0.999417))
        self       = <dimos.core.transport.LCMTransport object at 0x3dec99370>
dimos/core/transport.py:177: in broadcast
    self.start()
        _          = None
        msg        = Odom pos(→ Vector Vector([      1.955     0.12131      0.3083])), rot(Quaternion(0.000003, -0.029749, 0.016729, -0.999417))
        self       = <dimos.core.transport.LCMTransport object at 0x3dec99370>
dimos/core/transport.py:161: in start
    self.lcm.start()
        self       = <dimos.core.transport.LCMTransport object at 0x3dec99370>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3dec99760>

    def start(self) -> None:
        """Start dispatch, or raise if a previous shutdown is still in progress.
    
        Wait for the first poll to finish (or for its callback to stop dispatch).
        After callback shutdown or a join timeout, retry only once dispatch exits.
        """
        with self._call_pool.lock:
            if self._thread is not None and self._thread.is_alive() and self._stop_event.is_set():
                raise RuntimeError("LCM service is still stopping; cannot restart yet")
            self._call_pool.start()
            if self._thread is None or not self._thread.is_alive():
                # Reinitialize LCM if it's None (e.g. after stop())
                if self.l is None:
                    # Any pending unsubscribes reference the destroyed instance.
                    self._pending_unsubs.clear()
                    if self.config.lcm:
                        self.l = self.config.lcm
                    else:
                        self.l = lcm_mod.LCM(self.config.url) if self.config.url else lcm_mod.LCM()
    
                self._stop_event = threading.Event()
                # Waiters retain their own run's event across a later restart.
                self._startup_complete = threading.Event()
                self._thread = threading.Thread(target=self._lcm_loop)
                self._thread.daemon = True
                self._thread.start()
            thread = self._thread
            startup_complete = self._startup_complete
        # The first dispatch may call stop(), which also needs the lifecycle lock.
        if threading.current_thread() is not thread and not startup_complete.wait(timeout=5.0):
>           raise RuntimeError("LCM handler thread failed to start within 5s")
E           RuntimeError: LCM handler thread failed to start within 5s

self       = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x3dec99760>
startup_complete = <threading.Event at 0x3de80e9f0: set>
thread     = <Thread(Thread-1634 (_lcm_loop), stopped daemon 16644108288)>

.../protocol/service/lcmservice.py:137: RuntimeError

dimos.hardware.sensors.lidar.livox.test_e2e::test_live_loopback_handshake_and_stream

Stack Traces | 30.9s run time

self = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
topics = {'/e2e_live_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_live_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}
seconds = 10.0, enough = {'/e2e_live_imu': 120, '/e2e_live_lidar': 3}

    @contextmanager
    def collect(
        self, topics: dict[str, type[PointCloud2 | Imu]], seconds: float, enough: dict[str, int]
    ) -> Iterator[dict[str, list[bytes]]]:
        """Subscribe before yielding, then collect after the caller starts the binaries."""
        raw: dict[str, list[bytes]] = {topic: [] for topic in topics}
        lc = lcm_module.LCM()
        subscriptions = []
        try:
            for topic in topics:
                # Buffer bytes only. Decoding inline drops packets.
                def on_msg(_ch: str, data: bytes, topic: str = topic) -> None:
                    raw[topic].append(data)
    
                msg_type = topics[topic]
                subscriptions.append(
                    lc.subscribe(f"{topic}#sensor_msgs.{msg_type.__name__}", on_msg)
                )
            yield raw
            end = time.monotonic() + seconds
            while not all(len(raw[topic]) >= count for topic, count in enough.items()):
                self._assert_running()
                if time.monotonic() >= end:
                    counts = {topic: len(messages) for topic, messages in raw.items()}
                    logs = "\n".join(
                        f"{binary.name} stderr:\n{stderr.read_text(errors='replace')[-4000:]}"
                        for binary, _, stderr in self._spawned
                    )
>                   pytest.fail(
                        f"Timed out collecting streams: {counts}, expected {enough}\n{logs}"
                    )
E                   Failed: Timed out collecting streams: {'/e2e_live_lidar': 0, '/e2e_live_imu': 0}, expected {'/e2e_live_lidar': 3, '/e2e_live_imu': 120}
E                   mid360_native stderr:
E                   :module"}
E                   {"timestamp":"2026-11-11T11:55:18.847815Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.847825Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.865620Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.865647Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.883671Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.883720Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955047Z","level":"WARN","fields":{"message":"clock offset jumped, restarting host clock estimate","estimate_ns":122038,"offset_ns":1002722050},"target":"livox_wire::pipeline"}
E                   {"timestamp":"2026-11-11T11:55:18.955126Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955149Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955161Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955172Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955184Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955194Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:18.955214Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:19.022618Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:19.022642Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:19.022655Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:19.022665Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:19.022673Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_live_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   
E                   virtual_mid360 stderr:
E                   {"timestamp":"2026-11-11T11:55:16.469890Z","level":"INFO","fields":{"message":"config loaded","exe":"virtual_mid360","config":"Config { pcap: \".../pytest-13/livox_e2e0/synth_mid360.pcap\", rate: 0.5, delay: 0.0, lidar_ip: \"127.0.0.1\", host_ip: \"127.0.0.1\", lidar_netns: \"\", mcast_data: \"224.1.1.5\" }"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:55:16.470068Z","level":"INFO","fields":{"message":"virtual_mid360 started","lidar":"127.0.0.1","host":"127.0.0.1","rate":0.5,"delay":0.0},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958124Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":2},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958169Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958194Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":3},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958199Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958249Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":4},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958265Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958300Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":5},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958307Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958360Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":6},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958369Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958410Z","level":"INFO","fields":{"message":"control REQ -> ACK","cmd_id":"0x0100","seq":7},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:16.958416Z","level":"INFO","fields":{"message":"work-mode cmd 0x0100 acked -> arming data stream"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:17.012246Z","level":"INFO","fields":{"message":"streaming 400 packets at 0.5x"},"target":"virtual_mid360"}
E                   {"timestamp":"2026-11-11T11:55:19.022544Z","level":"INFO","fields":{"message":"data stream finished"},"target":"virtual_mid360"}

counts     = {'/e2e_live_imu': 0, '/e2e_live_lidar': 0}
end        = 26081.9915825
enough     = {'/e2e_live_imu': 120, '/e2e_live_lidar': 3}
lc         = <LCM object at 0x366e07690>
logs       = 'mid360_native stderr:\n:module"}\n{"timestamp":"2026-11-11T11:55:18.847815Z","level":"ERROR","fields":{"message":"pub...:"2026-11-11T11:55:19.022544Z","level":"INFO","fields":{"message":"data stream finished"},"target":"virtual_mid360"}\n'
msg_type   = <class 'dimos.msgs.sensor_msgs.Imu.Imu'>
on_msg     = <function Spawner.collect.<locals>.on_msg at 0x36512b9c0>
raw        = {'/e2e_live_imu': [], '/e2e_live_lidar': []}
seconds    = 10.0
self       = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
subscription = None
subscriptions = [None, None]
topic      = '/e2e_live_imu'
topics     = {'/e2e_live_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_live_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}

.../lidar/livox/test_e2e.py:189: Failed

During handling of the above exception, another exception occurred:

spawn = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
synth_pcap = PosixPath('.../pytest-13/livox_e2e0/synth_mid360.pcap')

    @pytest.mark.native_e2e
    def test_live_loopback_handshake_and_stream(spawn: Spawner, synth_pcap: Path) -> None:
        driver_binary = _require_binary("mid360_native")
        virtual_binary = _require_binary("virtual_mid360")
    
        virtual_config = VirtualMid360Config(
            pcap=str(synth_pcap),
            rate=0.5,
            # A positive delay also enables streaming without a handshake. Wait for
            # the driver's request instead, which comes after its receive sockets bind.
            delay=0.0,
            lidar_ip="127.0.0.1",
            host_ip="127.0.0.1",
            mcast_data=SDK_MULTICAST_GROUP,
        )
        virtual_blob = {
            "topics": {},
            "config": virtual_config.to_config_dict(),
            "session": {"id": "pointlio-e2e", "links": []},
        }
        config = Mid360Config(
            host_ip="127.0.0.1",
            lidar_ip="127.0.0.1",
            multicast_ip=None,
            point_format="full",
        )
        driver_blob = {
            "topics": {
                "lidar": "/e2e_live_lidar#sensor_msgs.PointCloud2",
                "imu": "/e2e_live_imu#sensor_msgs.Imu",
            },
            "config": config.to_config_dict(),
            "session": {"id": "pointlio-e2e", "links": []},
        }
    
        # Subscribe before either process can publish the finite, half-rate capture.
>       with spawn.collect(
            {"/e2e_live_lidar": PointCloud2, "/e2e_live_imu": Imu},
            seconds=10.0,
            enough={"/e2e_live_lidar": 3, "/e2e_live_imu": 120},
        ) as received:

config     = Mid360Config(rpc_transport=<class 'dimos.protocol.rpc.zenohrpc.ZenohRPC'>, default_rpc_timeout=120.0, rpc_timeouts={'b...a_port=56400, host_cmd_data_port=56101, host_push_msg_port=56201, host_point_data_port=56301, host_imu_data_port=56401)
driver_binary = PosixPath('.../result/bin/mid360_native')
driver_blob = {'config': {'cmd_data_port': 56100, 'enable_imu': True, 'frame_id': 'lidar_link', 'frequency': 10.0, ...}, 'session': ..., 'links': []}, 'topics': {'imu': '/e2e_live_imu#sensor_msgs.Imu', 'lidar': '/e2e_live_lidar#sensor_msgs.PointCloud2'}}
received   = {'/e2e_live_imu': [], '/e2e_live_lidar': []}
spawn      = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
synth_pcap = PosixPath('.../pytest-13/livox_e2e0/synth_mid360.pcap')
virtual_binary = PosixPath('.../result/bin/virtual_mid360')
virtual_blob = {'config': {'delay': 0.0, 'host_ip': '127.0.0.1', 'lidar_ip': '127.0.0.1', 'lidar_netns': '', ...}, 'session': {'id': 'pointlio-e2e', 'links': []}, 'topics': {}}
virtual_config = VirtualMid360Config(rpc_transport=<class 'dimos.protocol.rpc.zenohrpc.ZenohRPC'>, default_rpc_timeout=120.0, rpc_timeo...27.0.0.1', host_ip='127.0.0.1', lidar_netns='', mcast_data='224.1.1.5', setup_network=True, alias_iface='dimos-mid360')

.../lidar/livox/test_e2e.py:330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/contextlib.py:144: in __exit__
    next(self.gen)
        self       = <contextlib._GeneratorContextManager object at 0x366e07740>
        traceback  = None
        typ        = None
        value      = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
topics = {'/e2e_live_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_live_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}
seconds = 10.0, enough = {'/e2e_live_imu': 120, '/e2e_live_lidar': 3}

    @contextmanager
    def collect(
        self, topics: dict[str, type[PointCloud2 | Imu]], seconds: float, enough: dict[str, int]
    ) -> Iterator[dict[str, list[bytes]]]:
        """Subscribe before yielding, then collect after the caller starts the binaries."""
        raw: dict[str, list[bytes]] = {topic: [] for topic in topics}
        lc = lcm_module.LCM()
        subscriptions = []
        try:
            for topic in topics:
                # Buffer bytes only. Decoding inline drops packets.
                def on_msg(_ch: str, data: bytes, topic: str = topic) -> None:
                    raw[topic].append(data)
    
                msg_type = topics[topic]
                subscriptions.append(
                    lc.subscribe(f"{topic}#sensor_msgs.{msg_type.__name__}", on_msg)
                )
            yield raw
            end = time.monotonic() + seconds
            while not all(len(raw[topic]) >= count for topic, count in enough.items()):
                self._assert_running()
                if time.monotonic() >= end:
                    counts = {topic: len(messages) for topic, messages in raw.items()}
                    logs = "\n".join(
                        f"{binary.name} stderr:\n{stderr.read_text(errors='replace')[-4000:]}"
                        for binary, _, stderr in self._spawned
                    )
                    pytest.fail(
                        f"Timed out collecting streams: {counts}, expected {enough}\n{logs}"
                    )
                lc.handle_timeout(200)
        finally:
            for subscription in subscriptions:
>               lc.unsubscribe(subscription)
E               TypeError: argument 1 must be LCMSubscription, not None

counts     = {'/e2e_live_imu': 0, '/e2e_live_lidar': 0}
end        = 26081.9915825
enough     = {'/e2e_live_imu': 120, '/e2e_live_lidar': 3}
lc         = <LCM object at 0x366e07690>
logs       = 'mid360_native stderr:\n:module"}\n{"timestamp":"2026-11-11T11:55:18.847815Z","level":"ERROR","fields":{"message":"pub...:"2026-11-11T11:55:19.022544Z","level":"INFO","fields":{"message":"data stream finished"},"target":"virtual_mid360"}\n'
msg_type   = <class 'dimos.msgs.sensor_msgs.Imu.Imu'>
on_msg     = <function Spawner.collect.<locals>.on_msg at 0x36512b9c0>
raw        = {'/e2e_live_imu': [], '/e2e_live_lidar': []}
seconds    = 10.0
self       = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e07860>
subscription = None
subscriptions = [None, None]
topic      = '/e2e_live_imu'
topics     = {'/e2e_live_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_live_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}

.../lidar/livox/test_e2e.py:195: TypeError

dimos.core.test_stream::test_subscription[ClassicSubscriber]

Stack Traces | 33.5s run time

dimos = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x364e29b50>
subscriber_class = <class 'dimos.core.test_stream.ClassicSubscriber'>

    @pytest.mark.self_hosted
    @pytest.mark.parametrize("subscriber_class", [ClassicSubscriber, RXPYSubscriber])
    def test_subscription(dimos, subscriber_class) -> None:
        robot = dimos.deploy(MockRobotClient)
    
        robot.lidar.transport = SpyLCMTransport("/lidar", PointCloud2)
        robot.odometry.transport = SpyLCMTransport("/odom", Odometry)
        robot.mov.transport = pLCMTransport("/mov")
    
        subscriber = dimos.deploy(subscriber_class)
    
        subscriber.odom.connect(robot.odometry)
    
        robot.start()
        subscriber.sub1()
>       assert subscriber.wait_for_sub1_msg(), "no odometry reached sub1 within its wait"
E       AssertionError: no odometry reached sub1 within its wait
E       assert False
E        +  where False = <dimos.core.rpc_client.RpcCall object at 0x364cde7b0>()
E        +    where <dimos.core.rpc_client.RpcCall object at 0x364cde7b0> = <dimos.core.rpc_client.RPCClient object at 0x364e290a0>.wait_for_sub1_msg

dimos      = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x364e29b50>
robot      = <dimos.core.rpc_client.RPCClient object at 0x364e289b0>
subscriber = <dimos.core.rpc_client.RPCClient object at 0x364e290a0>
subscriber_class = <class 'dimos.core.test_stream.ClassicSubscriber'>

dimos/core/test_stream.py:196: AssertionError

dimos.core.test_stream::test_subscription[RXPYSubscriber]

Stack Traces | 43.2s run time

dimos = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x3651446e0>
subscriber_class = <class 'dimos.core.test_stream.RXPYSubscriber'>

    @pytest.mark.self_hosted
    @pytest.mark.parametrize("subscriber_class", [ClassicSubscriber, RXPYSubscriber])
    def test_subscription(dimos, subscriber_class) -> None:
        robot = dimos.deploy(MockRobotClient)
    
        robot.lidar.transport = SpyLCMTransport("/lidar", PointCloud2)
        robot.odometry.transport = SpyLCMTransport("/odom", Odometry)
        robot.mov.transport = pLCMTransport("/mov")
    
        subscriber = dimos.deploy(subscriber_class)
    
        subscriber.odom.connect(robot.odometry)
    
        robot.start()
        subscriber.sub1()
>       assert subscriber.wait_for_sub1_msg(), "no odometry reached sub1 within its wait"
E       AssertionError: no odometry reached sub1 within its wait
E       assert False
E        +  where False = <dimos.core.rpc_client.RpcCall object at 0x365145400>()
E        +    where <dimos.core.rpc_client.RpcCall object at 0x365145400> = <dimos.core.rpc_client.RPCClient object at 0x365144fb0>.wait_for_sub1_msg

dimos      = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x3651446e0>
robot      = <dimos.core.rpc_client.RPCClient object at 0x3651445c0>
subscriber = <dimos.core.rpc_client.RPCClient object at 0x365144fb0>
subscriber_class = <class 'dimos.core.test_stream.RXPYSubscriber'>

dimos/core/test_stream.py:196: AssertionError

dimos.core.test_stream::test_hot_getter

Stack Traces | 52.9s run time

dimos.protocol.rpc.rpc_utils.RemoteError: [Remote builtins.TimeoutError] No value received after 30.0 s

Remote traceback:
Traceback (most recent call last):
  File ".../protocol/rpc/zenohrpc.py", line 166, in _execute_rpc
    response = f(*args[0], **args[1])
               ^^^^^^^^^^^^^^^^^^^^^^
  File ".../protocol/rpc/spec.py", line 116, in override_f
    return getattr(module, fname)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/core/test_stream.py", line 143, in start_hot_getter
    self.hot = self.odom.hot_latest()
               ^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/core/stream.py", line 70, in hot_latest
    return reactive.getter_streaming(self.observable())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/utils/reactive.py", line 255, in getter_streaming
    raise TimeoutError(f"No value received after {timeout} s")
TimeoutError: No value received after 30.0 s


The above exception was the direct cause of the following exception:

dimos = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x365146900>

    @pytest.mark.self_hosted
    def test_hot_getter(dimos) -> None:
        robot = dimos.deploy(MockRobotClient)
    
        robot.lidar.transport = SpyLCMTransport("/lidar", PointCloud2)
        robot.odometry.transport = SpyLCMTransport("/odom", Odometry)
        robot.mov.transport = pLCMTransport("/mov")
    
        subscriber = dimos.deploy(RXPYSubscriber)
        subscriber.odom.connect(robot.odometry)
    
        robot.start()
    
        # we are robust to multiple calls
>       subscriber.start_hot_getter()

dimos      = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x365146900>
robot      = <dimos.core.rpc_client.RPCClient object at 0x365146060>
subscriber = <dimos.core.rpc_client.RPCClient object at 0x3650f6f30>

dimos/core/test_stream.py:269: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/rpc_client.py:94: in __call__
    result, unsub_fn = self._rpc.call_sync(f"{self._remote_name}/{self._name}", (args, kwargs))
        args       = ()
        kwargs     = {}
        self       = <dimos.core.rpc_client.RpcCall object at 0x3650f6ea0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.rpc.zenohrpc.ZenohRPC object at 0x365146d50>
name = 'RXPYSubscriber/start_hot_getter', arguments = ((), {})
rpc_timeout = 120.0

    def call_sync(
        self, name: str, arguments: Args, rpc_timeout: float | None = None
    ) -> tuple[Any, Callable[[], None]]:
        if rpc_timeout is None:
            method = name.rsplit("/", 1)[-1]
            rpc_timeout = self.rpc_timeouts.get(name) or self.rpc_timeouts.get(
                method, self.default_rpc_timeout
            )
        event = threading.Event()
    
        def receive_value(val) -> None:  # type: ignore[no-untyped-def]
            event.result = val  # type: ignore[attr-defined]  # attach to event
            event.set()
    
        unsub_fn = self.call(name, arguments, receive_value)
        if not event.wait(rpc_timeout):
            # Retries register new callbacks. Remove this expired callback so
            # repeated timeouts do not accumulate entries in the shared response map.
            unsub_fn()
            raise TimeoutError(f"RPC call to '{name}' timed out after {rpc_timeout} seconds")
    
        # Check if the result is an exception and raise it
        result = event.result  # type: ignore[attr-defined]
        if isinstance(result, BaseException):
>           raise result
E           TimeoutError: No value received after 30.0 s

arguments  = ((), {})
event      = <threading.Event at 0x365147140: set>
method     = 'start_hot_getter'
name       = 'RXPYSubscriber/start_hot_getter'
receive_value = <function RPCClient.call_sync.<locals>.receive_value at 0x365129a80>
result     = TimeoutError('No value received after 30.0 s')
rpc_timeout = 120.0
self       = <dimos.protocol.rpc.zenohrpc.ZenohRPC object at 0x365146d50>
unsub_fn   = <function ZenohRPC.call_cb.<locals>.unsubscribe_callback at 0x365129760>

.../protocol/rpc/spec.py:88: TimeoutError

View the full list of 8 ❄️ flaky test(s)

dimos.core.test_core::test_basic_deployment

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 29.4s run time

dimos.protocol.rpc.rpc_utils.RemoteError: [Remote builtins.RuntimeError] LCM handler thread failed to start within 5s

Remote traceback:
Traceback (most recent call last):
  File ".../protocol/rpc/zenohrpc.py", line 166, in _execute_rpc
    response = f(*args[0], **args[1])
               ^^^^^^^^^^^^^^^^^^^^^^
  File ".../protocol/rpc/spec.py", line 116, in override_f
    return getattr(module, fname)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/core/test_core.py", line 49, in start
    unsub = self.odometry.subscribe(_odom)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/core/stream.py", line 377, in subscribe
    return self.transport.subscribe(cb, self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../dimos/core/transport.py", line 140, in subscribe
    self.start()
  File ".../dimos/core/transport.py", line 144, in start
    self.lcm.start()
  File ".../protocol/service/lcmservice.py", line 137, in start
    raise RuntimeError("LCM handler thread failed to start within 5s")
RuntimeError: LCM handler thread failed to start within 5s


The above exception was the direct cause of the following exception:

dimos = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x364d246b0>

    @pytest.mark.self_hosted
    def test_basic_deployment(dimos) -> None:
        robot = dimos.deploy(MockRobotClient)
    
        print("\n")
        print("lidar stream", robot.lidar)
        print("odom stream", robot.odometry)
    
        nav = dimos.deploy(Navigation)
    
        # this one encodes proper LCM messages
        robot.lidar.transport = LCMTransport("/lidar", PointCloud2)
    
        # odometry & mov on the untyped (CBOR) LCM transport
        robot.odometry.transport = pLCMTransport("/odom")
        nav.mov.transport = pLCMTransport("/mov")
    
        nav.lidar.connect(robot.lidar)
        nav.odometry.connect(robot.odometry)
        robot.mov.connect(nav.mov)
    
        robot.start()
>       nav.start()

dimos      = <dimos.core.coordination.module_coordinator.ModuleCoordinator object at 0x364d246b0>
nav        = <dimos.core.rpc_client.RPCClient object at 0x364cdd610>
robot      = <dimos.core.rpc_client.RPCClient object at 0x364d27470>

dimos/core/test_core.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/core/rpc_client.py:94: in __call__
    result, unsub_fn = self._rpc.call_sync(f"{self._remote_name}/{self._name}", (args, kwargs))
        args       = ()
        kwargs     = {}
        self       = <dimos.core.rpc_client.RpcCall object at 0x364cdec00>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.protocol.rpc.zenohrpc.ZenohRPC object at 0x364cddaf0>
name = 'Navigation/start', arguments = ((), {}), rpc_timeout = 1200.0

    def call_sync(
        self, name: str, arguments: Args, rpc_timeout: float | None = None
    ) -> tuple[Any, Callable[[], None]]:
        if rpc_timeout is None:
            method = name.rsplit("/", 1)[-1]
            rpc_timeout = self.rpc_timeouts.get(name) or self.rpc_timeouts.get(
                method, self.default_rpc_timeout
            )
        event = threading.Event()
    
        def receive_value(val) -> None:  # type: ignore[no-untyped-def]
            event.result = val  # type: ignore[attr-defined]  # attach to event
            event.set()
    
        unsub_fn = self.call(name, arguments, receive_value)
        if not event.wait(rpc_timeout):
            # Retries register new callbacks. Remove this expired callback so
            # repeated timeouts do not accumulate entries in the shared response map.
            unsub_fn()
            raise TimeoutError(f"RPC call to '{name}' timed out after {rpc_timeout} seconds")
    
        # Check if the result is an exception and raise it
        result = event.result  # type: ignore[attr-defined]
        if isinstance(result, BaseException):
>           raise result
E           RuntimeError: LCM handler thread failed to start within 5s

arguments  = ((), {})
event      = <threading.Event at 0x364e91220: set>
method     = 'start'
name       = 'Navigation/start'
receive_value = <function RPCClient.call_sync.<locals>.receive_value at 0x364cf4400>
result     = RuntimeError('LCM handler thread failed to start within 5s')
rpc_timeout = 1200.0
self       = <dimos.protocol.rpc.zenohrpc.ZenohRPC object at 0x364cddaf0>
unsub_fn   = <function ZenohRPC.call_cb.<locals>.unsubscribe_callback at 0x364cf6200>

.../protocol/rpc/spec.py:88: RuntimeError

dimos.hardware.sensors.lidar.livox.test_e2e::test_real_capture_replay_publishes_streams

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 41.6s run time

self = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>
topics = {'/e2e_real_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_real_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}
seconds = 15.0, enough = {'/e2e_real_imu': 800, '/e2e_real_lidar': 20}

    @contextmanager
    def collect(
        self, topics: dict[str, type[PointCloud2 | Imu]], seconds: float, enough: dict[str, int]
    ) -> Iterator[dict[str, list[bytes]]]:
        """Subscribe before yielding, then collect after the caller starts the binaries."""
        raw: dict[str, list[bytes]] = {topic: [] for topic in topics}
        lc = lcm_module.LCM()
        subscriptions = []
        try:
            for topic in topics:
                # Buffer bytes only. Decoding inline drops packets.
                def on_msg(_ch: str, data: bytes, topic: str = topic) -> None:
                    raw[topic].append(data)
    
                msg_type = topics[topic]
                subscriptions.append(
                    lc.subscribe(f"{topic}#sensor_msgs.{msg_type.__name__}", on_msg)
                )
            yield raw
            end = time.monotonic() + seconds
            while not all(len(raw[topic]) >= count for topic, count in enough.items()):
                self._assert_running()
                if time.monotonic() >= end:
                    counts = {topic: len(messages) for topic, messages in raw.items()}
                    logs = "\n".join(
                        f"{binary.name} stderr:\n{stderr.read_text(errors='replace')[-4000:]}"
                        for binary, _, stderr in self._spawned
                    )
>                   pytest.fail(
                        f"Timed out collecting streams: {counts}, expected {enough}\n{logs}"
                    )
E                   Failed: Timed out collecting streams: {'/e2e_real_lidar': 0, '/e2e_real_imu': 0}, expected {'/e2e_real_lidar': 20, '/e2e_real_imu': 800}
E                   mid360_native stderr:
E                   {"timestamp":"2026-11-11T11:54:55.936149Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.936163Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.939200Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_lidar#sensor_msgs.PointCloud2","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.950659Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.950693Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.952731Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.958960Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.962502Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.971768Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.971795Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.976882Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.981522Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.987377Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.994400Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:55.997486Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:56.004434Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:56.006668Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:56.011782Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}
E                   {"timestamp":"2026-11-11T11:54:56.018648Z","level":"ERROR","fields":{"message":"publish error","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}

counts     = {'/e2e_real_imu': 0, '/e2e_real_lidar': 0}
end        = 26045.791537833
enough     = {'/e2e_real_imu': 800, '/e2e_real_lidar': 20}
lc         = <LCM object at 0x366e08cc0>
logs       = 'mid360_native stderr:\n{"timestamp":"2026-11-11T11:54:55.936149Z","level":"ERROR","fields":{"message":"publish error"...","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}\n'
msg_type   = <class 'dimos.msgs.sensor_msgs.Imu.Imu'>
on_msg     = <function Spawner.collect.<locals>.on_msg at 0x364b4c400>
raw        = {'/e2e_real_imu': [], '/e2e_real_lidar': []}
seconds    = 15.0
self       = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>
subscription = None
subscriptions = [None, None]
topic      = '/e2e_real_imu'
topics     = {'/e2e_real_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_real_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}

.../lidar/livox/test_e2e.py:189: Failed

During handling of the above exception, another exception occurred:

spawn = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>

    @pytest.mark.self_hosted
    def test_real_capture_replay_publishes_streams(spawn: Spawner) -> None:
        """Replay a real Mid-360 recording: the non-circular wire-format check."""
        binary = _require_binary("mid360_native")
        pcap = get_data("mid360_shake_stairs/mid360_shake_stairs.pcap")
        config = Mid360Config(
            pcap=str(pcap),
            replay_rate=1.0,
            multicast_ip=None,
            point_format="full",
        )
        blob = {
            "topics": {
                "lidar": "/e2e_real_lidar#sensor_msgs.PointCloud2",
                "imu": "/e2e_real_imu#sensor_msgs.Imu",
            },
            "config": config.to_config_dict(),
            "session": {"id": "pointlio-e2e", "links": []},
        }
>       with spawn.collect(
            {"/e2e_real_lidar": PointCloud2, "/e2e_real_imu": Imu},
            seconds=15.0,
            enough={"/e2e_real_lidar": 20, "/e2e_real_imu": 800},
        ) as received:

binary     = PosixPath('.../result/bin/mid360_native')
blob       = {'config': {'cmd_data_port': 56100, 'enable_imu': True, 'frame_id': 'lidar_link', 'frequency': 10.0, ...}, 'session': ..., 'links': []}, 'topics': {'imu': '/e2e_real_imu#sensor_msgs.Imu', 'lidar': '/e2e_real_lidar#sensor_msgs.PointCloud2'}}
config     = Mid360Config(rpc_transport=<class 'dimos.protocol.rpc.zenohrpc.ZenohRPC'>, default_rpc_timeout=120.0, rpc_timeouts={'b...a_port=56400, host_cmd_data_port=56101, host_push_msg_port=56201, host_point_data_port=56301, host_imu_data_port=56401)
pcap       = PosixPath('.../data/mid360_shake_stairs/mid360_shake_stairs.pcap')
received   = {'/e2e_real_imu': [], '/e2e_real_lidar': []}
spawn      = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>

.../lidar/livox/test_e2e.py:265: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/contextlib.py:144: in __exit__
    next(self.gen)
        self       = <contextlib._GeneratorContextManager object at 0x366e08560>
        traceback  = None
        typ        = None
        value      = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>
topics = {'/e2e_real_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_real_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}
seconds = 15.0, enough = {'/e2e_real_imu': 800, '/e2e_real_lidar': 20}

    @contextmanager
    def collect(
        self, topics: dict[str, type[PointCloud2 | Imu]], seconds: float, enough: dict[str, int]
    ) -> Iterator[dict[str, list[bytes]]]:
        """Subscribe before yielding, then collect after the caller starts the binaries."""
        raw: dict[str, list[bytes]] = {topic: [] for topic in topics}
        lc = lcm_module.LCM()
        subscriptions = []
        try:
            for topic in topics:
                # Buffer bytes only. Decoding inline drops packets.
                def on_msg(_ch: str, data: bytes, topic: str = topic) -> None:
                    raw[topic].append(data)
    
                msg_type = topics[topic]
                subscriptions.append(
                    lc.subscribe(f"{topic}#sensor_msgs.{msg_type.__name__}", on_msg)
                )
            yield raw
            end = time.monotonic() + seconds
            while not all(len(raw[topic]) >= count for topic, count in enough.items()):
                self._assert_running()
                if time.monotonic() >= end:
                    counts = {topic: len(messages) for topic, messages in raw.items()}
                    logs = "\n".join(
                        f"{binary.name} stderr:\n{stderr.read_text(errors='replace')[-4000:]}"
                        for binary, _, stderr in self._spawned
                    )
                    pytest.fail(
                        f"Timed out collecting streams: {counts}, expected {enough}\n{logs}"
                    )
                lc.handle_timeout(200)
        finally:
            for subscription in subscriptions:
>               lc.unsubscribe(subscription)
E               TypeError: argument 1 must be LCMSubscription, not None

counts     = {'/e2e_real_imu': 0, '/e2e_real_lidar': 0}
end        = 26045.791537833
enough     = {'/e2e_real_imu': 800, '/e2e_real_lidar': 20}
lc         = <LCM object at 0x366e08cc0>
logs       = 'mid360_native stderr:\n{"timestamp":"2026-11-11T11:54:55.936149Z","level":"ERROR","fields":{"message":"publish error"...","topic":"/e2e_real_imu#sensor_msgs.Imu","error":"No route to host (os error 65)"},"target":"dimos_module::module"}\n'
msg_type   = <class 'dimos.msgs.sensor_msgs.Imu.Imu'>
on_msg     = <function Spawner.collect.<locals>.on_msg at 0x364b4c400>
raw        = {'/e2e_real_imu': [], '/e2e_real_lidar': []}
seconds    = 15.0
self       = <dimos.hardware.sensors.lidar.livox.test_e2e.Spawner object at 0x366e088f0>
subscription = None
subscriptions = [None, None]
topic      = '/e2e_real_imu'
topics     = {'/e2e_real_imu': <class 'dimos.msgs.sensor_msgs.Imu.Imu'>, '/e2e_real_lidar': <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>}

.../lidar/livox/test_e2e.py:195: TypeError

dimos.navigation.bench.test_driver::test_driver_resets_seeds_sends_the_goal_and_ends_on_arrival

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 70.2s run time

tmp_path = PosixPath('.../pytest-of-admin/pytest-13/test_driver_resets_seeds_sends0')

    @pytest.mark.self_hosted
    def test_driver_resets_seeds_sends_the_goal_and_ends_on_arrival(tmp_path: Path) -> None:
        scene = office(1)
        start = (scene.start[0], scene.start[1], 0.0)
        case = Case(
            family="office",
            seed=1,
            params={},
            start=start,
            goal=(start[0] + 3.0, start[1], float(scene.params["z0"])),
            tag="mined",
            id="mined-s1-test",
            split="dev",
            difficulty=Difficulty(0.5, 0, 1.0, 0),
            route_length=3.0,
            scene_digest=scene.digest(),
        )
        Manifest("test", Rules(), 0, [case]).save(tmp_path / "suite.json")
        transports = {
            name: LCMTransport(f"/test_driver/{name}", kind)
            for name, kind in (
                ("ground_truth", PoseStamped),
                ("goal", PointStamped),
                ("goal_reached", Bool),
                ("cmd_vel", Twist),
            )
        }
        driver = EpisodeDriver(
            manifest=tmp_path / "suite.json",
            case_id=case.id,
            out_dir=tmp_path,
            settle_s=0.2,
            seed_settle_s=0.2,
            goal_resend_s=0.2,
            goal_wait_s=3.0,
        )
        for name, transport in transports.items():
            getattr(driver, name).transport = transport
        world = FakeWorld(transports["ground_truth"], z=scene.params["z0"])
        driver._world = world
        clicks: list[PointStamped] = []
        routes: list[PathMsg] = []
        phases: list[str] = []
        driver.clicked_point.subscribe(clicks.append)
        driver.reference_path.subscribe(routes.append)
        driver.phase.subscribe(lambda msg: phases.append(msg.data))
        world.thread.start()
        driver.start()
        try:
>           _until(lambda: clicks, 30.0)

case       = Case(family='office', seed=1, params={}, start=(1.0, 1.0, 0.0), goal=(4.0, 1.0, 0.011532769017570698), tag='mined', id...1.0, clutter_near=0), route_length=3.0, scene_digest='7af500c57b3a5cc5', level=None, feature=None, premap_objects=True)
clicks     = []
driver     = <dimos.navigation.bench.driver.EpisodeDriver object at 0x369205c40>
name       = 'cmd_vel'
phases     = []
routes     = []
scene      = Scene(name='office_1', params={'width': 15.07092974820154, 'length': 12.80185478530374, 'z0': 0.011532769017570698, 'r...s=0, at=6.95769402356104, start=10.804005098099532, width=0.9318926865996369)], start=(1.0, 1.0, 0.011532769017570698))
start      = (1.0, 1.0, 0.0)
tmp_path   = PosixPath('.../pytest-of-admin/pytest-13/test_driver_resets_seeds_sends0')
transport  = <dimos.core.transport.LCMTransport object at 0x366fa7350>
transports = {'cmd_vel': <dimos.core.transport.LCMTransport object at 0x366fa7350>, 'goal': <dimos.core.transport.LCMTransport obje...ansport.LCMTransport object at 0x366fa70e0>, 'ground_truth': <dimos.core.transport.LCMTransport object at 0x3679bda30>}
world      = <dimos.navigation.bench.test_driver.FakeWorld object at 0x3679b42f0>

.../navigation/bench/test_driver.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

done = <function test_driver_resets_seeds_sends_the_goal_and_ends_on_arrival.<locals>.<lambda> at 0x366f73740>
timeout_s = 30.0

    def _until(done: object, timeout_s: float) -> None:
        deadline = time.time() + timeout_s
        while not done() and time.time() < deadline:  # type: ignore[operator]
            time.sleep(0.05)
>       assert done()  # type: ignore[operator]
E       assert []
E        +  where [] = <function test_driver_resets_seeds_sends_the_goal_and_ends_on_arrival.<locals>.<lambda> at 0x366f73740>()

deadline   = 1794398345.0084128
done       = <function test_driver_resets_seeds_sends_the_goal_and_ends_on_arrival.<locals>.<lambda> at 0x366f73740>
timeout_s  = 30.0

.../navigation/bench/test_driver.py:82: AssertionError

dimos.protocol.pubsub.test_spec::test_high_volume_messages[lcm_context-topic1-values1]

Flake rate in main: 50.00% (Passed 1 times, Failed 1 times)

Stack Traces | 40.4s run time

pubsub_context = <function lcm_context at 0x317f5be20>
topic = Topic(topic='/test_topic', lcm_type=<class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>)
values = [Vector([          1           2           3]), Vector([          4           5           6]), Vector([          7           8           9])]

    @pytest.mark.flaky(reruns=1)
    @pytest.mark.self_hosted
    @pytest.mark.parametrize("pubsub_context, topic, values", testdata)
    def test_high_volume_messages(
        pubsub_context: Callable[[], Any], topic: Any, values: list[Any]
    ) -> None:
        """Test that all messages are received correctly under moderate volume.
        This is an acceptance test, not a benchmark, so volume is kept low (500)
        to avoid flakiness. Might want separate expectations per transport later.
        """
        if pubsub_context is zenoh_cbor_context and os.getenv("CI"):
            pytest.skip("flaky in CI, disabled until fixed")
        with pubsub_context() as x:
            received_messages: list[Any] = []
    
            def callback(message: Any, topic: Any) -> None:
                received_messages.append(message)
    
            x.subscribe(topic, callback)
    
            num_messages = 500
            for _ in range(num_messages):
                x.publish(topic, values[0])
    
            # Wait for the whole batch. A loaded runner delivers with pauses, so
            # "quiet for 100 ms" is not a sign that delivery has finished.
            deadline = time.monotonic() + 10.0
            while len(received_messages) < num_messages and time.monotonic() < deadline:
                time.sleep(0.05)
    
            # Capture count and clear list to avoid printing huge list on failure
            received_len = len(received_messages)
            received_messages.clear()
>           assert received_len == num_messages, f"Expected {num_messages} messages, got {received_len}"
E           AssertionError: Expected 500 messages, got 0
E           assert 0 == 500

_          = 499
callback   = <function test_high_volume_messages.<locals>.callback at 0x367b539c0>
deadline   = 26385.748210458
num_messages = 500
pubsub_context = <function lcm_context at 0x317f5be20>
received_len = 0
received_messages = []
topic      = Topic(topic='/test_topic', lcm_type=<class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>)
values     = [Vector([          1           2           3]), Vector([          4           5           6]), Vector([          7           8           9])]
x          = <dimos.protocol.pubsub.impl.lcmpubsub.LCM object at 0x36964fa70>

.../protocol/pubsub/test_spec.py:416: AssertionError

dimos.web.relay_bridge.test_relay_bridge_e2e::test_costmap_full_grid_arrives_and_resends_on_subscribe

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 25.2s run time

bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
expected_cells = b'\xff\x002d\x00\xff'

    async def _watch_costmap(bridge: RelayBridgeModule, expected_cells: bytes = COSTMAP_CELLS) -> Any:
        """Attach a fresh viewer, wait for one costmap frame, return that frame."""
        async with await _viewer(bridge) as viewer:
            await viewer.hello()
            await attach_viewer(viewer, ROBOT_ID, ["global_costmap"])
            frames = await collect_until(
                viewer,
                lambda fs: any(f.header.ch == "global_costmap" for f in fs),
                timeout=15.0,
            )
>           frame = next(f for f in frames if f.header.ch == "global_costmap")
E           StopIteration

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
expected_cells = b'\xff\x002d\x00\xff'
frames     = []
viewer     = <dimos.web.relay_bridge.wt_client.RelayClient object at 0x3e38af620>

.../web/relay_bridge/test_relay_bridge_e2e.py:306: StopIteration

The above exception was the direct cause of the following exception:

bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>

    def test_costmap_full_grid_arrives_and_resends_on_subscribe(bridge: RelayBridgeModule) -> None:
        publisher = _CostmapPublisher()
        publisher.start()
        try:
>           first = asyncio.run(_watch_costmap(bridge))

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
publisher  = <dimos.web.relay_bridge.test_relay_bridge_e2e._CostmapPublisher object at 0x3dea86840>

.../web/relay_bridge/test_relay_bridge_e2e.py:336: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:195: in run
    return runner.run(main)
        debug      = None
        loop_factory = None
        main       = <coroutine object _watch_costmap at 0x3dea245e0>
        runner     = <asyncio.runners.Runner object at 0x3de730ce0>
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:118: in run
    return self._loop.run_until_complete(task)
        context    = <_contextvars.Context object at 0x3dea76fc0>
        coro       = <coroutine object _watch_costmap at 0x3dea245e0>
        self       = <asyncio.runners.Runner object at 0x3de730ce0>
        sigint_handler = functools.partial(<bound method Runner._on_sigint of <asyncio.runners.Runner object at 0x3de730ce0>>, main_task=<Task ...s/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:296> exception=RuntimeError('coroutine raised StopIteration')>)
        task       = <Task finished name='Task-104' coro=<_watch_costmap() done, defined at /Users/admin/actions-runner/_work/dimos/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:296> exception=RuntimeError('coroutine raised StopIteration')>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>
future = <Task finished name='Task-104' coro=<_watch_costmap() done, defined at /Users/admin/actions-runner/_work/dimos/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:296> exception=RuntimeError('coroutine raised StopIteration')>

    def run_until_complete(self, future):
        """Run until the Future is done.
    
        If the argument is a coroutine, it is wrapped in a Task.
    
        WARNING: It would be disastrous to call run_until_complete()
        with the same coroutine twice -- it would wrap it in two
        different Tasks and that can't be good.
    
        Return the Future's result, or raise its exception.
        """
        self._check_closed()
        self._check_running()
    
        new_task = not futures.isfuture(future)
        future = tasks.ensure_future(future, loop=self)
        if new_task:
            # An exception is raised if the future didn't complete, so there
            # is no need to log the "destroy pending task" message
            future._log_destroy_pending = False
    
        future.add_done_callback(_run_until_complete_cb)
        try:
            self.run_forever()
        except:
            if new_task and future.done() and not future.cancelled():
                # The coroutine raised a BaseException. Consume the exception
                # to not log a warning, the caller doesn't have access to the
                # local task.
                future.exception()
            raise
        finally:
            future.remove_done_callback(_run_until_complete_cb)
        if not future.done():
            raise RuntimeError('Event loop stopped before Future completed.')
    
>       return future.result()
E       RuntimeError: coroutine raised StopIteration

future     = <Task finished name='Task-104' coro=<_watch_costmap() done, defined at /Users/admin/actions-runner/_work/dimos/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:296> exception=RuntimeError('coroutine raised StopIteration')>
new_task   = False
self       = <_UnixSelectorEventLoop running=False closed=True debug=False>

../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/base_events.py:691: RuntimeError

dimos.web.relay_bridge.test_relay_bridge_e2e::test_costmap_replay_reflects_publishes_while_unwatched

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 20.2s run time

bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>

    def test_costmap_replay_reflects_publishes_while_unwatched(bridge: RelayBridgeModule) -> None:
        # A different grid published with zero viewers must land in the raw cache,
        # so the next subscriber gets it - stamped with its arrival time (honest
        # staleness on the wire), not the replay time.
        _wait_costmap_unsubscribed(bridge)
        grid_b = OccupancyGrid(
            grid=np.array([[100, 100, 100], [0, 0, -1]], dtype=np.int8),
            resolution=0.05,
            origin=Pose(-1.25, 2.5, 0.0),
            ts=43.0,
        )
        t0 = time.time()
        publisher = _CostmapPublisher(grid_b)
        publisher.start()
        try:
            deadline = time.monotonic() + 10
            while time.monotonic() < deadline:
                cached = bridge._last_msg.get("global_costmap")
                if cached is not None and cached[0].grid[0, 0] == 100:
                    break
                time.sleep(0.05)
        finally:
            publisher.close()
        t1 = time.time()
        cached = bridge._last_msg.get("global_costmap")
>       assert cached is not None and cached[0].grid[0, 0] == 100, "cache never saw grid B"
E       AssertionError: cache never saw grid B
E       assert (None is not None)

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
cached     = None
deadline   = 26645.703619791
grid_b     = OccupancyGrid(width=3, height=2, resolution=0.05, frame_id='world', occupied=3, free=2, unknown=1)
publisher  = <dimos.web.relay_bridge.test_relay_bridge_e2e._CostmapPublisher object at 0x3de887bc0>
t0         = 1794398670.146608
t1         = 1794398690.3168151

.../web/relay_bridge/test_relay_bridge_e2e.py:376: AssertionError

dimos.web.relay_bridge.test_relay_bridge_e2e::test_full_session_flow_and_lazy_encode

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 81.2s run time

async def flow() -> None:
        async with await _viewer(bridge) as viewer:
            await viewer.hello()
            # robots push carries the bridge's identity (drain until watch lands).
            await attach_viewer(viewer, ROBOT_ID, ["odom", "color_image"])
    
            frames = await collect_until(
                viewer,
                lambda fs: any(f.header.ch == "odom" for f in fs)
                and any(f.header.ch == "color_image" for f in fs),
                timeout=15.0,
            )
>           odom = next(f for f in frames if f.header.ch == "odom")
E           StopIteration

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
frames     = []
viewer     = <dimos.web.relay_bridge.wt_client.RelayClient object at 0x3def6d8e0>

.../web/relay_bridge/test_relay_bridge_e2e.py:239: StopIteration

The above exception was the direct cause of the following exception:

bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
publisher = <dimos.web.relay_bridge.test_relay_bridge_e2e._Publisher object at 0x3def6f110>

    @pytest.mark.flaky(reruns=1)
    def test_full_session_flow_and_lazy_encode(
        bridge: RelayBridgeModule, publisher: _Publisher
    ) -> None:
        async def flow() -> None:
            async with await _viewer(bridge) as viewer:
                await viewer.hello()
                # robots push carries the bridge's identity (drain until watch lands).
                await attach_viewer(viewer, ROBOT_ID, ["odom", "color_image"])
    
                frames = await collect_until(
                    viewer,
                    lambda fs: any(f.header.ch == "odom" for f in fs)
                    and any(f.header.ch == "color_image" for f in fs),
                    timeout=15.0,
                )
                odom = next(f for f in frames if f.header.ch == "odom")
                pose = json.loads(odom.payload)
                assert pose == {"x": 1.5, "y": -2.5, "z": 0.25, "yaw": 0.0, "ts": 42.5}
                assert odom.header.delivery == "reliable"
    
                image = next(f for f in frames if f.header.ch == "color_image")
                assert bytes(image.payload[:2]) == b"\xff\xd8"  # real TurboJPEG output
                assert image.header.meta == {"w": 64, "h": 48}
                assert image.header.delivery == "latest"
    
                # Unsub color_image: the bridge must stop encoding it entirely
                # while odom (still subscribed) keeps flowing.
                viewer.send_control(Unsub(ch="color_image"))
                deadline = time.monotonic() + 10
                while (
                    bridge._session is not None
                    and "color_image" in bridge._session.unsubs
                    and time.monotonic() < deadline
                ):
                    await asyncio.sleep(0.05)
                assert bridge._session is not None
                assert "color_image" not in bridge._session.unsubs, "bridge never heard the unsub"
    
                encoded_before = bridge.encoded["color_image"]
                await asyncio.sleep(0.5)  # publisher keeps publishing the whole time
                assert bridge.encoded["color_image"] == encoded_before
                assert "odom" in bridge._session.unsubs
    
>       asyncio.run(flow())

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3def6e600>
flow       = <function test_full_session_flow_and_lazy_encode.<locals>.flow at 0x3dea8b380>
publisher  = <dimos.web.relay_bridge.test_relay_bridge_e2e._Publisher object at 0x3def6f110>

.../web/relay_bridge/test_relay_bridge_e2e.py:267: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:195: in run
    return runner.run(main)
        debug      = None
        loop_factory = None
        main       = <coroutine object test_full_session_flow_and_lazy_encode.<locals>.flow at 0x3dea79840>
        runner     = <asyncio.runners.Runner object at 0x3def6ecf0>
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:118: in run
    return self._loop.run_until_complete(task)
        context    = <_contextvars.Context object at 0x3e3a30080>
        coro       = <coroutine object test_full_session_flow_and_lazy_encode.<locals>.flow at 0x3dea79840>
        self       = <asyncio.runners.Runner object at 0x3def6ecf0>
        sigint_handler = functools.partial(<bound method Runner._on_sigint of <asyncio.runners.Runner object at 0x3def6ecf0>>, main_task=<Task ...s/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:227> exception=RuntimeError('coroutine raised StopIteration')>)
        task       = <Task finished name='Task-93' coro=<test_full_session_flow_and_lazy_encode.<locals>.flow() done, defined at /Users/adm...os/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:227> exception=RuntimeError('coroutine raised StopIteration')>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>
future = <Task finished name='Task-93' coro=<test_full_session_flow_and_lazy_encode.<locals>.flow() done, defined at /Users/adm...os/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:227> exception=RuntimeError('coroutine raised StopIteration')>

    def run_until_complete(self, future):
        """Run until the Future is done.
    
        If the argument is a coroutine, it is wrapped in a Task.
    
        WARNING: It would be disastrous to call run_until_complete()
        with the same coroutine twice -- it would wrap it in two
        different Tasks and that can't be good.
    
        Return the Future's result, or raise its exception.
        """
        self._check_closed()
        self._check_running()
    
        new_task = not futures.isfuture(future)
        future = tasks.ensure_future(future, loop=self)
        if new_task:
            # An exception is raised if the future didn't complete, so there
            # is no need to log the "destroy pending task" message
            future._log_destroy_pending = False
    
        future.add_done_callback(_run_until_complete_cb)
        try:
            self.run_forever()
        except:
            if new_task and future.done() and not future.cancelled():
                # The coroutine raised a BaseException. Consume the exception
                # to not log a warning, the caller doesn't have access to the
                # local task.
                future.exception()
            raise
        finally:
            future.remove_done_callback(_run_until_complete_cb)
        if not future.done():
            raise RuntimeError('Event loop stopped before Future completed.')
    
>       return future.result()
E       RuntimeError: coroutine raised StopIteration

future     = <Task finished name='Task-93' coro=<test_full_session_flow_and_lazy_encode.<locals>.flow() done, defined at /Users/adm...os/dimos/.../web/relay_bridge/test_relay_bridge_e2e.py:227> exception=RuntimeError('coroutine raised StopIteration')>
new_task   = False
self       = <_UnixSelectorEventLoop running=False closed=True debug=False>

../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/base_events.py:691: RuntimeError

dimos.web.relay_bridge.test_relay_bridge_e2e::test_relay_child_death_respawns_and_recovers

Flake rate in main: 16.67% (Passed 5 times, Failed 1 times)

Stack Traces | 46.4s run time

respawn_bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3dea87680>
publisher = <dimos.web.relay_bridge.test_relay_bridge_e2e._Publisher object at 0x3def6f110>

    def test_relay_child_death_respawns_and_recovers(
        respawn_bridge: RelayBridgeModule, publisher: _Publisher
    ) -> None:
        bridge = respawn_bridge
        assert bridge._relay is not None and bridge._relay._process is not None
        assert bridge._relay_info is not None
        old_wt_url = bridge._relay_info.wt_url
        bridge._relay._process.kill()  # SIGKILL: no CONNECTION_CLOSE reaches the bridge
    
        # The child watchdog notices, the supervisor respawns the relay (new QUIC
        # port + cert), rediscovers it through /api/info and reconnects.
        def rediscovered() -> bool:
            info = bridge._relay_info
            return info is not None and info.wt_url != old_wt_url
    
        deadline = time.monotonic() + 30
        while time.monotonic() < deadline:
            if rediscovered() and bridge._relay.poll() is None and _session_live(bridge):
                break
            time.sleep(0.1)
        assert rediscovered(), "relay child was never respawned"
    
        async def flow() -> None:
            # A fresh viewer attaches to the new relay and frames flow again.
            async with await _viewer(bridge) as viewer:
                await viewer.hello()
                await attach_viewer(viewer, ROBOT_ID, ["odom"])
                frames = await collect_until(
                    viewer, lambda fs: any(f.header.ch == "odom" for f in fs), timeout=15.0
                )
                assert any(f.header.ch == "odom" for f in frames)
    
>       asyncio.run(flow())

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3dea87680>
deadline   = 26706.562364791
flow       = <function test_relay_child_death_respawns_and_recovers.<locals>.flow at 0x3de8cad40>
old_wt_url = 'https://127.0.0.1:60421'
publisher  = <dimos.web.relay_bridge.test_relay_bridge_e2e._Publisher object at 0x3def6f110>
rediscovered = <function test_relay_child_death_respawns_and_recovers.<locals>.rediscovered at 0x3dea8a480>
respawn_bridge = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3dea87680>

.../web/relay_bridge/test_relay_bridge_e2e.py:414: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:195: in run
    return runner.run(main)
        debug      = None
        loop_factory = None
        main       = <coroutine object test_relay_child_death_respawns_and_recovers.<locals>.flow at 0x3deac2ce0>
        runner     = <asyncio.runners.Runner object at 0x3deb02990>
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/runners.py:118: in run
    return self._loop.run_until_complete(task)
        context    = <_contextvars.Context object at 0x3ded31cc0>
        coro       = <coroutine object test_relay_child_death_respawns_and_recovers.<locals>.flow at 0x3deac2ce0>
        self       = <asyncio.runners.Runner object at 0x3deb02990>
        sigint_handler = functools.partial(<bound method Runner._on_sigint of <asyncio.runners.Runner object at 0x3deb02990>>, main_task=<Task ...ny(<generator object test_relay_child_death_respawns_and_recovers.<locals>.flow.<locals>.<genexpr> at 0x3de810110>)')>)
        task       = <Task finished name='Task-175' coro=<test_relay_child_death_respawns_and_recovers.<locals>.flow() done, defined at /Us...any(<generator object test_relay_child_death_respawns_and_recovers.<locals>.flow.<locals>.<genexpr> at 0x3de810110>)')>
../../../../..../uv/python/cpython-3.12.16-macos-aarch64-none/lib/python3.12/asyncio/base_events.py:691: in run_until_complete
    return future.result()
        future     = <Task finished name='Task-175' coro=<test_relay_child_death_respawns_and_recovers.<locals>.flow() done, defined at /Us...any(<generator object test_relay_child_death_respawns_and_recovers.<locals>.flow.<locals>.<genexpr> at 0x3de810110>)')>
        new_task   = False
        self       = <_UnixSelectorEventLoop running=False closed=True debug=False>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    async def flow() -> None:
        # A fresh viewer attaches to the new relay and frames flow again.
        async with await _viewer(bridge) as viewer:
            await viewer.hello()
            await attach_viewer(viewer, ROBOT_ID, ["odom"])
            frames = await collect_until(
                viewer, lambda fs: any(f.header.ch == "odom" for f in fs), timeout=15.0
            )
>           assert any(f.header.ch == "odom" for f in frames)
E           assert False
E            +  where False = any(<generator object test_relay_child_death_respawns_and_recovers.<locals>.flow.<locals>.<genexpr> at 0x3de810110>)

bridge     = <dimos.web.relay_bridge.relay_bridge_module.RelayBridgeModule object at 0x3dea87680>
frames     = []
viewer     = <dimos.web.relay_bridge.wt_client.RelayClient object at 0x3de80fb60>

.../web/relay_bridge/test_relay_bridge_e2e.py:412: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

fix(cli): restore the standalone rerun-bridge command

ac9ce10

paul-nechifor requested review from Dreamsorcerer, TomCC7, aclauer, jeff-hykin, leshy, mustafab0 and spomichter as code owners November 11, 2026 11:29

github-actions Bot reviewed Nov 11, 2026

View reviewed changes

Navigation Menu

Repository navigation

fix(cli): restore the standalone rerun-bridge command - #5191

fix(cli): restore the standalone rerun-bridge command#5191
paul-nechifor wants to merge 1 commit into
mainfrom
paul/fix/rerun-bridge-entrypoint

paul-nechifor commented Nov 11, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Nov 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment

Uh oh!

codecov Bot commented Nov 11, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

paul-nechifor commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why do we need this?

What is this feature?

Uh oh!

greptile-apps Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

go2 replay realtime (arm64)

Uh oh!

codecov Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 26 Tests Failed:

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

paul-nechifor commented Nov 11, 2026 •

edited

Loading

greptile-apps Bot commented Nov 11, 2026 •

edited

Loading

codecov Bot commented Nov 11, 2026 •

edited

Loading