Skip to content

fix(limrun): recover when the Limrun instance ends under a held lease #4229

Closed

Description

@thymikee

Purpose

When Limrun ends an instance that a session still holds, every later command fails with WebSocket is not connected until the lease expires or the user runs disconnect. The session never recovers on its own.

Evidence (live, Limrun iOS, found while validating #4220)

  • Timing: the last command was at 05:25:08Z. Limrun's API reports the instance terminated at 05:28:08.9Z, with terminationReason: InactivityTimeout and spec.inactivityTimeout: "3m0s". agent-device did not set a timeout then, so the API default of 3 minutes applied. fix(cli): one five-minute idle TTL for remote device leases #4220 now requests 5m on create.
  • Daemon log: WebSocket error: Unexpected server response: 404.
  • What fails: the daemon lease was still held. The next snapshot, open and snapshot all failed with WebSocket is not connected. Nothing reconnects or reallocates.

The same state follows any instance end that agent-device did not request: the inactivity timeout, a Limrun-side stop, or quota.

Required behavior

  • A Limrun instance that ends while a session holds its lease is detected: a 404 on the instance WebSocket, the socket closing with no reopen, or the instance API status terminated.
  • The provider then ends the lease with a typed reason, as the lease-expiry path does (LEASE_EXPIRED / endSessionsForLease).
  • The next command gets SESSION_NOT_FOUND with the existing hint to open again, or the next lease-using command allocates a new instance, matching what happens after lease expiry.
  • The user-facing error names the cause, for example "the Limrun instance ended (InactivityTimeout)", and does not report WebSocket is not connected.
  • Do not match error text: carry the termination reason as a typed detail.

Completion conditions

  • A provider test fakes an instance WebSocket that answers 404, or an instance status of terminated. It asserts that the lease ends with a typed reason, and that the next command takes the post-expiry path.
  • Live: with keep-alive off, a gap longer than the instance inactivity timeout is followed by a command. That command reports the instance end, and open then starts a new instance.

Dependencies

Related: #4220, which sets the 5-minute lease and the instance inactivity timeout.

Activity

  1. apex-sentinel-staging commented on Nov 5, 2026

    @apex-sentinel-staging

    Apex Sentinel · Issue review

    Suggested label: bug

    The issue reports existing broken behavior where a Limrun instance that ends under a held lease causes subsequent commands to fail with WebSocket not connected until lease expiry. It provides concrete evidence, required recovery behavior, and test/completion conditions a maintainer can use.

    Required sections

    • Steps to reproduce (missing)
    • Expected and actual behavior (missing)
    • Version or environment (missing)
  2. added
    bugSomething isn't working
    and removed on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions