Skip to content

Connect to Restate Cloud without a port: restate.connect() and python -m restate connect - #258

Merged
slinkydeveloper merged 14 commits into
mainfrom
cloud-exp-revamp
Nov 4, 2026
Merged

slinkydeveloper merged 14 commits into
mainfrom
cloud-exp-revamp

Conversation

@slinkydeveloper

Copy link
Copy Markdown
Contributor

Depends on restatedev/sdk-shared-core#108: Cargo.toml patches restate-sdk-shared-core to that branch until it is released; the patch section goes away with the version bump.

What

The application dials Restate Cloud outbound and receives its invocations over that connection, through the unchanged ASGI app. One line to run it:

RESTATE_CONNECT_KEY=rstcloudv1_... python -m restate connect app:app --name greeter

or from code:

if __name__ == "__main__":
    asyncio.run(restate.connect(app, name="greeter"))

connect() handles SIGINT/SIGTERM (first drains in-flight invocations for up to two minutes, second cuts) and runs until the connection stops. For doing something while connected:

async with restate.with_connection(app, name="greeter") as connection:
    print(connection.info.deployment_url)
    await connection.wait()

The name and key come from the arguments, RESTATE_CONNECT_NAME / RESTATE_CONNECT_KEY, or the restate-operator's RESTATE_INPROC_* variables. restate.create_client() without an URL reads RESTATE_INGRESS_KEY.

How

  • _internal (PyO3): TunnelEngineOptions, TunnelEngine (awaitable ready / terminated / shutdown(grace)), TunnelInfo, IngressKey, ConnectError, over the shared core's tunnel feature; the tokio runtime is pyo3-async-runtimes' global one.
  • src/asgi.rs: the bridge. Each forwarded request becomes one app(scope, receive, send) task on the event loop: pull-based receive (one DATA frame per call, http.disconnect after the stream closes), backpressured send (one frame in flight), the http.response.trailers extension, hypercorn-like message validation, 500 for an exception before the response starts and a stream reset after.
  • restate.server.install_shutdown_hook: a per-loop registry so the ASGI app's SIGTERM hook and connect()'s handler coexist (asyncio keeps one handler per signal; the app used to replace any earlier one silently).
  • With RESTATE_CORE_LOG unset, the tunnel's lifecycle (the registration hint, connection changes) is logged at info.

Tests

tests/connect.py drives the whole stack through a python-h2 fake Cloud peer (handshake, forwarding with signed identity, discovery, body streaming, trailers, app failures, graceful shutdown, fatal handshakes, with_connection() and connect()), with a pure-Python ed25519 signer. Verified manually against a Cloud environment: connect, discovery through the tunnel, invocations via curl and create_client(), SIGTERM drain.

No build-system changes.

🤖 Generated with Claude Code

slinkydeveloper and others added 3 commits November 3, 2026 17:19
…and `restate.connect()`

The tunnel engine of restate-sdk-shared-core (`tunnel` feature) serves the unchanged ASGI application through
an outbound connection to Restate Cloud. `_internal` exposes `TunnelEngineOptions`, `TunnelEngine` (awaitable
`ready` / `terminated` / `shutdown(grace)`), `TunnelInfo`, `IngressKey` and `ConnectError`; the ASGI bridge
(`src/asgi.rs`) maps every forwarded request to one app task on the event loop, with a pull-based `receive`,
backpressured `send`, and the `http.response.trailers` extension.

Python: `python -m restate connect app:app --name X` (SIGINT/SIGTERM drain, then cut on a second signal),
`async with restate.connect(app, name=, key=)`, `create_client()` resolving `RESTATE_INGRESS_KEY`, and a per-loop
shutdown-hook registry so the app's SIGTERM hook and the command's handler coexist. With `RESTATE_CORE_LOG` unset,
the tunnel's lifecycle (registration hint, connection changes) is logged at info.

Tests drive the whole stack through a python-h2 fake Cloud peer. The shared core is taken from its
`cloud-exp-revamp` branch until the release that carries the new features.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… `restate.connection.Connection`

`restate.connect(app, name=, key=)` owns the event loop and the signal handling and blocks until the
connection stops. `Connection(app, name=, key=)` in `restate.connection` is the same under manual control for
programs that already run a loop: `start()`, `wait()`, `shutdown(grace)`. The async context manager is gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`asyncio.run(restate.connect(app))` is the whole program: signals, drain, run until stopped.
`async with restate.with_connection(app) as connection:` is the form for doing something while connected
(`connection.info`, `wait()`, `shutdown(grace)`); leaving the block drains for up to ten seconds. The
yielded handle is not meant to be constructed; there is no public `Connection` to build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues. Three spots worth fixing inline: one error-path bug in the ASGI bridge, one signal-handler leak, one masked exception.

Reviewed changes

I reviewed the whole PR at 7e2d85f: the Python connect API, the CLI, IngressKey support in the client, the shutdown-hook registry, the PyO3 tunnel bindings, the Rust ASGI bridge and the fake-Cloud test suite.

  • restate.connect() / restate.with_connection(): serve the ASGI app over an outbound tunnel. connect() handles signals: the first SIGINT/SIGTERM drains for 120s, the second cuts at once.
  • python -m restate connect: loads the app from module:attribute and runs connect(). Name and key come from flags or RESTATE_CONNECT_*.
  • create_client() without a URL: reads RESTATE_INGRESS_KEY for the URL and bearer token.
  • install_shutdown_hook: a registry of hooks per loop and per signal, so the ASGI app's SIGTERM hook and connect()'s handlers no longer replace each other.
  • src/asgi.rs: a hyper Service → ASGI-3 bridge. Request bodies are pulled on demand, responses are sent with backpressure, and it supports trailers, message validation, 500s and stream resets.
  • src/tunnel.rs / src/lib.rs: Python bindings for TunnelEngine and its options, plus TunnelInfo, IngressKey and ConnectError. The tunnel lifecycle is logged at info level by default.
  • tests/connect.py: end-to-end tests against a fake Cloud peer built on python-h2.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread src/asgi.rs Resolved
try:
install_shutdown_hook(loop, on_signal, signum)
except (NotImplementedError, RuntimeError, ValueError):
pass # Windows or non-main thread: no signal handling, stop through shutdown()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These hooks are never removed. After connect() returns on a loop that keeps running (e.g. await restate.connect(...) inside a larger main()), SIGINT goes to the old on_signal instead of raising KeyboardInterrupt. That handler then calls shutdown on a connection that has already stopped, so Ctrl-C does nothing. Each further connect() call adds more stale hooks. install_shutdown_hook could return a remover, called in a finally here, that drops the hook and restores the default handler via loop.remove_signal_handler once no hooks remain for that signal.

info = await engine.ready()
except BaseException:
await engine.shutdown(0.0)
raise

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If ready() failed because the engine hit a fatal error (e.g. unauthorized), engine.shutdown(0.0) can raise ConnectError itself. Connection.shutdown swallows it for exactly this reason. Here, that new error replaces the original and chains onto it (During handling of the above exception…). It can also turn a CancelledError from an outer wait_for into a ConnectError.

Suggested change
raise
except BaseException:
try:
await engine.shutdown(0.0)
except ConnectError:
pass
raise

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Test Results

  8 files  ±0    8 suites  ±0   3m 23s ⏱️ -36s
 71 tests ±0   71 ✅ ±0  0 💤 ±0  0 ❌ ±0 
322 runs  ±0  322 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 2ce9a0f. ± Comparison against base commit 5e6cfed.

♻️ This comment has been updated with latest results.

slinkydeveloper and others added 4 commits November 4, 2026 12:37
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…esponse, reset on failure with a buffered frame

Findings of a review of the bridge against the SDK's use of receive/send and hypercorn's semantics:

- A stream cancelled (reset, connection lost, force shutdown) after the request body ended but before the
  response head never turned `receive()` into `http.disconnect`: the app blocked forever and its task leaked.
  The request future now cancels the stream token when dropped before the head exists.
- An app failing before completing the response, while the peer's flow-control window was closed, could end
  the stream cleanly (the abort was pushed into the full frame channel and lost). The abort is now recorded
  beside the channel and the body fails after its buffered frame: a reset, never a truncated "complete" response.
- `send()` validates the message against the protocol state before the disconnect short-circuit, so a message
  after the final one raises as under hypercorn instead of being silently dropped; the test now asserts it.
- `receive()` results are built where the Python future is resolved, and the scope/coroutine setup runs on a
  blocking thread, so hyper's worker threads no longer wait for the GIL.
- Bridge-level errors are visible with the default log filter; a cancelled app task (loop teardown) is logged
  at debug, not as an application error.
- Trailers spread over several `http.response.trailers` messages append instead of replacing same-named values.
- The ASGI app's shutdown hook is registered for SIGINT too (Ctrl-C of `python -m restate connect` unblocks
  invocations waiting for Restate to close their input), once per event loop instead of once per app object.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The bridge fixes hold up, and the buffered-frame reset from the last review is fixed. One must-fix: the new SIGINT hook takes Ctrl-C away from in-process ASGI servers. Also one small regression in trailer handling.

Reviewed changes

I reviewed 4f4a6eb, the only commit with source changes since the review at 7e2d85f. The three commits in between only moved the Cargo.lock pin.

  • Disconnect before the response head: a drop guard on the request future cancels closed, so a stream reset before the head still makes receive() return http.disconnect.
  • Reset with a buffered frame: app_finished records the abort reason next to the channel. ClosingBody turns the final None into an error, so the stream resets after the queued frames go out.
  • Protocol validation after disconnect: send() now runs step() before the closed short-circuit, so a message sent after the response is complete raises, as it does under hypercorn.
  • GIL off hyper workers: the scope and coroutine are now built in spawn_blocking, and receive() events are turned into dicts where the Python future is resolved.
  • Logging: bridge errors now show up under the default filter, and a cancelled app task is logged at debug.
  • Trailers across messages: values from several http.response.trailers messages are now appended instead of overwritten.
  • Shutdown hook: the ASGI app registers its hook once per event loop, for both SIGTERM and SIGINT.
  • Tests: covers a reset before the head and a failure while the peer stalls, and the validation test now checks the errors it expects.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread python/restate/server.py Resolved
Comment thread src/asgi.rs Resolved
slinkydeveloper and others added 3 commits November 4, 2026 14:20
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Registering a SIGINT handler from inside the app replaced the Ctrl-C handler of hypercorn's serve() or
uvicorn from the first request on. The app now joins the dispatcher `restate.connect()` installs, and
leaves SIGINT alone under any other host.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ASGI app keeps its SIGTERM-only hook; the registry exists solely so that registration no longer
replaces the handler `restate.connect()` installs. The SIGINT additions of the last two commits are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ The SIGINT hijack is fixed. One small ordering gap remains, described inline.

Reviewed changes

I reviewed 95e6f5e, the only commit with source changes since the review at 4f4a6eb. 09315b2 only moved the Cargo.lock pin.

  • SIGINT join without takeover: install_shutdown_hook takes an only_if_present flag and returns whether the hook was registered. The ASGI app uses the flag for SIGINT, so it joins a dispatcher that restate.connect() installed but never replaces the Ctrl-C handler of hypercorn serve() or uvicorn.
  • Test: test_app_joins_but_never_installs_the_sigint_handler checks that only SIGTERM is installed when there is no host dispatcher, and that the app joins one when there is.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread python/restate/server.py Resolved
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in these commits. Two earlier threads on connection.py are still open: the masked shutdown error in with_connection() and the signal hooks that connect() never removes.

Reviewed changes

I reviewed ee8266d and 04e89d8, the commits since the review at 95e6f5e.

  • Reduced the shutdown-hook change: the ASGI app's hook is SIGTERM-only again, as on main. install_shutdown_hook lost only_if_present and its return value. server.py now only adds the per-loop registry, so connect() and the app share one SIGTERM dispatcher in either registration order. The SIGINT test in tests/server.py was removed with it.
  • Kept repeated trailer values: src/asgi.rs iterates the trailer HeaderMap by reference, so every value of a repeated name gets appended. The trailers test now sends x-t across two messages and checks trailer_pairs in order.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in this delta. Both threads it addressed are resolved. The two earlier connection.py threads (the masked exception in with_connection() and the connect() signal hooks that are never removed) are still open, since that file has not changed.

Reviewed changes

I reviewed ee8266d and 04e89d8, the commits since the review at 95e6f5e.

  • Smaller shutdown hook change: install_shutdown_hook no longer takes only_if_present and returns None again. The ASGI app registers only its SIGTERM hook, once per app as on main, and never touches SIGINT. That removes the join race flagged last time. The SIGINT test in tests/server.py is gone with it.
  • Repeated trailer names: http.response.trailers headers are now iterated by reference, so every value of a repeated name reaches pending_trailers. The trailers test sends x-t across two messages and checks all the pairs in order.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

slinkydeveloper and others added 3 commits November 4, 2026 14:36
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…core

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@slinkydeveloper
slinkydeveloper merged commit 670c823 into main Nov 4, 2026
11 checks passed
@slinkydeveloper
slinkydeveloper deleted the cloud-exp-revamp branch November 4, 2026 14:42
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 4, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant