Repository navigation
Connect to Restate Cloud without a port: restate.connect() and python -m restate connect - #258
Conversation
…and `restate.connect()` The tunnel engine of restate-sdk-shared-core (`tunnel` feature) serves the unchanged ASGI application through an outbound connection to Restate Cloud. `_internal` exposes `TunnelEngineOptions`, `TunnelEngine` (awaitable `ready` / `terminated` / `shutdown(grace)`), `TunnelInfo`, `IngressKey` and `ConnectError`; the ASGI bridge (`src/asgi.rs`) maps every forwarded request to one app task on the event loop, with a pull-based `receive`, backpressured `send`, and the `http.response.trailers` extension. Python: `python -m restate connect app:app --name X` (SIGINT/SIGTERM drain, then cut on a second signal), `async with restate.connect(app, name=, key=)`, `create_client()` resolving `RESTATE_INGRESS_KEY`, and a per-loop shutdown-hook registry so the app's SIGTERM hook and the command's handler coexist. With `RESTATE_CORE_LOG` unset, the tunnel's lifecycle (registration hint, connection changes) is logged at info. Tests drive the whole stack through a python-h2 fake Cloud peer. The shared core is taken from its `cloud-exp-revamp` branch until the release that carries the new features. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… `restate.connection.Connection` `restate.connect(app, name=, key=)` owns the event loop and the signal handling and blocks until the connection stops. `Connection(app, name=, key=)` in `restate.connection` is the same under manual control for programs that already run a loop: `start()`, `wait()`, `shutdown(grace)`. The async context manager is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`asyncio.run(restate.connect(app))` is the whole program: signals, drain, run until stopped. `async with restate.with_connection(app) as connection:` is the form for doing something while connected (`connection.info`, `wait()`, `shutdown(grace)`); leaving the block drains for up to ten seconds. The yielded handle is not meant to be constructed; there is no public `Connection` to build. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ No blocking issues. Three spots worth fixing inline: one error-path bug in the ASGI bridge, one signal-handler leak, one masked exception.
Reviewed changes
I reviewed the whole PR at 7e2d85f: the Python connect API, the CLI, IngressKey support in the client, the shutdown-hook registry, the PyO3 tunnel bindings, the Rust ASGI bridge and the fake-Cloud test suite.
restate.connect()/restate.with_connection(): serve the ASGI app over an outbound tunnel.connect()handles signals: the first SIGINT/SIGTERM drains for 120s, the second cuts at once.python -m restate connect: loads the app frommodule:attributeand runsconnect(). Name and key come from flags orRESTATE_CONNECT_*.create_client()without a URL: readsRESTATE_INGRESS_KEYfor the URL and bearer token.install_shutdown_hook: a registry of hooks per loop and per signal, so the ASGI app's SIGTERM hook andconnect()'s handlers no longer replace each other.src/asgi.rs: a hyperService→ ASGI-3 bridge. Request bodies are pulled on demand, responses are sent with backpressure, and it supports trailers, message validation, 500s and stream resets.src/tunnel.rs/src/lib.rs: Python bindings forTunnelEngineand its options, plusTunnelInfo,IngressKeyandConnectError. The tunnel lifecycle is logged at info level by default.tests/connect.py: end-to-end tests against a fake Cloud peer built on python-h2.
claude-opus-5-5 | 𝕏
| try: | ||
| install_shutdown_hook(loop, on_signal, signum) | ||
| except (NotImplementedError, RuntimeError, ValueError): | ||
| pass # Windows or non-main thread: no signal handling, stop through shutdown() |
There was a problem hiding this comment.
These hooks are never removed. After connect() returns on a loop that keeps running (e.g. await restate.connect(...) inside a larger main()), SIGINT goes to the old on_signal instead of raising KeyboardInterrupt. That handler then calls shutdown on a connection that has already stopped, so Ctrl-C does nothing. Each further connect() call adds more stale hooks. install_shutdown_hook could return a remover, called in a finally here, that drops the hook and restores the default handler via loop.remove_signal_handler once no hooks remain for that signal.
| info = await engine.ready() | ||
| except BaseException: | ||
| await engine.shutdown(0.0) | ||
| raise |
There was a problem hiding this comment.
If ready() failed because the engine hit a fatal error (e.g. unauthorized), engine.shutdown(0.0) can raise ConnectError itself. Connection.shutdown swallows it for exactly this reason. Here, that new error replaces the original and chains onto it (During handling of the above exception…). It can also turn a CancelledError from an outer wait_for into a ConnectError.
| raise | |
| except BaseException: | |
| try: | |
| await engine.shutdown(0.0) | |
| except ConnectError: | |
| pass | |
| raise |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…esponse, reset on failure with a buffered frame Findings of a review of the bridge against the SDK's use of receive/send and hypercorn's semantics: - A stream cancelled (reset, connection lost, force shutdown) after the request body ended but before the response head never turned `receive()` into `http.disconnect`: the app blocked forever and its task leaked. The request future now cancels the stream token when dropped before the head exists. - An app failing before completing the response, while the peer's flow-control window was closed, could end the stream cleanly (the abort was pushed into the full frame channel and lost). The abort is now recorded beside the channel and the body fails after its buffered frame: a reset, never a truncated "complete" response. - `send()` validates the message against the protocol state before the disconnect short-circuit, so a message after the final one raises as under hypercorn instead of being silently dropped; the test now asserts it. - `receive()` results are built where the Python future is resolved, and the scope/coroutine setup runs on a blocking thread, so hyper's worker threads no longer wait for the GIL. - Bridge-level errors are visible with the default log filter; a cancelled app task (loop teardown) is logged at debug, not as an application error. - Trailers spread over several `http.response.trailers` messages append instead of replacing same-named values. - The ASGI app's shutdown hook is registered for SIGINT too (Ctrl-C of `python -m restate connect` unblocks invocations waiting for Restate to close their input), once per event loop instead of once per app object. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Important
The bridge fixes hold up, and the buffered-frame reset from the last review is fixed. One must-fix: the new SIGINT hook takes Ctrl-C away from in-process ASGI servers. Also one small regression in trailer handling.
Reviewed changes
I reviewed 4f4a6eb, the only commit with source changes since the review at 7e2d85f. The three commits in between only moved the Cargo.lock pin.
- Disconnect before the response head: a drop guard on the request future cancels
closed, so a stream reset before the head still makesreceive()returnhttp.disconnect. - Reset with a buffered frame:
app_finishedrecords the abort reason next to the channel.ClosingBodyturns the finalNoneinto an error, so the stream resets after the queued frames go out. - Protocol validation after disconnect:
send()now runsstep()before theclosedshort-circuit, so a message sent after the response is complete raises, as it does under hypercorn. - GIL off hyper workers: the scope and coroutine are now built in
spawn_blocking, andreceive()events are turned into dicts where the Python future is resolved. - Logging: bridge errors now show up under the default filter, and a cancelled app task is logged at debug.
- Trailers across messages: values from several
http.response.trailersmessages are now appended instead of overwritten. - Shutdown hook: the ASGI app registers its hook once per event loop, for both SIGTERM and SIGINT.
- Tests: covers a reset before the head and a failure while the peer stalls, and the validation test now checks the errors it expects.
claude-opus-5-5 | 𝕏
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Registering a SIGINT handler from inside the app replaced the Ctrl-C handler of hypercorn's serve() or uvicorn from the first request on. The app now joins the dispatcher `restate.connect()` installs, and leaves SIGINT alone under any other host. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ASGI app keeps its SIGTERM-only hook; the registry exists solely so that registration no longer replaces the handler `restate.connect()` installs. The SIGINT additions of the last two commits are gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ The SIGINT hijack is fixed. One small ordering gap remains, described inline.
Reviewed changes
I reviewed 95e6f5e, the only commit with source changes since the review at 4f4a6eb. 09315b2 only moved the Cargo.lock pin.
- SIGINT join without takeover:
install_shutdown_hooktakes anonly_if_presentflag and returns whether the hook was registered. The ASGI app uses the flag for SIGINT, so it joins a dispatcher thatrestate.connect()installed but never replaces the Ctrl-C handler of hypercornserve()or uvicorn. - Test:
test_app_joins_but_never_installs_the_sigint_handlerchecks that only SIGTERM is installed when there is no host dispatcher, and that the app joins one when there is.
claude-opus-5-5 | 𝕏
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ No new issues in these commits. Two earlier threads on
connection.pyare still open: the maskedshutdownerror inwith_connection()and the signal hooks thatconnect()never removes.
Reviewed changes
I reviewed ee8266d and 04e89d8, the commits since the review at 95e6f5e.
- Reduced the shutdown-hook change: the ASGI app's hook is SIGTERM-only again, as on
main.install_shutdown_hooklostonly_if_presentand its return value.server.pynow only adds the per-loop registry, soconnect()and the app share one SIGTERM dispatcher in either registration order. The SIGINT test intests/server.pywas removed with it. - Kept repeated trailer values:
src/asgi.rsiterates the trailerHeaderMapby reference, so every value of a repeated name gets appended. The trailers test now sendsx-tacross two messages and checkstrailer_pairsin order.
claude-opus-5-5 | 𝕏
There was a problem hiding this comment.
ℹ️ No new issues in this delta. Both threads it addressed are resolved. The two earlier
connection.pythreads (the masked exception inwith_connection()and theconnect()signal hooks that are never removed) are still open, since that file has not changed.
Reviewed changes
I reviewed ee8266d and 04e89d8, the commits since the review at 95e6f5e.
- Smaller shutdown hook change:
install_shutdown_hookno longer takesonly_if_presentand returnsNoneagain. The ASGI app registers only its SIGTERM hook, once per app as onmain, and never touches SIGINT. That removes the join race flagged last time. The SIGINT test intests/server.pyis gone with it. - Repeated trailer names:
http.response.trailersheaders are now iterated by reference, so every value of a repeated name reachespending_trailers. The trailers test sendsx-tacross two messages and checks all the pairs in order.
claude-opus-5-5 | 𝕏
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…core Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Depends on restatedev/sdk-shared-core#108:
Cargo.tomlpatchesrestate-sdk-shared-coreto that branch until it is released; the patch section goes away with the version bump.What
The application dials Restate Cloud outbound and receives its invocations over that connection, through the unchanged ASGI app. One line to run it:
or from code:
connect()handles SIGINT/SIGTERM (first drains in-flight invocations for up to two minutes, second cuts) and runs until the connection stops. For doing something while connected:The name and key come from the arguments,
RESTATE_CONNECT_NAME/RESTATE_CONNECT_KEY, or the restate-operator'sRESTATE_INPROC_*variables.restate.create_client()without an URL readsRESTATE_INGRESS_KEY.How
_internal(PyO3):TunnelEngineOptions,TunnelEngine(awaitableready/terminated/shutdown(grace)),TunnelInfo,IngressKey,ConnectError, over the shared core'stunnelfeature; the tokio runtime is pyo3-async-runtimes' global one.src/asgi.rs: the bridge. Each forwarded request becomes oneapp(scope, receive, send)task on the event loop: pull-basedreceive(one DATA frame per call,http.disconnectafter the stream closes), backpressuredsend(one frame in flight), thehttp.response.trailersextension, hypercorn-like message validation, 500 for an exception before the response starts and a stream reset after.restate.server.install_shutdown_hook: a per-loop registry so the ASGI app's SIGTERM hook andconnect()'s handler coexist (asyncio keeps one handler per signal; the app used to replace any earlier one silently).RESTATE_CORE_LOGunset, the tunnel's lifecycle (the registration hint, connection changes) is logged at info.Tests
tests/connect.pydrives the whole stack through a python-h2 fake Cloud peer (handshake, forwarding with signed identity, discovery, body streaming, trailers, app failures, graceful shutdown, fatal handshakes,with_connection()andconnect()), with a pure-Python ed25519 signer. Verified manually against a Cloud environment: connect, discovery through the tunnel, invocations via curl andcreate_client(), SIGTERM drain.No build-system changes.
🤖 Generated with Claude Code