ASGI app SIGTERM handler silently replaces the hosting server's handler #236

On the first call into restate.app(...) of any scope, asgi_app registers loop.add_signal_handler(signal.SIGTERM, _on_sigterm) (python/restate/server.py:254-267). Because loop.add_signal_handler replaces the existing handler without chaining, SIGTERM no longer reaches the hosting ASGI server's shutdown path after the first request.

Observed behavior

Measured on restate-sdk 1.0.5 (main 63d2fe5), Python 3.13:

• Hypercorn 0.18: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving restate.app(...), hitting /health once, and sending SIGTERM leaves the process running past a 15 s grace period until SIGKILL.
• Uvicorn 0.54.0: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving restate.app(...) after one request hangs the same way until SIGKILL.
• Lifespan scope: a single lifespan.startup call is enough to displace the host handler, because the signal handler is installed before the scope["type"] == "lifespan" check (python/restate/server.py:269-270).

_on_sigterm only calls notify_shutdown() on active ReceiveChannel instances. Nothing notifies the host server to stop accepting connections or exit.

Why both loop.add_signal_handler and signal.signal matter

ASGI servers register SIGTERM in two ways on Unix:

1. Uvicorn uses signal.signal(signal.SIGTERM, ...).
2. Hypercorn uses loop.add_signal_handler(signal.SIGTERM, ...) (hypercorn/asyncio/run.py). Once loop.add_signal_handler is active, Python's signal.getsignal(signal.SIGTERM) returns asyncio.unix_events._sighandler_noop, and the actual host callback lives in loop._signal_handlers[signal.SIGTERM] (an asyncio.Handle storing the callback and its arguments).

PR #226 adds re-dispatch using signal.getsignal(signal.SIGTERM) before loop.add_signal_handler. Tested on top of 1.0.5, #226 fixes Uvicorn, but Hypercorn still hangs for 15 s because signal.getsignal(signal.SIGTERM) only captures _sighandler_noop rather than the previous loop handler.

Suggested fix

Before calling loop.add_signal_handler(signal.SIGTERM, _on_sigterm):

1. Check getattr(loop, "_signal_handlers", {}).get(signal.SIGTERM) first. If present, save that asyncio.Handle and invoke prev_handle._run() from _on_sigterm after notifying the channels.
2. Otherwise, fall back to signal.getsignal(signal.SIGTERM) and invoke it as prev(signal.SIGTERM, None) when callable and not asyncio.unix_events._sighandler_noop.
3. Optionally move the registration after the scope["type"] == "lifespan" check so a rejected lifespan call does not touch process signals.

Minimal repro (no external server required)

# uv run --no-project --python 3.13 --with restate-sdk==1.0.5 python repro.py
import asyncio
import signal
import restate

calls: list[str] = []


def build_app():
    svc = restate.Service("probe")

    @svc.handler()
    async def noop(ctx: restate.Context) -> str:
        return "ok"

    return restate.app(services=[svc])


async def one_call(app):
    async def receive():
        return {"type": "lifespan.startup"}

    async def send(msg):
        pass

    try:
        await app({"type": "lifespan", "asgi": {"version": "3.0"}}, receive, send)
    except Exception:
        pass


def run_case(style: str) -> bool:
    calls.clear()
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    if style == "loop.add_signal_handler":
        loop.add_signal_handler(signal.SIGTERM, lambda: calls.append("host_ran"))
    elif style == "signal.signal":
        signal.signal(signal.SIGTERM, lambda *_: calls.append("host_ran"))

    app = build_app()
    loop.run_until_complete(one_call(app))
    signal.raise_signal(signal.SIGTERM)
    loop.run_until_complete(asyncio.sleep(0.05))
    print(f"{style:<24} -> {calls or ['displaced (nothing ran)']}")
    return "host_ran" in calls


for mode in ("loop.add_signal_handler", "signal.signal"):
    run_case(mode)

Output on restate-sdk 1.0.5:

loop.add_signal_handler  -> ['displaced (nothing ran)']
signal.signal            -> ['displaced (nothing ran)']