Speak protocol v1 (SPEC.md draft 2) through awp tunnel

.github/workflows/ci.yml

@@ -25,9 +25,9 @@ jobs:
           go-version: "1.27"
       - name: Install the package and the test tools
         run: python -m pip install -e ".[dev]"
-      - name: Build awp, for the conformance suite
-        run: go install github.com/agentwireprotocol/awp/cmd/awp@v0.6.0
+      - name: Build awp, for the tunnel and the conformance suite
+        run: go install github.com/agentwireprotocol/awp/cmd/awp@1bd8112984bf9a323034d2fc031e019c1bbbac4a
       - name: Tests, including the conformance suite
         run: AWP_BIN="$(go env GOPATH)/bin/awp" python -m pytest -v
       - name: Tests with the pure-Python Ed25519
-        run: AWP_PURE_ED25519=1 python -m pytest -q -k "conversation"
+        run: AWP_BIN="$(go env GOPATH)/bin/awp" AWP_PURE_ED25519=1 python -m pytest -q -k "conversation or pure_python"

README.md

@@ -2,9 +2,12 @@
 
 The Python SDK for the [Agent Wire Protocol](https://agentwireprotocol.com) (AWP), the peer-to-peer messaging protocol for coding agents. A `Peer` listens and connects, sends messages in threads, and delivers what arrives as events, with everything the protocol asks for inside: resume with an outbox on disk, acks, dedup by id, blobs in chunks, grants and introductions, ping/pong, reconnection with backoff.
 
-Python 3.11 or later, asyncio, no required dependencies. Ed25519 uses the [`cryptography`](https://pypi.org/project/cryptography/) package when it is installed and a pure-Python implementation otherwise.
+Python 3.11 or later, asyncio, no required Python dependencies. Ed25519 uses the [`cryptography`](https://pypi.org/project/cryptography/) package when it is installed and a pure-Python implementation otherwise.
+
+Every connection is a WireGuard tunnel between the two peers' keys (SPEC.md draft 2). Python has no WireGuard of its own, so a Peer runs the reference implementation's `awp tunnel` as a helper process, which needs the `awp` binary:
 
 ```sh
+curl -fsSL https://agentwireprotocol.com/install.sh | sh     # awp, for the tunnel
 pip install awp            # or: pip install "awp[crypto]"
 ```
 
@@ -16,10 +19,10 @@ import awp
 
 async def main():
     async with awp.Peer("~/.mybot", name="mybot@builder") as peer:
-        address = await peer.listen("tailcat")          # or "tcp:127.0.0.1:7000", "unix:/tmp/awp.sock"
-        print("share this:", address)
+        address = await peer.listen("tailcat")          # or "udp:0.0.0.0:41641", "cloudflare", "unix:/tmp/awp.sock"
+        print("share this:", address)                    # awp1...
 
-        key = await peer.connect("tc...")               # an address shared out of band
+        key = await peer.connect("awp1...")             # an address shared out of band
         sent = peer.send(key, "Please run make test at a1b2c3.", subject="Run the suite")
 
         async for event in peer.events():
@@ -36,26 +39,26 @@ asyncio.run(main())
 
 ## What a Peer does
 
-- **`Peer(state_dir, name=...)`** loads or creates the Ed25519 identity under `state_dir`, along with the outbox, received ids, threads, grants and blobs. `None` uses a temporary directory with a fresh identity, removed on close. Use it as an async context manager, or call `await peer.start()` and `await peer.close()`.
-- **`listen(addr)`** accepts connections on `tcp:HOST:PORT`, `unix:/path`, or `tailcat`: a WireGuard tunnel through the [tailcat CLI](https://github.com/tailscale/tailcat) with an address any peer can reach, through NAT, with no account. It returns the address to share.
-- **`connect(addr)`** dials an address and returns the peer's key once the handshake is done. The Peer keeps dialing with exponential backoff, capped at a minute, whenever there are unacked messages or open threads with that peer. Sleeping sandboxes wake on connect.
+- **`Peer(state_dir, name=..., awp=None)`** loads or creates the Ed25519 identity under `state_dir`, along with the outbox, received ids, threads, grants and blobs. `None` uses a temporary directory with a fresh identity, removed on close. `awp` names the binary (default `$AWP_BIN` or `awp` on PATH). Use it as an async context manager, or call `await peer.start()` and `await peer.close()`.
+- **`listen(carrier)`** accepts connections on a carrier and returns the address to share (`awp1...`), which lists every carrier the Peer listens on: `tailcat` (reachable from anywhere, through NAT, no account), `udp:HOST:PORT`, `ws:HOST:PORT` or `ws:HOST:PORT=URL`, `cloudflare` (a WebSocket through a Cloudflare quick tunnel; needs `cloudflared`), `unix:/path`. **`address`** is the current one. **`rotate_psk()`** replaces its admission secret: strangers holding old copies are shut out, peers already met are not.
+- **`connect(address)`** dials an address and returns the peer's key once the handshake is done. An explicit `connect` also lifts a `bye`. The Peer keeps dialing with exponential backoff, capped at a minute, whenever there are unacked messages or open threads with that peer. Sleeping sandboxes wake on connect.
 - **`send(to, text, thread=None, subject=None, reply_to=None, parts=None, files=())`** queues a message. It never fails because the peer is away: the message is on disk and goes out on the next resume. Files travel as blobs in 256 KiB chunks ahead of the message. **`wait_ack(id)`** waits for the peer's ack; `Acked` events carry it too.
 - **`set_state(to, thread, state, note=None)`** sets this side's state on a thread: `working`, `waiting`, `done`, `failed`, `closed`, or any word the two agents agree on.
 - **`events()`** and **`next_event(timeout)`**: `Connected`, `Disconnected`, `Message`, `State`, `Acked`, `Blob`, `GrantReceived`, `Introduced`, `PeerError`, `Bye`, `Error`. Messages are stored and acked before they are reported.
-- **`grant(to, caps, ttl)`**, **`caps(to)`**: capabilities beyond the defaults (`exec`, `fs:read`, `fs:write`, `introduce`, `admin`, or your own strings) as signed grants. Grants issued by this Peer and by keys in `trust=` are honored, plus one level of delegation through `introduce`. A request in a message (a data part of a `vnd.awp` type) arrives in `Message.requests`, marked allowed or not; denied ones are answered with `err forbidden` already.
+- **`grant(to, caps, ttl)`**, **`caps(to)`**, **`grants()`**, **`revoke(hash)`**, **`introduce(to, peer, caps)`**: capabilities beyond the defaults (`exec`, `fs:read`, `fs:write`, `introduce`, `admin`, or your own strings) as signed grants. Grants issued by this Peer and by keys in `trust=` are honored, plus one level of delegation through `introduce`. Grants this Peer holds are presented in its hello to the peer they concern. A request in a message (a data part of a `vnd.awp` type) arrives in `Message.requests`, marked allowed or not; denied ones are answered with `err forbidden` already.
 - **`bye(to)`** closes gracefully and parks the peer. **`list_peers()`**, **`threads()`**, **`connected(to)`**: what the Peer knows.
 
 ## A peer driven over stdin and stdout
 
-`python -m awp --state DIR listen tcp:127.0.0.1:7000` runs a peer that reads JSON commands on stdin and writes JSON events on stdout, for tests and for other languages. The [reference implementation's interop tests](https://github.com/agentwireprotocol/awp/tree/main/python) drive it.
+`python -m awp --state DIR listen udp:127.0.0.1:0` (or `connect awp1...`) runs a peer that reads JSON commands on stdin and writes JSON events on stdout, for tests and for other languages. `awp conform --listen udp:127.0.0.1:0 --run 'python -m awp --state DIR connect {addr}'` checks it.
 
 ## Conformance
 
-`pytest` runs a two-peer conversation with states, a reply, a multi-chunk blob, grants and bye; queued delivery across a `kill -9`; and, when `AWP_BIN` points at an `awp` binary, the protocol's conformance suite ([`awp conform`](https://docs.agentwireprotocol.com/reference/conformance)) against a `Peer`, both with the Peer listening and with the Peer dialing. CI does all of it.
+`pytest` needs an `awp` binary (`AWP_BIN`, or on PATH). It runs a two-peer conversation with states, a reply, a multi-chunk blob, grants and bye; queued delivery across a `kill -9`; introductions with a grant presented in hello; pre-shared key rotation; a raw client for the protocol's edges (a hello naming another key, bad versions, bad frames, oversized lines); and the protocol's conformance suite ([`awp conform`](https://docs.agentwireprotocol.com/reference/conformance)) against a `Peer`, both with the Peer listening and with the Peer dialing. CI does all of it.
 
 ## Status
 
-v0.1. The API may change before v1; the wire protocol is v0 and stable.
+v0.2. The API may change before v1. It speaks protocol v1 (SPEC.md draft 2).
 
 ## License
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "awp"
-version = "0.1.0"
+version = "0.2.0"
 description = "The Python SDK for the Agent Wire Protocol: peer-to-peer messaging between coding agents."
 readme = "README.md"
 license = "Apache-2.0"

src/awp/__init__.py

@@ -6,7 +6,7 @@
 
     async with awp.Peer("~/.mybot", name="mybot@host") as peer:
         address = await peer.listen("tailcat")
-        key = await peer.connect("tc...")
+        key = await peer.connect("awp1...")
         peer.send(key, "Please run make test.", subject="Run the suite")
         async for event in peer.events():
             if isinstance(event, awp.Message):
@@ -19,7 +19,7 @@
 from .wire import (AwpError, CommandError, StartupError, PROTOCOL_VERSION, mint_grant,
                    verify_grant)
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 
 __all__ = [
     "Peer", "PeerInfo", "Sent", "Thread",

src/awp/cli.py

@@ -2,8 +2,10 @@
 
 ::
 
-    python -m awp --state DIR [--name NAME] listen tcp:127.0.0.1:7000
-    python -m awp --state DIR [--name NAME] connect tc...
+    python -m awp --state DIR [--name NAME] listen tailcat        (or udp:HOST:PORT, ws:HOST:PORT, cloudflare, unix:/path)
+    python -m awp --state DIR [--name NAME] connect awp1...
+
+The tunnel is `awp tunnel` (--awp, $AWP_BIN, or awp on PATH).
 
 One JSON command per line on stdin, one JSON event per line on stdout:
 
@@ -166,7 +168,7 @@ def reader() -> None:
 async def amain(args: argparse.Namespace) -> int:
     try:
         peer = Peer(args.state, name=args.name, about=args.about, trust=args.trust or (),
-                    ping_interval=args.ping_interval, blob_limit=args.max_blob)
+                    ping_interval=args.ping_interval, blob_limit=args.max_blob, awp=args.awp)
     except AwpError as e:
         emit({"event": "error", "detail": str(e)})
         return 1
@@ -210,11 +212,13 @@ def main(argv: list[str] | None = None) -> int:
                     help="largest blob accepted, in bytes (default 50 MiB)")
     ap.add_argument("--trust", action="append", default=[], help="trust grants issued by this key (repeatable)")
     ap.add_argument("--about", default=None, help="free text for hello.about")
+    ap.add_argument("--awp", default=None, help="the awp binary, whose `awp tunnel` carries the connections")
     ap.add_argument("-v", "--verbose", action="store_true", help="debug logging on stderr")
     sub = ap.add_subparsers(dest="mode", required=True, metavar="{listen,connect}")
-    for mode in ("listen", "connect"):
-        sp = sub.add_parser(mode, help=f"{mode} on ADDR (tcp:HOST:PORT, unix:/path, or tailcat)")
-        sp.add_argument("addr")
+    sp = sub.add_parser("listen", help="listen on a carrier: tailcat, udp:HOST:PORT, ws:HOST:PORT, cloudflare or unix:/path")
+    sp.add_argument("addr", metavar="carrier")
+    sp = sub.add_parser("connect", help="connect to an address (awp1...)")
+    sp.add_argument("addr", metavar="address")
     args = ap.parse_args(argv)
     logging.basicConfig(stream=sys.stderr, level=logging.DEBUG if args.verbose else logging.INFO,
                         format="%(asctime)s [awp] %(message)s", datefmt="%H:%M:%S")

src/awp/connection.py

@@ -1,5 +1,7 @@
-"""One connection to a peer: the handshake, the reader, the FIFO writer and
-the timers (SPEC.md sections 7 and 9)."""
+"""One connection to a peer: the hello exchange, the reader, the FIFO
+writer and the timers (SPEC.md sections 10 and 12). The stream comes from
+the tunnel helper, which has already checked that the peer's hello names
+the key on the other end of the tunnel."""
 
 from __future__ import annotations
 
@@ -9,15 +11,14 @@
 import traceback
 
 from ._log import log, debug
-from .wire import (ACKED_TYPES, AUTH_CONTEXT, BYE_TIMEOUT, CLOSING_ERR_CODES, HANDSHAKE_TIMEOUT,
-                   MAX_LINE, PRE_AUTH_FORBIDDEN, PROTOCOL_VERSION, READ_SIZE, RESUME_TIMEOUT,
-                   b64decode_any, b64url, dumps_line, parse_key)
-from .wire import ed25519_verify
+from .wire import (ACKED_TYPES, BYE_TIMEOUT, CLOSING_ERR_CODES, HANDSHAKE_TIMEOUT,
+                   MAX_LINE, PRE_HELLO_FORBIDDEN, PROTOCOL_VERSION, READ_SIZE, RESUME_TIMEOUT,
+                   dumps_line, parse_key, x25519_public)
 
 class Connection:
     """One transport connection: handshake, reader, FIFO writer, timers."""
 
-    def __init__(self, node: "Node", reader, writer, label: str):
+    def __init__(self, node: "Node", reader, writer, label: str, remote: bytes | None = None):
         self.node = node
         self.reader = reader
         self.writer = writer
@@ -29,7 +30,10 @@ def __init__(self, node: "Node", reader, writer, label: str):
         self.closing = False
         self.close_reason = None
         self.linger = False
-        self.hello = node.make_hello()
+        # remote is the peer's tunnel (X25519) key, from the helper: it
+        # picks the grants worth presenting before the peer's hello arrives.
+        self.remote = remote
+        self.hello = node.make_hello(remote)
         self.my_hello_line = dumps_line(self.hello)
         self.peer_hello_line = None
         self.peer_hello = None
@@ -358,29 +362,14 @@ def _handshake(self, line: bytes, obj: dict, t) -> None:
             if raw == node.identity.pub or line == self.my_hello_line:
                 self.fail("auth", "hello carries our own key (reflected handshake)", re=oid)
                 return
+            if self.remote is not None and x25519_public(raw) != self.remote:
+                self.fail("auth", "hello key is not the key on the other end of the tunnel", re=oid)
+                return
             self.peer_hello_line = line
             self.peer_hello = obj
             self.peer_key_raw = raw
             self.peer_key_str = obj.get("key")
             log(f"{self.label}: hello from {self.peer_key_str} name={obj.get('name')!r}")
-            sig = node.identity.sign(AUTH_CONTEXT + b"\x00" + self.my_hello_line + b"\x00" + line)
-            held = node.held.valid()
-            auth = node.envelope("auth", sig=b64url(sig), grants=held if held else None)
-            self._write(dumps_line(auth))
-            return
-        if t == "auth":
-            if self.peer_hello is None:
-                self.fail("auth", "auth received before hello", re=oid)
-                return
-            node.emit_recv(self, obj)
-            try:
-                sig = b64decode_any(obj.get("sig"))
-            except ValueError:
-                sig = b""
-            transcript = AUTH_CONTEXT + b"\x00" + self.peer_hello_line + b"\x00" + self.my_hello_line
-            if not ed25519_verify(self.peer_key_raw, sig, transcript):
-                self.fail("auth", "auth signature does not verify", re=oid)
-                return
             self._on_established(obj)
             return
         if t == "err":
@@ -396,22 +385,22 @@ def _handshake(self, line: bytes, obj: dict, t) -> None:
             node.emit_recv(self, obj)
             self.close("bye before handshake completed")
             return
-        if t in PRE_AUTH_FORBIDDEN:
-            self.fail("auth", f"{t!r} received before the handshake completed", re=oid)
+        if t in PRE_HELLO_FORBIDDEN:
+            self.fail("bad_frame", f"{t!r} received before hello", re=oid)
             return
         node.emit_recv(self, obj)
         log(f"{self.label}: ignoring unknown message type {t!r} during handshake")
 
-    def _on_established(self, auth: dict) -> None:
+    def _on_established(self, hello: dict) -> None:
         node = self.node
         self.established = True
         self.established_at = self.loop.time()
         self.ps = node.bind_peer(self.peer_key_raw, self.peer_key_str, self.peer_hello)
         node.register_connection(self)
-        grants = auth.get("grants")
+        grants = hello.get("grants")
         if isinstance(grants, list):
             for g in grants:
-                node.receive_grant(self, g, "auth")
+                node.receive_grant(self, g, "hello")
         h = self.peer_hello
         name = h.get("name") if isinstance(h.get("name"), str) else ""
         caps = h.get("caps") if isinstance(h.get("caps"), list) else []
@@ -462,8 +451,8 @@ def _dispatch(self, obj: dict, t) -> None:
             self._on_bye(obj)
         elif t == "err":
             self._on_err(obj)
-        elif t in ("hello", "auth"):
-            log(f"{self.label}: ignoring {t} after the handshake")
+        elif t == "hello":
+            log(f"{self.label}: ignoring hello after the handshake")
         else:
             debug(f"{self.label}: ignoring unknown message type {t!r}")