Repository navigation
Persist stop lifecycles with shared SQLite and PostgreSQL migrations - #26
Draft
Richard-LZ-Zhang wants to merge 45 commits into
Draft
Richard-LZ-Zhang wants to merge 45 commits into
Richard-LZ-Zhang wants to merge 45 commits into
Conversation
…, hosted, bucket books)
synpath serve now carries every data feature the terminal needs, so the
terminal's web backend can be a router with no venue connections or keys.
- Auth: every market-data route and stream requires the serve token.
- /ws/market: live books, trades and quotes for all venues, one shared
reference-counted venue connection per market, shared with the
engine's feeds; reconnect with a fresh snapshot.
- Batch reads and search: /markets?ids=, /books?ids=, /search/markets,
/search/events, with per-id errors.
- Hosted routes: /matching/market|event|arbitrage and /history/* through
serve's Synpath key, with mapped errors.
- Buckets: client ids, PATCH, a merged bucket book on /ws/market built
with router.view/merge, and POST /trading/buckets/{id}/plan.
- BucketMember.enabled: an Off member is not streamed, merged or routed.
Alembic revision 0003 adds bucket_members.enabled (default on); schema 8.
- tests/serve_harness.py: in-process serve with mocked venues, SQLite and
PostgreSQL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On CI's slower runners the bucket tests reached for the fake Polymarket stream before the hub had opened it (KeyError: 'polymarket'). The harness now waits for it, polling the dict directly so the helper also works inside conditions that already run on the server's loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l, stop trigger, one TIF) - S1: GET /trading/orders?scope=open|all with market_id, venue, book, since, limit, cursor; venue orders plus engine-held parents, newest first. journal.orders_page(); Alembic 0004 (indexes, schema 9); Client.fetch_orders(). - S2: a send with no venue answer raises SubmissionInDoubt; serve returns 202 submission_in_doubt / submission_pending (retryable, with client_order_id); remote.py maps them. Retrying the same id never sends twice. - S3: cancelling a filled order keeps it closed: on the venue's not-found the order is read back; CancelUnconfirmed (409) when the venue has no record. - Stops evaluate a book they are handed: Feeds.sync() and a new stop check an already-loaded book at once (a quiet market never triggered). - One time in force per stop order, like IBKR: stop_market and stop_limit take GTC, DAY or GTD; IOC/FOK refused; the child inherits the parent's TIF and expiry. Stored IOC/FOK stop limits still load. - Market order tests on SQLite and PostgreSQL; lifecycle docs updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 150 ms lease renewed every 20 ms could lapse on a slow CI runner (Python 3.10, pure Python), so the stop's dispatch was refused for an unrelated reason. 600 ms / 50 ms with a 0.9 s price wait keeps the test's point: a wait longer than the lease must keep it renewed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checkpoint of the order-gaps work the terminal runs on. - Orders, fills and requests carry `outcome` (yes/no) and `action` (buy/sell) in place of `reduce_only`, which is removed everywhere, adapters included. - Fair values are removed from the engine (migration 0005 drops the table). - Positions are marked to the live book: the exit price (best bid for a long, best ask for a short), falling back to the last trade. - Stops trigger on the last trade by default. - An order log: every action on an order, newest first, served by serve. - Docs and lifecycle notes updated to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps #30's venue_streams / market_stream / MarketStreams structure and fits main into it: - venue_streams accepts a MarketDataKey (Opinion, predict.fun, SX Bet) and builds the market stream alone from it, with a keyed catalog where no adapter supplies one; market_stream and default_streams(market_data) go through it. - Betfair and Smarkets market streams (Smarkets without credentials). - SX Bet's account stream only with an account address, warning otherwise. - serve always loads market_data, passes it to default_streams and to the /ws/market factory, and gives the read app's default VenueRegistry its dotenv keys; #30's auth dependency, hosted routes and buckets stay. - The dotenv read-app test sends serve's access token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges cleanly. Main's trading changes (SX Bet's key-only trading, Betfair and Smarkets instruments, init/doctor) build no OrderRequest, Order or Fill and bring back no reduce_only or fair values, so nothing needed porting to outcome + action. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sync main into the experimental line (#34)
serve: library parity for data and buckets (into experimental branch)
engine: order gaps for the terminal (list, in-doubt, cancel-after-fill, stop trigger, one TIF)
Request-for-quote and combo trading for the terminal, on the experimental
order model (outcome + action). Plan and venue research: docs/rfq-combo.md.
- Models: Rfq, RfqQuote, Combo/ComboLeg, AcceptQuote, MakerQuote, status
enums, sizing by contracts or by amount (synpath/trading/money.py), and
capability keys rfq_taker, rfq_quoter and combos on every adapter.
- One typed interface on TradingExchange, implemented by:
- Polymarket US: taker, quoter, combos.
- Kalshi: taker, quoter, multivariate combos; accepted_side names the
maker's bid, so buying YES sends "no" (checked on the demo exchange).
- Polymarket: taker, quoter, combos, with Exchange v3 order signing
(matched against independently built EIP-712 vectors, incl. the
Deposit Wallet ERC-7739 wrap).
- SX Bet: the parlay quoter side.
- Engine (engine.rfq): accepts and confirms go through risk and are
journaled before the venue call; crash recovery reads the quote back;
fills land in positions; a combo is its own instrument; a side mismatch
alerts; halt cancels open RFQs and quotes. Incoming RFQs are pushed to
quoting callbacks where the venue streams them.
- serve: /trading/rfqs, /trading/quotes, /trading/combos and quote events
on /ws/events. Accept and confirm return 202 at once, 200 when waited.
Offline tests only; live runs are phase P10.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RFQ and combos: Polymarket US, Kalshi, Polymarket, SX Bet (experimental line)
Rename risk and child-order helpers, decompose routed replanning, and document current lifecycle behavior and proposed cancellation work. Verified 289 offline tests passed; 23 skipped.
On the experimental line, for the terminal's Combos page.
- Cross-venue RFQ (engine.rfq_groups):
- One request fans out to every venue that lists the same bucket,
market or combo (legs mapped through hosted matching). A venue
missing a leg is skipped with the reason.
- Quotes are ranked by price after each venue's fee.
- Accepting one claims the group in the journal, cancels the other
venues' RFQs first and then accepts. A quote is named by venue and
id, so the same id on two venues is unambiguous.
- Quote windows use each venue's own expiry; rfq_group.* events.
- Preview: POST /trading/rfq-groups/preview prices the same combo on
every venue without creating anything.
- Combo board: GET /trading/combos/board groups eligible markets into
games with moneyline/spread/total, from the venues' own fields
(Polymarket gamma sportsMarketType, line and teams; Kalshi event and
market sub titles; Polymarket US marketType and team abbreviations),
checked against real samples.
- Combos: POST /trading/combos takes collection_id (Kalshi);
GET /trading/combos/markets lists Polymarket and Polymarket US
eligible markets; collections carry associated_event_tickers.
- Fix: Kalshi size_max 0 means no maximum, not zero legs.
Offline tests only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cross-venue RFQ, combo board and preview, combo routes (experimental line)
…cycle Fix routed cancellation lifecycle and clarify order execution code
serve now builds every combo venue's board in the background at start-up and again every refresh_interval_s ([combo_board] warm, refresh_interval_s, pause_s), keeping the last good board, so the board route never waits on a build. The board is served in parts: GET /trading/combos/board/categories for the rail, and category / cursor / limit / q / sort pages on the board route. Polymarket's catalogue is about 214,000 markets; Gamma reads are cached between builds, and a first build publishes partial boards. The build stays off the event loop that answers health checks and orders: its CPU steps run in worker threads that hand back the GIL every 100 items, the GIL switch interval drops to 0.5 ms while boards are warmed, and the board's objects are frozen out of the cyclic collector as they are built. Requests stay under 0.8 s during a whole-catalogue build (before: 1-6 s). An RFQ group now matches every leg before skipping a venue, and each skip lists missing_legs (market_id, label, code, mapped_id, detail), with codes missing_leg, matching_error, ineligible_leg and outside_collections; labels come from the warm board, else a shortened catalogue question. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Warm combo boards in serve and name the legs a venue lacks
fix: make routed smart-order parents venue-neutral
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stop instructions previously depended on managed state without an explicit, shared storage contract across SQLite and PostgreSQL. This adds durable user instructions, atomic trigger/child dispatch planning, restart recovery and a consistent execution owner, while keeping venue acceptance and fills authoritative at the venue.
Behavior and storage
market_orders,limit_orders,stop_market_ordersandstop_limit_ordersretain original requests and lifecycle snapshots. Pending child intent commits with the irreversible trigger before venue dispatch.usersregistry and instruction/submission foreign keys. Conflicting instruction ownership fails; passive historical observations retain their original attribution.Experimental dependency
Keep this PR draft and unmerged. Terminal development depends on
codex/order-lifecycle-storageat verified commit4f296193c6a9b7d9938e433b02a78d38e213aa84, recorded in the terminal repository. No SDK release, tag, publication, or merge into public main is requested. The branch incorporates current public main (including its existing version 0.8.1).Validation
Offline validation covers mocked Kalshi, Polymarket and Polymarket US adapters; fresh and upgraded databases; single-owner constraints; cancellation/expiry races; process crashes; retries; and uncertain execution recovery. The full offline SDK suite passed: 2,403 tests, 3 skipped, 137 live/demo tests deselected. After the final cancellation fix, all 124 migration, stop-recovery and PostgreSQL checks passed. Packaging verification confirmed Alembic revisions ship in the sdist.
Independent review identified and fixed cancellation during COMMIT and repeated cancellation during BEGIN rollback cleanup. Added regressions prove triggered stops resume after restart and concurrent acknowledged writes survive cleanup.
Earlier bounded, explicitly authorized PostgreSQL checks completed a one-contract Kalshi stop-limit threshold crossing and matched exit, and a 0.01-contract Polymarket US already-eligible stop-market fill and native close. Both finished flat with no test orders; sanitized costs were $0.055 and $0.0001 respectively. No additional live orders were used to verify the SQLAlchemy/Alembic migration changes.
Limits
One owner and one execution worker only; PostgreSQL does not enable multiple workers. Stops cannot observe prices during downtime. International Polymarket live verification remains blocked by host eligibility. Polymarket/US response-loss adoption requires durable native identities; US ordinary orders lack general reduce-only support. The US eligible-trigger test does not prove a future threshold crossing. Migrating armed instructions between databases is a separate operation; downgrade requires a verified backup.