[lesomnus]

Fixes #19.

On top of #21. This branch's first commit is #21's; merge #21 first, and this PR's diff is then its own commit, 4c7bfb2. lesomnus/payday#173 pins 4c7bfb2, which a --merge keeps in master's history.

What was wrong

sqlpage.After narrowed a page to the rows past the cursor with the order written out:

c1 > a1 OR (c1 = a1 AND c2 > a2)

That is the right rows on every engine, and on PostgreSQL it is no range to start an index at. PostgreSQL uses the index for the rest of the query and applies the OR to every row from the first, so each page reads every row before it, and paging through a table costs the square of its size. Every generated List and Watch that pages by a cursor goes through After. lesomnus/payday#154 found the same shape in payday's own trail paging and fixed it there with a row value.

What changed

After says "after" in the shape each engine starts reading at:

• PostgreSQL and SQLite, an order that runs one way: the row value (c1, c2) > (a1, a2), or < when it descends. It is the whole bound.
• Otherwise, the OR behind an inclusive bound on the columns that lead the order one way: c1 >= a1, or (c1, c2) >= (a1, a2) where the engine takes a row value. A row value is wrong across a change of direction. The bound is implied by the OR and is where the index starts; what the read passes over are the rows tied with the cursor in those columns.
• MySQL and MariaDB get the OR, behind the same bound. Their range optimizer reads the OR as ranges of the index already, the first starting at the cursor, and the bound as the same range. A row value they do not seek on: they look up what precedes it in the index and filter (below).
• One column stays one comparison everywhere.

The bound alone in front of the OR, for every order, would also seek on PostgreSQL. payday#154 found what it costs: it reads a run of rows sharing one c1 again from its first on every page, and an order that runs one way need not pay that.

UPSTREAM.md's Paging row says what After does now.

Measured

PostgreSQL 17, a table of 1,000,000 rows in two tenants of 500,000, (tenant_id, date_created, id) and (tenant_id, alias, id) indexed, date_created three rows to an instant. The page of 500 after the middle of one tenant's rows, as After writes it (EXPLAIN (ANALYZE, BUFFERS) of a prepared statement, first run and second):

order		time	rows removed by filter	buffers	Index Cond
date_created, id	before: the OR	801.9 ms, 30.3 ms	250,001	13,942	tenant_id = …
	after: the row value	0.19 ms, 0.13 ms	0	36	tenant_id = … AND ROW(date_created, id) > ROW(…)
alias DESC, id	before: the OR	697.0 ms, 357.4 ms	250,001	256,252	tenant_id = …
	after: the bound and the OR	1.37 ms, 0.50 ms	1	518	tenant_id = … AND alias <= …

The generic plan (plan_cache_mode = force_generic_plan) keeps it: Index Cond: ((tenant_id = $1) AND (ROW(date_created, id) > ROW($2, $3))), 0.21 ms.

A walk through one tenant's 500,000 rows in pages of 500, through After itself: 65.6 s with the OR, 0.75 s now.

MySQL and MariaDB, 200,000 rows, (tenant, c1, c2) indexed, the page of 50 after the middle of one tenant's (EXPLAIN ANALYZE on MySQL, ANALYZE on MariaDB):

	MySQL 8.0 and 8.4	MariaDB 10.11 and 11.4
the OR	range scan over (tenant = 1 AND c1 = a AND b < c2) OR (tenant = 1 AND a < c1), 50 rows read	range, key_len 12, r_rows 50
the OR behind c1 >= a	the same range	the same
a row value	index lookup on tenant = 1, 26,301 rows read and filtered	ref, key_len 4, r_rows 26,301

SQLite (3.53.4, as ncruces/go-sqlite3 bundles it): the OR was searched by (tenant=?) alone, and the row value by (tenant=? AND (c1,id)>(?,?)).

Tests

• entc/integration/keyset pages a tenant's rows in four orders -- up, down, mixed, and mixed after two columns one way -- across pages that end inside runs of one c1, one run longer than a page, with another tenant's rows between them.
  • On every engine, every row comes back once, in the order asked.
  • On PostgreSQL each page is run again under EXPLAIN ANALYZE, with sequential and bitmap scans off so that a small table is planned as a large one is, and fails if its index scan threw away more rows than are tied with the cursor in the columns its bound covers.
  • On SQLite the plan's search must be constrained by the order's first column.
  • With master's After back, every order fails on PostgreSQL and SQLite from its second page. With the bound alone in front of the OR, PostgreSQL fails the one-way orders and the one that leads with two columns one way ("a page read 55 rows by Index Scan keyset_c1 (((tenant = 1) AND (c1 >= 8))) and threw 5 of them away").
• dialect/sql/sqlpage renders the predicate on each dialect.
• Run here, green: go test -race ./... in dialect; .github/lint.sh over the files changed; entc/integration/keyset on all six engines; the whole entc/integration with -race -count=2 on all six engines. Twenty packages passed together; the root package, enttime and migrate ran into go test's ten minutes on this machine, which was loaded throughout (load average 14 to 20 on 16 cores), and passed run one at a time (-p 1 -timeout 60m). CI ran the same in about four minutes.

Anything else with the OR

Nothing else in this repository pages by an expanded OR: sqlpage is the one keyset. In lesomnus/payday, ent#154 took the trail's four reads off it, and the generated List and Watch are this.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KxtnpRW533bdErnW8QPhsk