Repository navigation
docs: settle the v0.5 contract's API surface and interchange follow-ups (#278) - #392
Conversation
…ps (#278) Each row of #278 is now settled in the contract text, and each names the §24 row that tests it. Nothing is implemented; the M item in each row implements it. Owner rows apply the choices recorded in Deliverable Q: - AC3: len(t) raises TypeError pointing to count(), so list(t), sorted(t) and pandas.DataFrame(t) no longer run a hidden second execution through the length hint (S3). This turns Q's stated preference into a rule, so the owner should confirm it. - U2: no bound on tail(n), and distinct stays out of the MUST-stream list, with the R3 measurements recorded (R3). - OD-E: the general path is the reference, and a fast path ships only if it passes parity (P5, F1). Other rows: - AC6: every integer argument is SupportsIndex, read with operator.index; bool raises LTSeqTypeError (§2.3, §22). - AC7: the call and execution stages of an eager call are told apart by the demand rule, not by timing (E2). - AC8: supported versions in a new §2.4. - AC9: ltseq.typing.Literal is renamed LiteralValue and evaluates at runtime (S1). - AC10: is_in refuses a str or bytes argument (D7). - AC11: select and derive carry sort keys by one rule (B2, B3, O1). - AC12: t.concat() is defined exactly, and the UNION idiom is distinct(keep="any") (U2). - AC13: idioms for top_k and skew (A2). - AC16: ColumnNotFoundError is kept for r["x"], with the reason (E1). - X8: each scan reads its source when it opens it (Z1). - Run-end encoding and list views are normalized (T2, #239). - Importer exception classes are informative; the code and message prefix are normative (X3, P4). - N20: the float32 text form and where its exponent form starts (X4). Review record R holds the reasoning, the probes and what is still open. Refs #278
AI Code Review — Round 1Reviewed HEAD: FindingsR1-F1 [blocked]
|
R1-F1 (AC11): the rule for carrying keys records one name per key and
applies within one call, so derive(y=r.x) keeps x's key as x and a later
drop("x") truncates it; derive(y=r.x).drop("x") no longer claims to equal
select(y=r.x). B3 tests what the rule gives, and derive(y=r.x) equals
select(*t.columns, y=r.x). Section 9.2's truncation paragraph, section
11.2 and the derive bullet now say that a replaced key survives while a
result column holds it, as in derive(x=r.x) and the swap; G3 adds the
NestedTable swap.
R1-F2 (N20): the float32 notation bounds apply to the shortest decimal
the reference writes, not the binary value, so float32(1e-4), just below
1e-4, is written 0.0001. X4 adds it, 1e16 and their neighbors.
R1-F3 (AC13): section 14.2 gives a Python reference for the top_k idiom
that follows section 9.4's order (NaN first descending, NULL last, ties
in table order), and A2 uses it instead of heapq.nlargest, with the NaN
and NULL cases.
R1-F4 (AC8): section 2.4 declares the optional floors as extras, and a new
S6 row checks the wheel metadata against the floors and requires a CI job
that installs each floor exactly; AC8 names S6.
Refs #278
AI Code Review — Round 2Reviewed HEAD: FindingsR2-F1 [blocked]
|
R2-F1: section 2.3's single type statement named numpy, which only an if TYPE_CHECKING block imports, so reading LiteralValue.__value__ raised NameError with or without NumPy, and section 22.2 required the alias to be defined exactly that way. The type checker now reads a TYPE_CHECKING branch with the NumPy scalars. The runtime reads an else branch, whose lazy value imports NumPy when it is first read and falls back to the private _Scalar union when the import fails. Pyright 1.1.414 accepts the definition, and CPython 3.13 to 3.15 evaluate it with and without a NumPy module. Section 22.2 names both branches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R2-F2: S3 required list(t), tuple(t), sorted(t, key=...) and pandas.DataFrame(t) to equal to_dicts(), a list, which a tuple, a reordered list and a DataFrame never do. Each result is now compared with the matching conversion of rows = t.to_dicts(): the list itself, tuple(rows), sorted(rows, key=f) with the same key, and a DataFrame built from rows, checked with pandas.testing.assert_frame_equal. The single-execution count stays. The source is a CSV file without NaN, because two NaN objects compare unequal inside rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R2-F3: the moment ratio of the deviations underflowed for [1e-200, -1e-200], where d ** 2 and d ** 3 round to zero and give 0.0 / 0.0, and overflowed for [1e200, -1e200]. A2 nevertheless required the population skewness for every input. Section 14.2's idiom now divides each deviation by the group's largest, so every ratio lies in [-1, 1] and neither power overflows or underflows to zero. The section states the idiom's limits: x - mean cancels when the spread is tiny next to the mean, and a deviation beyond the float64 range gives NaN. A2 checks a 1e-12 tolerance against exact central moments for integer groups, 0.0 for the +-1e-200, 1e200 and 1e308 pairs, identical bits under power-of-two scaling, and NaN for constant, one-value and overflowing groups. A simulation of the idiom that rounds each step as section 14.2 specifies stayed within 4.4e-16 of the exact value over 3,000 random integer groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI Code Review — Round 3Reviewed HEAD: FindingsR3-F1 [non-blocked]
|
…ow limits (#278) R3-F1: the top_k idiom's joined string does not reproduce v0.4's. State the three differences (text form, ranking order, all-NULL group), how to restore "" for an all-NULL group, and add the joined string's cases to acceptance row A2 and the AC13 record row. R3-F2: scaling the deviations protects the largest deviation and the second-moment denominator, not every term. Show the group whose small deviations round to signed zero, bound the effect of underflow on the result, and add that group to row A2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI Code Review — Round 4Reviewed HEAD: FindingsNone. SpecThe issue’s rows are addressed in the contract and mapped to acceptance cases. Runtime implementation remains outside this documentation-only change. StandardsThe changes follow the repository’s modular contract and review-record structure. Repository guidance was reviewed. AssessmentThe revised contract provides implementation and acceptance guidance for the requested follow-ups. No actionable correctness or safety defect was identified in this round. ObservationsR3-F1 is addressed by explicitly documenting the top-k text, ordering and all-NULL differences. R3-F2 is addressed by qualifying the scaling discussion and describing the loss of small terms. AC3 applies the owner’s recorded preference; the review record identifies final owner confirmation as open. VerificationReviewed the bound diff, related contract and implementation sections, issue and PR context, prior reviews, repository guidance and supplied checks. All four CI checks succeeded; no controller validation was configured. SummaryNo findings require another fix round. Both round-3 findings are addressed. Needs another fix round: NO |
Why
The PR #250 assessment left seventeen API-surface and Arrow-interchange questions open. None blocked that merge, but each must be settled before the M item that implements it, or before the release. This PR settles every row in the contract text and names the §24 row that tests it. It changes no code: each rule is implemented by its M item, and the §24 rows stay specifications until then.
Owner rows
These apply the choices recorded in Deliverable Q on 2026-10-09:
len(t)raisesTypeErrorthat points tocount()(§3.2;__len__isNoReturnin §22; S3).list(t),sorted(t)andpandas.DataFrame(t)call__len__as a length hint, so an executinglenran the plan twice. Python treats aTypeErrorfrom the hint as no hint, so each now executes once. This was checked with CPython 3.13 and pandas 3.0.5.tail(n), anddistinctstays out of the MUST-stream list (§21.1, R3). The R3 measurements from test: pin the engine layout per test, add layout fixtures and the R3 memory harness (#274) #282 are recorded.Other rows
SupportsIndex, read once withoperator.index;booland non-index values raiseLTSeqTypeError(§2.3, §22 and every signature)ltseq.typing.LiteralbecomesLiteralValue, and its value evaluates with or without NumPyis_inrefuses astr,bytesorbytearrayargumentselectandderivecarry sort keys by one rule: bare references carry, any other expression truncates (§7.3)t.concat()is defined exactly; the SQLUNIONidiom isconcat(...).distinct(keep="any")(§13, §23.16)top_kandskew(§14.2)ColumnNotFoundErroris kept forr["x"], with the reason (§8.1)collect()is the snapshot (§5.1)list/large_list(§6.2)float32text keeps NumPy's shortest digits in Python'sreprlayout, with the exponent form outside 1e-4 to 1e16 (§16.5)The reasoning, probes and open items are in the new review record R (
review/history/follow-up-closure.md). Q gets a pointer, and the README, inventory and impact map are updated.Review focus
headandshowcannot disagree.derive(x=lambda r: r.x), which now keeps the key instead of truncating it.pyproject.tomlis unchanged. It already declares the two required floors. The pandas and NumPy floors would need extras, a packaging change left for the release gate.Validation
head(np.int64(3)),r.x.shift(np.int64(1)),ntile(np.int64(4)), ...) went from 16 errors to 0.float32text-form reference was checked on the finite values among 300,000 random bit patterns with NumPy 2.4.4. Every form read back to the same value and had NumPy's shortest digits.[§N]link in the contract and review modules resolves to a heading.git diff --checkis clean.Still open
read_parquettreats a file with no leaf columns whose footer records rows is unprobed; it is needed before M22.Closes #278