Skip to content

fix: charge json_fidelity only for VARIANT columns a statement may read - #102

Closed
kvlonge wants to merge 2 commits into
mainfrom
fix/unread-variant-columns
Closed

kvlonge wants to merge 2 commits into
mainfrom
fix/unread-variant-columns

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Why

Under strict mode, any table with a VARIANT, OBJECT or ARRAY column refused every query with SQ0008 json_fidelity, even when no semi-structured value was read. For example, SELECT ticket_id FROM tickets was refused because of an unread ARRAY column, and it was the largest strict blocker in the realistic-query corpus.

Changes

  • A base table's semi-structured column is charged json_fidelity only when the statement may read it: by name anywhere in the statement, through a star other than COUNT(*)'s, or through a NATURAL join.
  • The scan over-approximates. The runtime JSON guard stays on every such column, so a read the scan missed raises loudly rather than returning a wrong value.
  • The FROM relation rule's catalogue conditions describe this.

Verification

  • A CLI test under --builtin-policy checks that scalar reads and a UNION ALL branch convert. Naming the ARRAY column in the select list or WHERE, a star, or a NATURAL join still need json_fidelity.

A base table's VARIANT, OBJECT or ARRAY column was guarded, and charged
json_fidelity, whenever the table was read, so strict mode refused
SELECT ticket_id FROM tickets because of an unread ARRAY column. The
guard and the charge now apply only when the statement names the column,
uses a star other than COUNT(*)'s, or joins NATURAL; the guard's failure
is loud, so a missed read raises rather than returns a wrong value.
Only the json_fidelity charge depends on whether the statement may read
the column; the guard stays, so a read the scan missed raises loudly
instead of returning an unguarded value.

kvlonge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #133 (fix/builtin-launch-refusals), which implements the same contract: a base table's VARIANT, OBJECT or ARRAY column needs json_fidelity only where the statement may read it. #133 also skips the json guard for unread columns, and treats a positional $n, PIVOT/UNPIVOT and COLUMNS as reads. Its FROM_RELATION catalogue conditions take this PR's wording. Closing without merging.

@kvlonge kvlonge closed this Nov 6, 2026
@kvlonge
kvlonge deleted the fix/unread-variant-columns branch November 9, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant