Repository navigation
Conversation
6390849 to
2a4906d
Compare
d4715d3 to
c276fde
Compare
|
I'm not sure this is ready to merge, but would invite reviews! |
a71ca1b to
a835cec
Compare
Scope: a thin end-to-end slice in line with the merged design documents and the work order of #35. Two prepared Google Benchmark build directories are alternated over rounds by the calling script, one work order per side and round; bx run measures and delivers the results to the local store; the comparator evaluates them in run mode. Both the revisions (UC-03) and environments (UC-04 shape) profiles are demonstrated, ad hoc (no project, thin results) and tracked. Building and environment policy are out of scope (benchmark-environments.md); history mode is deferred. Eight success criteria and the decisions locked for the prototype. Design: one CLI over a Python API; the #35 work order, extended only by the round and slot it lists as open; a runner that records the zero-configuration snapshot, source identity, CMake build configuration, and declared facts, and refuses what it cannot apply exactly; a local store that is a single Parquet file in one place, ~/.benchx/store.parquet, holding #30's row per result plus the canonical document and its work order, rewritten atomically on ingest, with entities and series computed on query; run-mode comparison with profile invariants and the paired method benchx/paired-relative/v1; bx head for the latest results; a demonstration of both profiles; deliberate simplifications and implementation notes. Also adds docs/design/README.md, the reviewer-facing index of the merged design documents.
|
I think this has drifted a bit since |
main has since taken the mapping into src/benchx/parquet.py and moved the schemas into the package, and the result schema has changed under it. - benchx.parquet: add message_table() and MESSAGE_SCHEMA (the message columns alone), and record benchx.message_schema_sha256 in the file metadata of both the message table and the store, as the document says. - tools/json_to_parquet.py: a thin wrapper over benchx; it reads and checks documents as `bx validate` does and writes message_table(). - tests/test_parquet.py: pytest tests of benchx.parquet and the tool, replacing tests/test_json_to_parquet.py; strict reading is already covered by test_loader.py. - measurement-result-parquet.md: regenerate §3 for the current schema (string schema_version, quantity under measurement, protocol, system, measurement.kind), describe the store's derived columns and document column, use the shipped paths, and rerun the DuckDB query over the six examples. - List the document in the design index and the site nav, and replace the "#30" references in the code and docs with the document's name.
| |---|---| | ||
| | closed object (`additionalProperties: false`) | `struct` with the same field order; required properties are non-nullable | | ||
| | open object, or a value of mixed primitive types | `json`: the Arrow JSON extension type, written with the Parquet JSON logical type, holding RFC 8785 canonical text | | ||
| | `oneOf` of closed objects (`summaries`) | one `struct` with every property any alternative declares, in first-appearance order; non-nullable only when every alternative requires it; alternatives that give one property different types are an error, not a silent choice | |
There was a problem hiding this comment.
here it says
oneOfof closed objects
However, is that what the code does?
Lines 37 to 49 in 4778ea5
does it check that they're actually closed (i.e. additionalProperties: false)?
There was a problem hiding this comment.
Indeed code didn't match the docs. Fixed.
| | integer, integer const | `int64` | | ||
| | number | `float64` | | ||
| | boolean | `bool` | | ||
| | any other `const`, `enum`, or `oneOf` | an error, so that a schema change the rules do not cover is noticed rather than stored loosely | |
There was a problem hiding this comment.
this looks different to what's enforced?
it looks like we reject the case where no objects are present
Lines 40 to 41 in 4778ea5
but then wouldn't oneOf: [{"type": "string"}, <closed object>] still get through?
There was a problem hiding this comment.
Yes indeed it would. Addressed.
Describe the schema under which a validated measurement result is stored in Parquet, and add a small program that derives the Arrow schema from the message JSON Schema and writes messages to one Parquet file.
Reject duplicate object keys, non-finite number tokens, and numbers outside the RFC 8785 domain, including integers beyond the I-JSON safe range, before JSON Schema validation. Document why in the Parquet notes.
main has since taken the mapping into src/benchx/parquet.py and moved the schemas into the package, and the result schema has changed under it. - benchx.parquet: add message_table() and MESSAGE_SCHEMA (the message columns alone), and record benchx.message_schema_sha256 in the file metadata of both the message table and the store, as the document says. - tools/json_to_parquet.py: a thin wrapper over benchx; it reads and checks documents as `bx validate` does and writes message_table(). - tests/test_parquet.py: pytest tests of benchx.parquet and the tool, replacing tests/test_json_to_parquet.py; strict reading is already covered by test_loader.py. - measurement-result-parquet.md: regenerate §3 for the current schema (string schema_version, quantity under measurement, protocol, system, measurement.kind), describe the store's derived columns and document column, use the shipped paths, and rerun the DuckDB query over the six examples. - List the document in the design index and the site nav, and replace the "#30" references in the code and docs with the document's name.
A oneOf was accepted whenever it had an object alternative. An open object was merged into the struct, dropping undeclared keys, and any non-object alternative was stored as if it were a bare observation number. Accept only closed objects, or one number beside one closed object with a number value.
4778ea5 to
815bebc
Compare
…ll line The mapping took the first entry of an allOf whatever it was, so one that does not start with a reference was stored as JSON instead of being noticed. The converter is run from a checkout: tools/ is not in the installed package, and the released benchx has no message_table yet. Drop the note about a work-order design document in review, which was closed.
…ing and the round trip The design said a schema change "changes the file schema" and that the checksum names the message schema, which holds for a file written in one go but not for the local store: it rewrites its file at every ingest under permissive unification, so its checksum names the last write. §4 now says so and lists what appending does for each kind of change. Two tests back claims the document already made: - the listing in §3 is parsed and compared with the derived schema, so it cannot drift from schema.json unnoticed - each example is rebuilt from its row and compared with the message, so a field the mapping drops is noticed
The Parquet form of a validated measurement result (#2, §5.2 ingest object): the columns a result is stored under, which the local store's row begins with. It defines nothing about the message itself; where it and the schema differ, the schema wins.
docs/design/measurement-result-parquet.md: the mapping from the message JSON Schema to Arrow and Parquet types, the resulting schema for message schema 0.1.0, the file metadata, the columns a store adds beside them, and how to read the file, with a DuckDB query over the six example results. One row is one result; closed objects are structs, open objects are canonical JSON columns, and observations are one typed list column.src/benchx/parquet.py: the mapping already lives here onmain. This PR addsmessage_table()andMESSAGE_SCHEMA(the message columns alone), and recordsbenchx.message_schema_sha256in the file metadata, so files written under different drafts of 0.1.0 can be told apart. This includesstore.parquet. It also tightens the mapping so that aoneOforallOfoutside the documented shapes raises instead of being stored loosely.tools/json_to_parquet.py: a small script built onbenchx. It reads and checks each document the waybx validatedoes, then writesmessage_table()to one Parquet file. Nothing is written if any document is invalid.tests/test_parquet.py: the mapping over the examples (row order, nullability, canonical JSON, observations, metadata), the schema nodes the mapping must reject, and the tool.README.md,docs/design/README.md, and the site navigation. The "Add Parquet storage schema and a PyArrow example #30" references in the code and docs now name the document instead.