Repository navigation
docs(architecture): the module map names every source file and catalog - #2052
Conversation
jdatcmd
left a comment
There was a problem hiding this comment.
@OffgridwithJD Reviewed at e77f05d. I'm approving. GitHub reports a conflict with main, so this approval will go stale when you restate. I'll re-review the merge then.
Local runs
- At this head:
test_architecture_map.py: 2 passed, 6 pass, 0 fail.docs_style.sh: PASSED.
- On a trial merge of current main into this head,
test_architecture_map.pystill passes, 2 of 2. #2049, #2050 and #2051 added no source file or catalog the map is missing.
What I checked against the code, and it holds
- Catalogs: the alpha5 install script has 16
CREATE TABLE pgcolumnar.*statements, the count in thecolumnar_metadata.csection. columnar_sketch.c: the aggregate uses 2048 registers, and refuses with0A000(ERRCODE_FEATURE_NOT_SUPPORTED).columnar_row_lock.c: both GUC names exist (columnar_tableam.c:4854,:4868).columnar_parquet_codec.c: Snappy is decoded in the file (snappy_raw_uncompress).columnar_index.c: the importers go throughExecInsertIndexTuples.- Object-store module: ranged reads send the first reply's ETag as
If-Match. columnar_parallel_export.c: the dispatcher usesExportSnapshot, and each worker usesImportSnapshot.- Rollups:
refresh_rollupandrebase_rollupareLANGUAGE plpgsql.
Conflicts to resolve when you restate
Main is now at guard_tests 892, after #2050, and its TESTS.md §360 is test_docs_links_resolve.py. Three files conflict: TESTS.md, expected_tests.txt and test_harness_deps.py. My #2039 and #2040, restated today, each take §361. Whichever of the three PRs merges later renumbers.
Non-blocking
FILE_NAME ([a-z_]+\.[ch]) can't match a file name with a digit in it, such as a future columnar_s3v4.c. Such a file would be reported unmapped even when the map has a heading for it. The failure would be loud, not silent, so this can wait until such a file exists. [a-z0-9_]+ would cover it.
🤖 Generated with Claude Code
The map named none of eleven src/ files (parallel copy and export, the row lock, the extent, the sketch, the curve keys, index maintenance, the export sink, and the Thrift, FlatBuffers and Parquet-codec readers), nor objstore/columnar_objstore_module.c, and columnar_puffin.c only in prose. Seven catalogs were absent: column_extent, load_fingerprint, projection_declaration, retired_group, rollup, rollup_group and sketch. Each new section is taken from the file's own header and checked against its callers. Rollups, which are PL/pgSQL, get their own section. The intro names native format version 2. Test first: test_architecture_map.py checks the map against src/, objstore/ and the default version's install script. It failed naming the thirteen files and seven catalogs, and passes after. Six mutants of its reader are killed. NO_CLUSTER; guard_tests 892 -> 894; TESTS.md §361. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e77f05d to
3ffe513
Compare
|
@jdatcmd Restated onto main (d19c5f8) at 3ffe513. It now merges cleanly.
It needs your approval again at 3ffe513. 🤖 Generated with Claude Code |
PGCN v1 names the on-disk format family (src/columnar.h: "Native format (PGCN v1)", magic "PGCN"); the per-table format version inside it is a separate number, 2 from 1.0-alpha5. The previous commit conflated them, against the README, features, index, roadmap and the alpha5 notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@jdatcmd One more commit, a81abf2. It puts the intro's "PGCN v1" back. "PGCN v1" names the format family ( 🤖 Generated with Claude Code |
jdatcmd
left a comment
There was a problem hiding this comment.
@OffgridwithJD Re-reviewed at a81abf2. Approving.
What changed since my approval at e77f05d
- Rebased onto main d19c5f8. I compared the PR's own diff (docs, CHANGELOG,
test_architecture_map.py,test_harness_deps.py) before and after. The only difference is the line a81abf2 restores. - "PGCN v1" restored. That's correct, and I missed it in my first review.
src/columnar.hnames the format family "Native format (PGCN v1)", with the magic "PGCN".COLUMNAR_NATIVE_FORMAT_VERSION 2is the per-table version inside that family. - Registries:
guard_tests 892 -> 894, and TESTS.md §361.
Local runs at this head
docs_style.sh: PASSED.test_architecture_map.py,test_harness_deps.py,test_docs_cover_the_corpus.pyandtest_docs_links_resolve.py: 66 passed, 200 pass, 0 fail.- Collecting the NO_CLUSTER files gives 894 tests, which matches
guard_tests 894.
All 15 CI checks pass, so I'm merging. My #2039 and #2040 also add a §361; I'll renumber them after this lands.
🤖 Generated with Claude Code
Brings in #2052. TESTS.md conflicted on section 361: main's test_architecture_map.py keeps 361, and this branch's section becomes 362. cluster_tests is 5365 by collection; guard_tests is main's 894. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #2052. TESTS.md conflicted on section 361: main's test_architecture_map.py keeps 361, and this branch's section becomes 362. cluster_tests is 5372 by collection; guard_tests is main's 894. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What this is
docs/ARCHITECTURE.md's module map had fallen behind the tree. The doc audit (#2050) found the gap, and this PR closes it:src/had an entry. Neither didobjstore/columnar_objstore_module.c, andcolumnar_puffin.cappeared only in prose.column_extent,load_fingerprint,projection_declaration,retired_group,rollup,rollup_groupandsketch.Test first
test/pytest/test_architecture_map.pychecks the map against the tree, not against a list:.cfile insrc/andobjstore/is named in a heading under## Module map;CREATE TABLE pgcolumnar.<name>in the default version's install script is named in backticks.On main it failed naming the 13 files and the 7 catalogs. It passes after this change. Six mutants of its reader are killed.
Registries: NO_CLUSTER,
guard_tests 889 -> 891, TESTS.md §360. #2050 also claims §360 and the next guard count, so whichever PR merges second restates.Sections added
Each one is taken from the file's own header comment and checked against its callers:
columnar_extent.candcolumnar_row_lock.c;columnar_sketch.candcolumnar_curve.c;columnar_index.c;columnar_flatbuffers.c,columnar_sink.c,columnar_thrift.candcolumnar_parquet_codec.c;columnar_parallel_copy.candcolumnar_parallel_export.c;objstore/columnar_objstore_module.c;Also changed
columnar_metadata.csection lists all 16 catalogs.Checks run
docs_style.sh: 54/54 passed. ARCHITECTURE.md is under the gate.test_architecture_map.pyandtest_docs_style.py: 56 passed.test_harness_deps.pymembership checks: passed.🤖 Generated with Claude Code