Skip to content

Classify malformed CZDS records safely in preflight and aggregation - #9

Merged
tulik merged 5 commits into
mainfrom
fix/czds-record-shape-diagnostics
Oct 22, 2026
Merged

tulik merged 5 commits into
mainfrom
fix/czds-record-shape-diagnostics

Conversation

@tulik

@tulik tulik commented Oct 22, 2026

Copy link
Copy Markdown
Member

Context

Production gov.txt.gz failed non-leading-SOA preflight with reason=record_fields. The licensed record is not available in public CI, so this change classifies the structure of malformed records without making an unverified grammar relaxation.

Changes

  • Introduce classify_record_fields() in build_nsdb.py: record_blank, record_fields_comment, record_fields_directive, record_fields_space_delimited, record_fields_missing_tabs, and record_fields_partial_tabs.
  • Use the same classifier for malformed first records in preflight and malformed subsequent records in the production parser. The parser already verifies field count before name normalization, and that order is preserved.
  • Add synthetic compressed-zone fixtures for each class, at both leading and non-leading records. Verify preflight and full aggregation emit the same fixed codes without leaking the synthetic owners, NS targets, or RDATA.
  • Document the codes and protected-runner diagnosis.

Safety and interface boundaries

  • No grammar acceptance change: malformed records still fail before aggregation; SOA consistency, ordering, deduplication, and incomplete-input checks remain unchanged.
  • No record, domain, target, serial, offset, exact count, or underlying exception text enters public logs.
  • process_input() and aggregation interfaces remain unchanged; compatible with checkpoint issue Resume CZDS aggregation from verified per-zone checkpoints #6.
  • This is diagnostic instrumentation, not yet the resolution of the .gov input mismatch.

Verification required

CI must pass. After merge, rerun preflight with the protected cached CZDS set and determine the newly surfaced reason=<subcode> before designing a targeted parsing fix.

Related: #5, #6, #7

@tulik
tulik merged commit 78669f1 into main Oct 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant