docs: record each explanation once, in the venue that owns it - #396
Conversation
c95d6e9 to
87a87b0
Compare
ehinman
left a comment
There was a problem hiding this comment.
Mostly questions about what the text says and whether it makes sense to add an ADR0000 after ADRs 0001-11 have already been written. Does it make sense to edit the text of an old ADR, or add to it as an addendum? Seemed silly to read through editorial changes to past "Decisions".
| states the same order as seven rungs, splitting three of these sources into the | ||
| scopes inside them. An adapter default is not one of the branches here: it | ||
| arrives as the ``default`` a read site like :func:`concurrency` passes for its | ||
| own adapter. |
There was a problem hiding this comment.
No idea what any of this means.
There was a problem hiding this comment.
Fair — the paragraph reconciled the four sources here against ADR 0011's seven rungs, which is unreadable without 0011 open. It now says the two list the same order, that 0011 is the finer grain, and that an adapter default arrives as the default argument rather than being a fifth source.
| lines restating decisions already recorded in ADRs 0003 through 0011. Three | ||
| examples: | ||
|
|
||
| - ``dataretrieval/configuration.py``'s module docstring spends 65 lines |
There was a problem hiding this comment.
Are these examples needed?
There was a problem hiding this comment.
Trimmed. The three bullets fold into one sentence in the paragraph above — their file-by-file detail belonged to the audit, not to the decision.
| (``__init__.py``) exposes the service-adapter seam: ``OgcDialect``, | ||
| ``prepare_request_args``, and ``get_ogc_data`` (whose ``cql_body`` | ||
| parameter covers verbatim-CQL2 queries). | ||
| Protocol subsystem for Water Data and NGWMN. A facade (``__init__.py``) |
There was a problem hiding this comment.
Can we not use "facade"? That's a very AI-y word.
There was a problem hiding this comment.
Leaving this one, with the reasoning: "facade" is the Gang of Four pattern name rather than an AI coinage, CONTEXT.md defines it as a re-export with no logic of its own, ADR 0007 is titled with it, and three architecture tests are named for it. Renaming it is a vocabulary change across the ADR set, the glossary, AGENTS.md, and the tests — worth its own PR if you want it. Say the word and I will open one.
There was a problem hiding this comment.
Fair enough. And I see it defined in the CONTEXT.md In another PR, can we link to the CONTEXT.md in a few more places? Like in the README? This is really helpful for people digging a little deeper into the architecture who are not familiar with the jargon.
There was a problem hiding this comment.
I recommend something like README -> CONTRIBUTING -> CONTEXT, etc. for humans, and AGENTS.md -> CONTEXT, etc. for bots
|
@ehinman, thanks for the review. The idea of AD000 was that it will become a general ADR for documentation that we can reuse across projects. That's the longerterm goal at least. When to create a new ADR versus amend an old one is also TBD. The ADR practice seems to be useful, but we need to adapt it for agents that are less human and more like the guy in Memento. |
Review of DOI-USGS#396 flagged prose that assumed vocabulary or context the reader does not have, and asked whether amendments to accepted ADRs are visible as amendments. No code changed. Wording: - "Sugar for" and "the dunders" named the mechanism in jargon rather than naming what runs; `__len__` and `__iter__` are now spelled out. - `credentials.py`'s note on why `session` is absent was in past tense, as though describing a bug that had been fixed rather than the reason the list reads as it does. - `configuration.py` reconciled its four resolution sources against ADR 0011's seven rungs, a mapping that meant nothing without 0011 open. It now says the orders are the same and that 0011 is the finer grain. - ADR 0000's "The problem is elsewhere" read as a claim about time rather than about place; a reviewer asked whether it was still true after this PR. It now names the internal modules. The three worked examples fold into that paragraph, since their detail was the audit's, not the decision's. - ADR 0012 said "the category" with no antecedent, and inventoried the four deprecation spellings module by module -- history the commit that removed them owns, per this PR's own policy. Amendments: Each amended record already logged its added clauses in `Notes`, at the bottom. A reader met the amended Decision text first and the fact of amendment last. `Status` on 0004, 0006, 0007, 0008, 0009, and 0011 now says the record was amended and where the log is, and ADR 0000 states that placement as the rule. `facade` was raised as jargon and is left as is: it is the GoF pattern name, CONTEXT.md defines it, ADR 0007 is titled with it, and three architecture tests are named for it. Changing it is a vocabulary decision for its own PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD
| or a config-file path — which names the concrete thing a value came from, for | ||
| error messages and `show_configuration()`. The source is the category; the | ||
| label is the instance. | ||
| setting does not displace another setting's value from a lower source. |
There was a problem hiding this comment.
This is confusing because you got rid of source in a lot of places and replaced it with tier and don't define tier anywhere.
There was a problem hiding this comment.
yeah, there's still some glossary drift. I'm hunting for the bug in the agent design, rather than chasing each instance.
There was a problem hiding this comment.
Correct, and it predates this PR — "it does not replace the package-wide tier" was already in CONTEXT.md before this record was written. Recording each explanation once made the gap visible rather than creating it.
Fixing the one word would have left the reason it got there, and this is the third review in a row to land on the same seam, so #405 addresses the seam: the glossary holds two kinds of term. Ones we invented (chunk, page, source) appear in no USGS API and take one spelling everywhere. Ones the services own (monitoring location, collection) are spelled four different ways upstream — site_no, Station, monitoring_location_id — so the glossary fixes one word for prose while each adapter keeps its own service`'s spelling in its parameters.
That gives your question an answer that does not depend on who reviews: in nwis a docstring may say site; in transport it may not.
There was a problem hiding this comment.
One correction since: swapping in source there was still the wrong axis. Package-wide vs adapter is what ADR 0010 calls a tier — its own word for what the glossary now defines as scope — while a source is block/environment/file/default. The sentence now names neither axis ("leaving that value standing for every other adapter"), and #405 defines Scope and the precedence ladder so source, scope, and rung each mean one thing.
Rationale had accumulated in docstrings alongside the ADRs that already owned it: ~500 lines restating decisions ADRs 0003-0011 already make, two of the copies already stale. New ADR 0000 records where each kind of explanation belongs -- contract in the docstring, local constraint in a comment, history in the commit message, cross-cutting decisions in an ADR cited by number, vocabulary in CONTEXT.md -- and asks that the sentence a citation leaves behind stand on its own. New ADR 0012 records the deprecation policy. 26 decisions the code carried become clauses in the records that own them (0004, 0006-0009, 0011), amendments marked in Status and logged in Notes; 36 sites across 29 files drop the argument and cite the record. CONTEXT.md gains the terms the ADRs used as if defined, and a new test fails on a citation naming no record. No package code changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QVhHGSDkVxWzeUMmHn6oAT
f214266 to
34c3e46
Compare
* docs: separate the terms we own from the terms the services own Review of #396 found `CONTEXT.md` using *tier* in its own prose without defining it. The word was there before that record was written; recording each explanation once made the gap visible rather than creating it. Fixing the one word would have left the reason it got there. The glossary is one flat list, and every place the code disagrees with it is filed under *Known legacy names* -- a heading whose framing is that the disagreement is debt. For most of the glossary that is right. For a few terms it is wrong, and being wrong about those has produced the same argument repeatedly: may a docstring say *site*, may `service=` name a collection, is prose about NWIS bound by a word taken from the Water Data API. The two sets differ in who has authority over them. *chunk*, *page*, *fan-out*, *plan*, *interruption*, *dialect*, *leaf* appear in no USGS API's vocabulary. They were invented here for machinery this package owns. Nothing outside constrains them, so spelling one of them three ways -- the resolution chain's layers have been a source, a tier and a rung -- is an inconsistency that can be removed by deciding, and should be. *monitoring location* and *collection* are not like that. The services name those things and disagree: `site_no` in NWIS, `Station` in WQP, `monitoring_location_id` in Water Data, a `sites` collection in NGWMN. No decision here reconciles them. A caller who read WQP's documentation looks for `Station`; an adapter that renamed it would be harder to use, and the parameter is public surface besides. One rule over both forces a choice between losing the glossary, and with it the vocabulary that lets transport and the OGC engine be written once, or enforcing it onto every adapter's public surface. So: ADR 0013 records the split. Core terms take one spelling everywhere, identifiers included, and a second spelling is a defect. Domain terms fix one word for prose while each adapter keeps its service's spelling in its parameters -- which is the adapter doing its job, not debt. `CONTEXT.md` follows. The preamble names the two kinds. *Monitoring location* and *collection* carry their per-service spellings in their own entries rather than as exceptions at the end. *Known legacy names* keeps only core-term spellings. And the package-wide **source** is called that, in the sentence that had called it a tier. This also settles what #400 is. Renaming the configuration chain's `tier` to `source` looked discretionary while the glossary bound only prose; the chain is core machinery, so one spelling binds its identifiers too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD * docs: read tier and rung as the records wrote them The glossary's Source entry claimed ADR 0010 calls a source a tier and ADR 0011 a rung. Neither is true. ADR 0009 and 0010 already say source -- source-major precedence is 0010's own phrase. 0010's tier names a different axis, the scope: the top-level tier that survives, the deferred gateway tier. 0011's rung is one position of its seven-rung ladder, where sources and scopes interleave. The code's tier-for-source identifiers misapplied 0010's scope word, and the glossary projected that confusion back onto the records; ADR 0013's standing example repeated it. Correcting the history also surfaced what was missing: Scope had no entry, though ADR 0010's whole subject is scoping, and the axis mixup is exactly the confusion a definition prevents. The Adapter-scoped setting entry carried the same error in the other direction -- "does not replace the package-wide source" named a scope level a source -- so the sentence now names the value rather than either axis. Also from this pass: portal is defined as the human-facing site fronting a service rather than left as an undefined near-synonym (CONTRIBUTING now says service where it meant the module split); the stall-timeout bullet no longer invites new prose to use either spelling; Facade is grounded in the design pattern it is named for; and the Source entry points NLDI's data_source at the domain-term rule. The preamble's define-before-use rule is restated without the ordering claim two mutually-referencing axes cannot satisfy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD * docs: give the precedence ladder and its rungs an entry ADR 0011, the user guide, and the configuration tests' comments all say rung; a word in use across three venues has earned its definition. With rung defined, the user guide's one "tiers below" was a second spelling of it and now reads "rungs below". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD * docs: drop the portal entry; group by service in the interface view Defining *portal* as the human-facing site created three problems it did not solve. The word already carries three unrelated senses here: WQP's proper name (the Water Quality Portal), anyio's blocking portal in transport/fanout.py, and the user guide's page title for a table of websites. The new definition made five existing wqp.py docstrings violations -- "the portal serves the legacy and WQX3 interfaces from one root" is about the service, and reads naturally precisely because that service is named Portal. A glossary entry that manufactures violations in correct prose is not carrying its weight. The CONTRIBUTING change it came with stands on its own: modules are grouped by service, which is what the Services section enumerates, and "data portal" maps to neither the modules nor the portal table cleanly. The architecture interface view said the same thing and now matches. ADR 0001 keeps its wording, being accepted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD * docs: state ADR 0013's split where agents actually read it AGENTS.md's first bullet said "when a term there conflicts with a name in the code, the term wins" -- the rule in its pre-0013 absolute form. That sentence is load-bearing: it is the entry point every agent and contributor reads before writing anything, and read literally it licenses renaming an adapter's public parameters to match the glossary. It did exactly that here, producing a rename that had to be reverted. ADR 0013 splits the rule, so the bullet now does too: a core term binds identifiers, a domain term binds prose only and each adapter keeps its service's spelling. The two examples are the ones reviewers keep asking about. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JAEQqs7XzQHGQQi2KakuXD --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Why
Rationale accumulated in docstrings alongside the ADRs that already owned it, because adding a paragraph where the reader is standing is easier than citing a record they would have to open.
An audit of
dataretrieval/found ~500 lines restating decisions ADRs 0003–0011 already state:configuration.py's module docstring re-derived ADRs 0009/0010/0011 in 65 lines — while citing them three times.transport/retry.py,transport/liveness.py, andtransport/fanout.py.This is a correctness problem rather than a tidiness one. Every copy is a place the rule can be updated without the others, and two had already gone stale — an overview paragraph describing concurrency caps a later ADR removed, and an ADR clause describing a credential rejection the code deliberately no longer performs. A reader has no way to tell which copy is current.
What ADR 0000 decides
Each explanation is recorded once, in the venue that owns that kind of knowledge:
Parameters,Returns,Raises,ExamplesCONTEXT.mdThree rules follow: cite rather than restate, write from the reader's position, and never edit an accepted record to reverse its meaning.
The middle rule was added after review. The venue rules answer where an explanation goes; nothing in them asked who it reads for, and review of this PR found eleven places where correctly-placed prose was still unreadable from outside the author's head. One of them broke this record's own history rule — ADR 0012 inventoried the four deprecation spellings module by module — which means the single review question this record shipped with was not enough.
Compliancenow asks a second: could a reader who has not opened the cited record follow this sentence?Public getter docstrings are explicitly out of scope. They hold 55% of the package's docstring lines at 2.5 lines of prose per line of code, and that is the deliverable — the policy says so in as many words, so this cannot be read as licence to trim them.
What changed
MAX_CONCURRENT_REQUESTSis gone), and ADR 0009's claim that the**queryablespath refusessession, whichcredentials.py:114deliberately does not.CONTEXT.mdgainsDialect,Single-shot adapter,Stall timeout,Fitness function— terms the ADRs used as if defined — and loses two self-contradictions. The largest fix is facade vs adapter: the glossary reserves "facade" for a re-export with no logic of its own, while the overview titled the public surface "facades" and then describednwdcas one that "builds CSV requests".Enforcement
test_adr_references_resolve_to_a_recordasserts everyADR NNNNcitation resolves to a record, across every venue ADR 0000 names — package,docs/,CONTEXT.md,CONTRIBUTING.md,AGENTS.md. A renumbered record fails the suite instead of leaving a dead pointer. It caught one bad citation in this PR's own diff.Whether a given paragraph should have been a citation stays a review judgement. ADR 0000 argues against a proxy metric explicitly: a prose-volume ratio would push contributors to delete parameter documentation to move a number.
Review notes
Status. Review asked whether these edits should be addenda. They were —Notesheld the log — but at the bottom, so a reader met the amended Decision text first and the fact of amendment last.Statuson each amended record now says it was amended and points atNotes.Noteswhat was added after acceptance, and where Decision text was corrected rather than added to — ADR 0006's retry scope was widened from "gateway 5xx", which was never true of a fanned-out call. No decision is reversed.base_urlwas-once-optional story, and the_configuration_core.pybenchmarks.Conflict with #388
Both branches amend ADR 0006 and both add a
Notessection at the end, so they will conflict. It is a straightforward both-added conflict; land #388 first, since it is further along.Verification
1130 passed,mypy --strictclean over 60 files, ruff / xenon / complexipy / import-linter all green, every ADR parses clean as RST.🤖 Generated with Claude Code