fix(nwis): keep censored peaks instead of dropping or dating them

[thodson-usgs]

Stacked on #344 — that branch is merged in, so the diff to review here is preformat_peaks_response and its tests. Rebase or merge once #344 lands.

The bug

nwis.get_discharge_peaks and get_record(service="peaks") silently discard every peak whose date is only partly known.

NWIS zero-fills the unknown part of a historical peak's date. From the RDB's own legend:

#   A ... Year of occurrence is unknown or not exact
#   Bd ... Day of occurrence is unknown or not exact
#   Bm ... Month of occurrence is unknown or not exact

So 1858-00-00 (month unknown) and 1844-06-00 (day unknown) are real, qualified records. Neither parses as a date, so preformat_peaks_response coerced both to NaT and the dropna below deleted the row — discharge value and all.

The loss is biased toward the records that matter most, because the imprecisely dated peaks are the old ones. Across six sites, 24 of 672 peaks were being dropped:

site	peaks	dropped before	after
14105700	167	20 (incl. 1859 @ 847,000 ft³/s)	0
02035000	129	2	0
09380000	104	1	0
06934500	99	1 (1844 @ 700,000 ft³/s)	0
01646500 / 01491000	95 / 78	0	0

Why the date is left as NaT rather than completed

An earlier revision of this branch pinned each censored date to the start of the known period (1858-00-00 → 1858-01-01), mirroring waterdata.get_peaks(). That was wrong twice over: it put a day in the frame that the record does not claim, and since the function popped peak_dt, it destroyed the only evidence the date was ever approximate.

There is no representation that avoids this within the datetime column. A datetime64 value is a full instant:

pd.Timestamp('1858')     # -> 1858-01-01 00:00:00
pd.Timestamp('1858-06')  # -> 1858-06-01 00:00:00

Mixed-frequency Period values fall back to object dtype and raise IncompatibleFrequency on PeriodIndex/astype, so they cannot form an index either. Any value in that column asserts a precision NWIS did not supply.

So the peak is kept and datetime is left NaT:

   peak_dt datetime  peak_va peak_cd
1858-00-00      NaT   563000      Bm
1859-00-00      NaT   847000      Bm
1844-06-00      NaT   700000      Bd

peak_dt is no longer popped. The peaks response has no water_yr, so it is the only column carrying a censored peak's year — and the only dependable way to tell an unknown day from a known one, since peak_cd holds the qualifier for just 22 of those 24 rows.

Rows whose peak_dt is blank are kept on the same reasoning: a missing date is not a reason to discard a discharge value.

Callers who want a resolved timestamp alongside explicit year/month/day and a qualifier field should use waterdata.get_peaks(), which the modern API already provides.

Notes

• Behavior change: peaks queries return more rows; datetime may be NaT, so a caller selecting on the datetime index will not see those peaks and should filter on peak_dt; and peak_dt is now present in the returned frame.
• ADR 0005 permits compatibility, security and correctness fixes on the deprecated nwis facade. This adds no public symbol, getter, or parameter.
• The waterservices_peaks fixture has no censored dates, so its row count is unchanged — its assertion moves 240 → 260 purely because peak_dt is retained. Restated as a shape so rows and columns are no longer conflated.

Verified: live across six sites, 672 rows in and 672 out, zero dropped, zero invented dates. 1136 tests pass; mypy --strict, ruff, Xenon, complexipy and import-linter all clean.

🤖 Generated with Claude Code

[thodson-usgs]

Reworked the approach after maintainer review — the PR body and title are updated to match, so this note is for anyone who read the earlier version.

What changed: the previous revision pinned each censored date to the start of the known period (1858-00-00 → 1858-01-01), mirroring waterdata.get_peaks(). That put a day in the frame the record does not claim. Worse, because the function popped peak_dt, it destroyed the only evidence the date was ever approximate — so a caller could not distinguish an imputed date from a real one.

Why there's no middle option. I checked whether the datetime column could carry mixed precision. It can't — a datetime64 value is a full instant, and pandas imputes for you the moment you construct one:

pd.Timestamp('1858')     # -> 1858-01-01 00:00:00
pd.Timestamp('1858-06')  # -> 1858-06-01 00:00:00

Mixed-frequency Period values degrade to object dtype and raise IncompatibleFrequency on PeriodIndex/astype, so they can't form an index either.

Two facts that decided the shape of the fix:

• The peaks RDB has no water_yr column, so peak_dt is the only carrier of a censored peak's year. It is no longer popped.
• peak_cd is not a dependable censoring flag — 22 of 24 censored dates across six sites carry a Bd/Bm qualifier; two have peak_cd = NaN. Censoring has to be derived from peak_dt itself.

So the fix is now smaller than the one it replaces: parse with errors="coerce" and stop. No normalization helper, no dropna, peak_dt retained.

Verified live across six sites: 672 rows in, 672 out, zero dropped, zero invented dates, 24 NaT. 1136 tests pass; mypy strict, ruff, Xenon, complexipy and import-linter clean.

One knock-on worth calling out: the waterservices_peaks fixture asserted df.size == 240. size is rows × columns, so retaining peak_dt moved it to 260 with the row count unchanged at 20. Restated as df.shape == (20, 13) so the two aren't conflated again.

[ehinman]

Looks good to me.

[thodson-usgs]

Thanks @ehinman. You might not have read the relevant ADRs but the gist is that: NWIS is frozen, but bug fixes are still allowed. You may overrule that when needed, but should ask your approval before doing so.

[ehinman]

Thanks @ehinman. You might not have read the relevant ADRs but the gist is that: NWIS is frozen, but bug fixes are still allowed. You may overrule that when needed, but should ask your approval before doing so.

I appreciate making bug fixes until the services are no longer available. Unfortunately, only people who are updating drpy and still using the legacy water services will reap the benefits.