Skip to content

Chart a bare response in received order (#129) - #147

Merged
cnicholas merged 1 commit into
mainfrom
feat/received-order-xmr
Oct 18, 2026
Merged

cnicholas merged 1 commit into
mainfrom
feat/received-order-xmr

Conversation

@cnicholas

Copy link
Copy Markdown
Owner

Closes #129

Summary

  • What: formulate(response=...) with no time, factors or plan is accepted and charted in the order received.
  • Why: a column of values charted in order is the most common individuals chart there is, and the first thing a newcomer tries. The front door refused it, and the factors=[] side door classified the whole column as one replicated cell (ADS 1, one-point Xbar).
  • Scope: formulate() numbers the rows 1..N in an Observation column (the same numbers as obs_id) on the raw frame, before detection, so the study is exactly the one time='Observation' gives. The design report says so.

Contract / Invariants (must remain true)

  • Default behavior unchanged: every formulation with time, factors or plan is byte-identical
  • Residuals unaffected: no detector, residual or series-length change
  • Row/index alignment preserved: obs_id still stamped before the sort; Observation equals it row for row
  • Lane boundaries preserved: single stream, no lanes
  • Output schema compatible: only the factors=[], no-time spelling changes (chart tables gain Observation)

Behavior Changes (explicit)

  • New API: pb.formulate(df, response='weight') -> ADS 2, recommended X, X/mR with sigma from N-1 moving ranges
  • New semantics:
    • factors=[] with no time is the same study (was ADS 1 / Xbar, X plotted against the response values)
    • Design report: Time: received order (no time column given)
    • An existing Observation column with no time raises, asking for time='Observation' or a rename

Key Design Decisions

  • Time source: a copy of obs_id under a display name, not obs_id itself as time_var (that would need deduping at 21 obs_id sites). The ordinary detector classifies the 1 x N grid; no special case like the one e429205 removed.
  • NaN policy: numbered on the raw frame, so an NA response is an empty cell (ODS 5, ADS 2), as with an explicit time column
  • Run rules policy: unchanged
  • Edge cases: input frame never mutated (copy before stamping); N = 1 behaves as the explicit-time path

Risks & Mitigations

  • P0 risks: a silent difference from the explicit-time study
  • Mitigations: tests assert frame-equality of dataset, X and mR charts, statistics, ODS/ADS and series length against time='Observation' on the T100 reference data

Tests

Added / Updated

  • TestReceivedOrderFormulation (test_process_behavior.py) — equality with explicit time on PM SDS 1 (factors omitted and []), NA path on PM SDS 2, report line, order sensitivity, Xbar/S refusal + why_not, name collision, no mutation, N = 1
  • test_response_only_is_one_stream_in_received_order (test_sds_detector.py) — replaces the refusal test

Regression Focus (must fail if broken)

  • bare response == time='Observation' study on reference data
  • user DataFrame untouched

Manual Verification

  • pytest tests/ — 2553 passed, 7 skipped
  • python validation/e2e_bishop_report.py — 1547 passed, 0 failed, 4 pending (14 runs); docs/reference/validation.md unchanged
  • App fast tests against the local library (PYTHONPATH): pass; its factors=["Sequence"] paste path is unchanged
  • Visual check: both plot x-axes read "Observation"

Notes

  • Docs: README quickstart leads with the two-call example; formulation, series-length and XmR pages; quickstart notebook gains a first section; formulate/read_clipboard docstrings; CHANGELOG Added + Changed
  • Follow-ups (not in this PR): app paste path could move to the bare form; README chart image; "Available analyses" lists R2-R6 rows on time-only studies (pre-existing)

## Summary
- **What:** `formulate(response=...)` with no time, factors or plan is accepted and charted in the order received.
- **Why:** a column of values charted in order is the most common individuals chart there is, and the first thing a newcomer tries. The front door refused it, and the `factors=[]` side door classified the whole column as one replicated cell (ADS 1, one-point Xbar).
- **Scope:** formulate() numbers the rows 1..N in an `Observation` column (the same numbers as obs_id) on the raw frame, before detection, so the study is exactly the one `time='Observation'` gives. The design report says so.

## Contract / Invariants (must remain true)
- [x] **Default behavior unchanged:** every formulation with time, factors or plan is byte-identical
- [x] **Residuals unaffected:** no detector, residual or series-length change
- [x] **Row/index alignment preserved:** obs_id still stamped before the sort; Observation equals it row for row
- [x] **Lane boundaries preserved:** single stream, no lanes
- [x] **Output schema compatible:** only the `factors=[]`, no-time spelling changes (chart tables gain `Observation`)

## Behavior Changes (explicit)
- **New API:** `pb.formulate(df, response='weight')` -> ADS 2, recommended X, X/mR with sigma from N-1 moving ranges
- **New semantics:**
  - `factors=[]` with no time is the same study (was ADS 1 / Xbar, X plotted against the response values)
  - Design report: `Time: received order (no time column given)`
  - An existing `Observation` column with no time raises, asking for `time='Observation'` or a rename

## Key Design Decisions
- **Time source:** a copy of obs_id under a display name, not obs_id itself as time_var (that would need deduping at 21 obs_id sites). The ordinary detector classifies the 1 x N grid; no special case like the one e429205 removed.
- **NaN policy:** numbered on the raw frame, so an NA response is an empty cell (ODS 5, ADS 2), as with an explicit time column
- **Run rules policy:** unchanged
- **Edge cases:** input frame never mutated (copy before stamping); N = 1 behaves as the explicit-time path

## Risks & Mitigations
- **P0 risks:** a silent difference from the explicit-time study
- **Mitigations:** tests assert frame-equality of dataset, X and mR charts, statistics, ODS/ADS and series length against `time='Observation'` on the T100 reference data

## Tests
### Added / Updated
- [x] TestReceivedOrderFormulation (test_process_behavior.py) — equality with explicit time on PM SDS 1 (factors omitted and `[]`), NA path on PM SDS 2, report line, order sensitivity, Xbar/S refusal + why_not, name collision, no mutation, N = 1
- [x] test_response_only_is_one_stream_in_received_order (test_sds_detector.py) — replaces the refusal test

### Regression Focus (must fail if broken)
- [x] bare response == `time='Observation'` study on reference data
- [x] user DataFrame untouched

## Manual Verification
- [x] `pytest tests/` — 2553 passed, 7 skipped
- [x] `python validation/e2e_bishop_report.py` — 1547 passed, 0 failed, 4 pending (14 runs); docs/reference/validation.md unchanged
- [x] App fast tests against the local library (PYTHONPATH): pass; its `factors=["Sequence"]` paste path is unchanged
- [x] Visual check: both plot x-axes read "Observation"

## Notes
- **Docs:** README quickstart leads with the two-call example; formulation, series-length and XmR pages; quickstart notebook gains a first section; formulate/read_clipboard docstrings; CHANGELOG Added + Changed
- **Follow-ups (not in this PR):** app paste path could move to the bare form; README chart image; "Available analyses" lists R2-R6 rows on time-only studies (pre-existing)
@cnicholas
cnicholas merged commit 02e4bf3 into main Oct 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose the received-order X/mR chart: formulate(response=...) with no time or factors

1 participant