Skip to content

Towards Unified Sequence Representation VI: Add MLP_V2 and LogisticRegression_V2 for packed records - #1355

Open
Logiquo wants to merge 2 commits into
masterfrom
feat/seq-v2-6
Open

Logiquo wants to merge 2 commits into
masterfrom
feat/seq-v2-6

Conversation

@Logiquo

@Logiquo Logiquo commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Sixth PR in the stack toward a unified sequence representation, on top of #1354. It adds two pooling baselines over sequence_v2 fields and static tensors, built on SequenceEmbeddingModel.

Changes

  • LogisticRegression_V2 embeds each level and mean-pools the records of every group bottom-up (an empty group gives zeros). It concatenates the field summaries with the raw static vectors and applies one linear layer.
  • MLP_V2 adds an MLP at every level and on each static field before pooling. hidden_dim, num_layers and dropout are set per field.
  • Per-step labels: each label record uses the mean of the top-level records at or before its time (a prefix mean), so it never sees later records. Fields need a series top level.
  • Both are exported from pyhealth.models and listed in the model guide, with API pages.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Test Coverage

Total: 75.23% (21532 / 28623 lines)

Modified file Coverage Lines
pyhealth/models/__init__.py 100.00% 56 / 56
pyhealth/models/logistic_regression.py 86.00% 129 / 150
pyhealth/models/mlp.py 76.67% 115 / 150

Workflow run · 2e08ba8

@Logiquo
Logiquo removed this pull request from stack #1339 November 1, 2026 09:45
@Logiquo
Logiquo added this pull request to stack #1359 November 1, 2026 09:45
@Logiquo Logiquo changed the title Towards Unified Sequence Representation VI: Add Unified_MLP and Unified_LogisticRegression for packed records Towards Unified Sequence Representation VI: Add MLP_V2 and LogisticRegression_V2 for packed records Nov 1, 2026
@Logiquo
Logiquo force-pushed the feat/seq-v2-6 branch 2 times, most recently from 67fcb0f to 0b8f8e4 Compare November 1, 2026 21:52
@Logiquo
Logiquo force-pushed the feat/seq-v2-6 branch 2 times, most recently from 8c3eff3 to 4842658 Compare November 1, 2026 23:59
Base automatically changed from feat/seq-v2-5 to master November 2, 2026 00:12
Two pooling baselines over sequence_v2 fields and static tensors, built
on SequenceEmbeddingModel.

- Unified_LogisticRegression embeds each level, mean-pools the records of
  every group bottom-up (an empty group gives zeros), concatenates the
  field summaries with the raw static vectors, and applies one linear
  layer.
- Unified_MLP adds an MLP at every level and on each static field before
  pooling; hidden_dim, num_layers and dropout are set per field.
- Per-step labels: each label record uses the mean of the top-level
  records at or before its time (a prefix mean), so it never sees later
  records. Fields need a series top level.
@Logiquo Logiquo added the status: automatic The PR will go through automatic review process. label Nov 2, 2026
@Logiquo

Logiquo commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 2e08ba8

torch.cat(rows, dim=-1)
)
offsets = embedded[field][level]["offsets"]
if self.per_step and level == processor.depth:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] Per-step and bottom-up logic duplicates RNN_V2 instead of sharing it

The per-step branch (label-time padding, searchsorted over padded times, prefix indexing) and the bottom-up level loop repeat RNN_V2._at_labels and _FieldRNN._bottom_up in pyhealth/models/rnn.py. The init field classification and per-step series check also mirror RNN_V2. Only the per-level reducer differs (mean here, RNN there). Fix: move the shared level walk and label-time readout into helpers in rnn.py (or a common module) parameterized by the per-level encoder, and have LogisticRegression_V2 and RNN_V2 both use them. This blocks merge because it is a copy of existing logic, not a reuse.

.unsqueeze(1),
right=True,
)[:, 0]
prefix = torch.cat([x.new_zeros(1, x.shape[1]), x.cumsum(0)])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] Prefix mean subtracts batch-wide float32 cumsums

prefix is a cumsum over every row in the batch, and each sample's mean is read as a difference of two global prefix values. Rounding error therefore grows with total batch size and makes each sample's output depend slightly on the other samples. The comment at line 348 says the result is independent. Tests use small batches, so this is not caught. Fix: take per-group prefix sums (for example, cumsum within each group via the padded layout already built above, or index a padded cumsum), so each sample's result depends only on its own rows.

Comment thread tests/core/test_mlp_v2.py
@@ -0,0 +1,587 @@
import copy

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] LogisticRegression_V2 is tested only inside the MLP_V2 test file

The file is named test_mlp_v2.py but also covers LogisticRegression_V2 throughout. Renaming it to a neutral name, or splitting out tests/core/test_logistic_regression_v2.py, would make the coverage easier to find. Not a merge blocker.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review

suggestion: need change
decision: need change

Thank you for this contribution. MLP_V2 and LogisticRegression_V2 add pooling baselines over sequence_v2 fields and static tensors. They include causal per-step prefix means, exports, API docs, and a thorough test file. The code looks correct as far as I can tell, and the tests cover hierarchy means, empty groups, masks, pretrained tokens, causality and Trainer runs.

To get this merged:

  • Remove the duplicated logic. LogisticRegression_V2 copies RNN_V2's per-step label indexing and its bottom-up level walk. It also copies RNN_V2's constructor validation. The encoder is different (mean pooling instead of an RNN), but the shared parts should be factored into a common helper so the two models don't drift apart. Reusing _groups and _pool is a good start. Please extend that to the label-time indexing and level traversal.
  • Raise the test coverage of pyhealth/models/mlp.py. It is at 76.67%, below the 80% bar for modified files. Add tests that cover the uncovered lines. Most of those lines are probably in the existing MLP class. A few tests for the paths you touched, or for the neighboring code, should be enough to reach 80%.

Whole-repo coverage is 75.23%, above the 60% bar. logistic_regression.py is at 86% and __init__.py at 100%.

Optional suggestions:

  • The prefix mean subtracts batch-wide float32 cumulative sums. That adds small rounding error that depends on batch composition. Computing the sums per sample, or in float64, would avoid this.
  • Consider adding an example script for each model.
  • Consider moving the LogisticRegression_V2 tests into their own test file, since they currently live in tests/core/test_mlp_v2.py.

Reviewed commit 2e08ba8

@Logiquo
Logiquo marked this pull request as ready for review November 2, 2026 04:38

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: automatic The PR will go through automatic review process.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant