Skip to content

Towards Unified Sequence Representation II: Add SequenceEmbeddingModel and Unified_RNN for hierarchical packed records - #1347

Draft
Logiquo wants to merge 10 commits into
feat/seq-v2from
feat/seq-v2-2
Draft

Logiquo wants to merge 10 commits into
feat/seq-v2from
feat/seq-v2-2

Conversation

@Logiquo

@Logiquo Logiquo commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

This is the second PR in the stack toward unified sequential features, building on #1343. It adds SequenceEmbeddingModel and three prediction baselines for packed SequenceProcessorV2 records: Unified_RNN, Unified_MLP, and Unified_LogisticRegression. It also migrates YAIB tasks and per-step labels to V2 and provides MIMIC-IV YAIB training examples.

Sequence embeddings and models

  • SequenceEmbeddingModel embeds multiple token namespaces and projects numeric values at each level, preserving offsets and timestamps. It supports spatial value layouts, optional missing-indicator concatenation, and pretrained token embeddings. Empty numeric batches now reshape correctly, including when spatial axes are retained.
  • All three models preserve hierarchical groups and encode bottom-up. Each parent combines its own embedding with its child summary. Empty groups receive zero summaries; a stay without lab orders still participates at the stay level.
  • Unified_RNN mean-pools set levels and runs an RNN over sequence/series groups, retaining the last output. Optional time-gap features capture irregular timing. There are no hierarchy or set_pooling selectors.
  • Unified_MLP applies independent MLPs to records at every hierarchy level before mean pooling. Static tensors are flattened per sample and passed through their own MLPs. Hidden widths, layer counts, and dropout can be configured per field.
  • Unified_LogisticRegression mean-pools record embeddings bottom-up and concatenates raw flattened static vectors before a linear head. It remains a linear head over learned embeddings, rather than classical regression on raw features.
  • The MLP and logistic regression models flatten/project spatial values within each record and do not model spatial locality. Their means weight parents equally and do not model temporal order. Existing model classes remain available.

Per-step labels and YAIB integration

  • TimeseriesBinaryLabelProcessor and TimeseriesRegressionLabelProcessor provide packed values, times, and batch offsets through SequenceProcessorV2. BaseModel and Trainer support packed per-step losses and inference.
  • At label time t, Unified_RNN uses the last available top-level state. Unified_MLP and Unified_LogisticRegression use the cumulative mean of top-level record representations with timestamps <= t. Before the first record, field summaries are zero. Static representations accompany each label record.
  • Per-step inputs require a series top level; resampled record i is available at i * step. A top-level timestamp must indicate availability of the complete record, including children. Bidirectional RNNs are rejected for per-step prediction.
  • Migrates YAIB AKI, kidney function, length of stay, mortality, and sepsis tasks to sequence_v2 features and per-step labels.
  • examples/yaib_mimic4_multimodal_rnn2.py trains Unified_RNN across the five tasks.
  • examples/yaib_mimic4_unified_mlp.py follows the same split, preflight, training, checkpoint, and reporting flow. Select --model mlp or --model logistic_regression.

Validation

  • Passed 64 tests across the new baselines, sequence embeddings, Unified_RNN, per-step labels, time-series modes, and legacy MLP/logistic regression tests. Coverage includes nested means, empty groups, static-only inputs, missing indicators, pretrained embeddings, CPU/CUDA comparisons, and Trainer integration.
  • Passed an additional focused regression test for all-empty numeric embeddings with flattened and retained spatial axes.
  • Existing YAIB integration validation passed 65 tests across the RNN, time-series, per-step-label, and YAIB test modules.
  • Pyright passed for the new example. CLI help and CPU --check runs passed for both model selections. Full MIMIC-IV training was not run for the new example.
  • git diff --check passed.

Following PRs

Update Transformer, CNN/TCN, and EHRMamba to consume sequence_v2 fields, and add a packed_to_padded adapter for dense sequence layers.

@Logiquo
Logiquo added this pull request to stack #1339 October 28, 2026 22:43
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Test Coverage

Total: 74.75% (21241 / 28416 lines)

Modified file Coverage Lines
pyhealth/models/__init__.py 100.00% 56 / 56
pyhealth/models/base_model.py 92.81% 129 / 139
pyhealth/models/logistic_regression.py 84.00% 126 / 150
pyhealth/models/mlp.py 76.67% 115 / 150
pyhealth/models/rnn.py 92.41% 341 / 369
pyhealth/models/sequence_embedding.py 98.65% 146 / 148
pyhealth/processors/timeseries_processor.py 87.38% 90 / 103
pyhealth/tasks/yaib_mimic4.py 93.06% 523 / 562
pyhealth/trainer.py 72.69% 197 / 271

Workflow run · 7e4e75a

… fields

SequenceEmbeddingModel embeds the packed records of sequence_v2 fields:
per level, one nn.Embedding per token namespace (<pad> embeds to zeros)
and a projection of the values, concatenated. Levels are kept apart with
their offsets and times, or flattened per field into one sequence of
level-0 records carrying their ancestors' embeddings. Spatial value axes
can be kept (token embeddings broadcast over them), masks ignored or
concatenated, and token namespaces initialized from pretrained vectors.

MultimodalRNN2 encodes each sequence_v2 field into one summary per
sample and uses TensorProcessor fields as statics (directly or
projected); other processors are rejected. Per field, the hierarchy is
"flat" (one RNN over the flattened records), "bottom_up" (level by
level, sets pooled, RNNs per group) or "interleaved" (each level's state
passed down to its children as context, step by step). Empty groups get
zero summaries, time gaps can be added as a feature, and per-step labels
use each field's top-level state after its last record before the step.
Every option takes one value or a {field: value} dict.
- TimeseriesBinaryLabelProcessor / TimeseriesRegressionLabelProcessor are
  restricted SequenceProcessorV2s: one series level of scalar labels,
  {"time", "values"} with strictly increasing times and no NaN. Batches
  are packed (values, time, batch offsets); there is no grid or mask.
- Per-step prediction is V2 only: MultimodalRNN2 predicts once per label
  record, from each field's state at or before the label's time; the
  timeseries losses average over label records, and Trainer.inference
  splits them per sample. RNN and MultimodalRNN reject per-step labels.
- BaseModel resolves a processor's mode from its most specific registered
  class, so the labels do not resolve to "sequence_v2".
- The YAIB tasks give "dynamic" as a sequence_v2 series in hours since ICU
  admission, preprocessed as before (forward fill, the training mean
  before the first value, z-score, missing-value mask), and the per-hour
  labels on the same hours.
Update the SequenceProcessorV2 RNN model class from MultimodalRNN2 to Unified_RNN and update model exports and test suites accordingly.
Add an end-to-end example script (examples/yaib_mimic4_multimodal_rnn2.py) to train and evaluate Unified_RNN across the five MIMIC-IV YAIB tasks (mortality, kidney function, length of stay, AKI, and sepsis), with preflight validation and metric logging.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant