Tool: evaluate layer-wise numerical-error propagation#525

Open

jlamypoirier wants to merge 30 commits into

mainfrom

jlp_evaluate_precision

Commits on May 27, 2026

Add tool to evaluate layer-wise numerical-error propagation

jlamypoirier
and
claude
committed
Collapse to a single config; require a checkpoint

jlamypoirier
and
claude
committed
Expose `model:` alongside `pretrained:` in the tool config

jlamypoirier
and
claude
committed
Inherit PretrainedGPTModelConfig; use Config update mechanism

jlamypoirier
and
claude
committed
Expand HF metadata allowlist for newer transformers configs

jlamypoirier
and
claude
committed
Reshape console table for readability

jlamypoirier
and
claude
committed
Merge tensor+kind, fix decimal precision in console table

jlamypoirier
and
claude
committed
Switch back to fixed-decimal formatting in the table

jlamypoirier
and
claude
committed
Wipe per-variant experiment dir before each run

jlamypoirier
and
claude
committed

Commits on May 28, 2026

Support pre-generated memmap dataset; misc table-format polish

jlamypoirier
and
claude
committed
Print per-variant summary at the end of the run

jlamypoirier
and
claude
committed
Reshape end-of-run summary: variants × aggregations, relative only

jlamypoirier
and
claude
committed
Clarify intermediate aggregation in summary header

jlamypoirier
and
claude
committed
Split summary across fw/bw rows; one extra precision digit

jlamypoirier
and
claude
committed
Two-row column header in summary; chronological column order

jlamypoirier
and
claude
committed
Add fp32_lm_head flag for vLLM precision parity

jlamypoirier
and
claude
committed
Extract layer-name labels for summary first/last columns

jlamypoirier
and
claude
committed
Add `debug_hidden_states_log` to capture named tensors via output_hidden_states

jlamypoirier
and
claude
committed
Capture logit gradients; expose them in the summary

jlamypoirier
and
claude
committed
Place logits after head in bw summary; widen format for sub-percent values

jlamypoirier
and
claude
committed
Pick per-column decimals to guarantee ≥2 sig figs

jlamypoirier
and
claude
committed
Tighten summary table spacing

jlamypoirier
and
claude
committed
Support HF Hub model ids in pretrained.path

jlamypoirier
and
claude
committed
Add example precision-evaluation configs

jlamypoirier
and
claude
committed
Drop bf16_no_fp32_gradients variant from example configs

jlamypoirier
and
claude
committed
Add weight gradients to per-variant report tables

jlamypoirier
and
claude
committed
Separate fw/bw/grad rows in per-variant tables

jlamypoirier
and
claude
committed
Split summary into three tables (fw, bw, grad)

jlamypoirier
and
claude
committed
Split grad summary by parameter category

jlamypoirier
and
claude
committed
Per-tensor sample-density overrides in TensorLogsConfig

jlamypoirier
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tool: evaluate layer-wise numerical-error propagation#525

Tool: evaluate layer-wise numerical-error propagation#525
jlamypoirier wants to merge 30 commits into
mainfrom
jlp_evaluate_precision

Commits on May 27, 2026

Add tool to evaluate layer-wise numerical-error propagation

Collapse to a single config; require a checkpoint

Expose `model:` alongside `pretrained:` in the tool config

Inherit PretrainedGPTModelConfig; use Config update mechanism

Expand HF metadata allowlist for newer transformers configs

Reshape console table for readability

Merge tensor+kind, fix decimal precision in console table

Switch back to fixed-decimal formatting in the table

Wipe per-variant experiment dir before each run

Commits on May 28, 2026

Support pre-generated memmap dataset; misc table-format polish

Print per-variant summary at the end of the run

Reshape end-of-run summary: variants × aggregations, relative only

Clarify intermediate aggregation in summary header

Split summary across fw/bw rows; one extra precision digit

Two-row column header in summary; chronological column order

Add fp32_lm_head flag for vLLM precision parity

Extract layer-name labels for summary first/last columns

Add `debug_hidden_states_log` to capture named tensors via output_hidden_states

Capture logit gradients; expose them in the summary

Place logits after head in bw summary; widen format for sub-percent values

Pick per-column decimals to guarantee ≥2 sig figs

Tighten summary table spacing

Support HF Hub model ids in pretrained.path

Add example precision-evaluation configs

Drop bf16_no_fp32_gradients variant from example configs

Add weight gradients to per-variant report tables

Separate fw/bw/grad rows in per-variant tables

Split summary into three tables (fw, bw, grad)

Split grad summary by parameter category

Per-tensor sample-density overrides in TensorLogsConfig