Skip to content

Count binary post_process variables as they are written - #2128

Open
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:wave1/post-binary-dbvars
Open

sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:wave1/post-binary-dbvars

Conversation

@sbryngelson

Copy link
Copy Markdown
Member

What

The Binary format header stores the number of variable records, and the
viz reader loops over exactly that many. s_initialize_data_output_module
predicted it with a hand-maintained copy of s_save_data's output
conditions that had drifted: it counted hypoelastic stresses and MHD
fields even when prim_vars_wrt = F (s_save_data writes them only with
it), and missed chemistry Y and T, alpha_rho_e, T_wrt, ib_markers, qm,
liutex, color_function, the bubbles_euler fields and voidFraction. An
overcount made the reader hit EOF; an undercount made it silently drop
the trailing variables.

Drop the prediction. Write a zero placeholder in the header, increment
dbvars in s_write_variable_to_formatted_database_file, and on close
reopen each file as a stream and write the final count over the
placeholder, after checking that the header record has the expected
length.

Verified with format = 2 runs read back through toolchain/mfc/viz/reader.py,
comparing the header count with the records in each file. Every file
now matches: 2D hypoelastic with prim_vars_wrt = T (17) and = F (2,
master wrote 5 and the reader failed with EOFError), 2D chemistry (25,
master wrote 14 so the reader dropped 11 species/T/derived fields), 1D
including the root file, and 2-rank 1D and 2D runs built with nvfortran.

Verification

Testing. Built with GPU offload (OpenMP, AMD) and ran the full test suite with this fix and seven other independent fixes applied together: 729 tests in 4 shards, 0 failures (two shards on MI300A, two on MI250X).

Prepared with Claude Code (AI-assisted); I reviewed the change and ran the tests.

The Binary format header stores the number of variable records, and the
viz reader loops over exactly that many. s_initialize_data_output_module
predicted it with a hand-maintained copy of s_save_data's output
conditions that had drifted: it counted hypoelastic stresses and MHD
fields even when prim_vars_wrt = F (s_save_data writes them only with
it), and missed chemistry Y and T, alpha_rho_e, T_wrt, ib_markers, qm,
liutex, color_function, the bubbles_euler fields and voidFraction. An
overcount made the reader hit EOF; an undercount made it silently drop
the trailing variables.

Drop the prediction. Write a zero placeholder in the header, increment
dbvars in s_write_variable_to_formatted_database_file, and on close
reopen each file as a stream and write the final count over the
placeholder, after checking that the header record has the expected
length.

Verified with format = 2 runs read back through toolchain/mfc/viz/reader.py,
comparing the header count with the records in each file. Every file
now matches: 2D hypoelastic with prim_vars_wrt = T (17) and = F (2,
master wrote 5 and the reader failed with EOFError), 2D chemistry (25,
master wrote 14 so the reader dropped 11 species/T/derived fields), 1D
including the root file, and 2-rank 1D and 2D runs built with nvfortran.
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Claude Code Review

Head SHA: c3e5df9

Files changed:

  • 1
  • src/post_process/m_data_output.fpp

Findings:
No issues found. Checked in step 4: src/post_process/m_data_output.fpp. The slave and master (n == 0) header records are written as m, n, p, dbvars with default integers behind a 4-byte record marker, which matches the backfill offset 1 + 4 + 3*int_bytes. dbvars is incremented once per variable write, covering both the slave and root files. The header check aborts instead of corrupting the file if the marker size assumption (for example, integer-8 builds) does not hold.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

Lines of Code

File Lines Diff
src/post_process/m_data_output.fpp 1205 -52
Directory Lines Diff
post_process 3432 -52
total 47578 -52

@sbryngelson
sbryngelson marked this pull request as ready for review November 3, 2026 14:17
@sbryngelson
sbryngelson requested a review from a team as a code owner November 3, 2026 14:17
@sbryngelson
sbryngelson requested review from wilfonba and removed request for a team November 3, 2026 14:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant