Skip to content

Host-capture Re_size in viscous and non-Newtonian device code - #2130

Open
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:wave1/amdflang-re-size-stale
Open

sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:wave1/amdflang-re-size-stale

Conversation

@sbryngelson

Copy link
Copy Markdown
Member

What

#1588 worked around amdflang reading the declare-target Re_size stale
(as 0) across translation units by copying it into firstprivate locals
in the Riemann solvers. Device code in other modules still read the
module variable directly, so on AMD flang it could silently drop
viscosity the same way:

  • s_compute_axis_inv_re (m_viscous), called from the four cylindrical
    axis stress kernels
  • s_compute_mixture_inv_re (m_hb_function), called from the axis
    routine and from the cylindrical and Cartesian viscous source-flux
    kernels in m_riemann_state

Apply the same pattern: each enclosing kernel takes Re_size_loc1/2 as
firstprivate host copies and passes them down, and the device routines
use merge(Re_size_loc1, Re_size_loc2, i == 1) in place of Re_size(i),
as s_compute_interface_reynolds already does.

Values are identical to Re_size, so results do not change on CPU or
NVIDIA: viscous, non-Newtonian and cylindrical tests pass unchanged and
the NVHPC OpenACC build compiles. The full suite also passes on AMD GPUs (MI300A, MI250X).

Verification

Testing. Built with GPU offload (OpenMP, AMD) and ran the full test suite with this fix and seven other independent fixes applied together: 729 tests in 4 shards, 0 failures (two shards on MI300A, two on MI250X).

Prepared with Claude Code (AI-assisted); I reviewed the change and ran the tests.

MFlowCode#1588 worked around amdflang reading the declare-target Re_size stale
(as 0) across translation units by copying it into firstprivate locals
in the Riemann solvers. Device code in other modules still read the
module variable directly, so on AMD flang it could silently drop
viscosity the same way:

- s_compute_axis_inv_re (m_viscous), called from the four cylindrical
  axis stress kernels
- s_compute_mixture_inv_re (m_hb_function), called from the axis
  routine and from the cylindrical and Cartesian viscous source-flux
  kernels in m_riemann_state

Apply the same pattern: each enclosing kernel takes Re_size_loc1/2 as
firstprivate host copies and passes them down, and the device routines
use merge(Re_size_loc1, Re_size_loc2, i == 1) in place of Re_size(i),
as s_compute_interface_reynolds already does.

Values are identical to Re_size, so results do not change on CPU or
NVIDIA: viscous, non-Newtonian and cylindrical tests pass unchanged and
the NVHPC OpenACC build compiles. Not tested on AMD hardware.
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Claude Code Review

Head SHA: 437af6f

Files changed:

  • 3
  • src/simulation/m_hb_function.fpp
  • src/simulation/m_riemann_state.fpp
  • src/simulation/m_viscous.fpp

Findings:
No issues found. Checked m_hb_function.fpp, m_riemann_state.fpp and m_viscous.fpp. The new Re_size_loc1/Re_size_loc2 arguments are passed at every call site of s_compute_mixture_inv_re and s_compute_axis_inv_re: two in m_riemann_state, four in m_viscous, and one inside s_compute_axis_inv_re. Each is set on the host before the parallel loop and passed by firstprivate. merge(loc1, loc2, i == 1) matches the old Re_size(i) for i = 1, 2.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

Lines of Code

File Lines Diff
src/simulation/m_viscous.fpp 1028 +7
src/simulation/m_riemann_state.fpp 1192 +4
src/simulation/m_hb_function.fpp 62 +1
Directory Lines Diff
simulation 28592 +12
total 47642 +12

@sbryngelson
sbryngelson marked this pull request as ready for review November 3, 2026 14:17
@sbryngelson
sbryngelson requested a review from a team as a code owner November 3, 2026 14:17
@sbryngelson
sbryngelson requested review from wilfonba and removed request for a team November 3, 2026 14:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant