Twice in two days (PR #526 first run, PR #527 first run) the test job died at the import smoke test, before any underworld3 code loaded:
ib_iface.c:1316 UCX ERROR mana_0: iface ... failed to create UD QP ... Operation not supported
ucp_worker.c:1415 UCX ERROR uct_iface_open(ud_verbs/mana_0:1) failed: Input/output error
MPIDI_UCX_init_worker(86): ucx function returned with failed status
UCX is trying an unsupported verbs transport on the runner's Azure MANA network interface; whether a run flakes depends on which VM type the job lands on. Rerun on a fresh runner passes (measured, #526).
Suggested immunisation: pin the transport in the workflow env for the test job, e.g.
env:
UCX_TLS: tcp,self,sm
(or UCX_NET_DEVICES=lo for the serial import smoke alone). Filed so the reruns stop being folklore.
Underworld development team with AI support from Claude Code
Twice in two days (PR #526 first run, PR #527 first run) the test job died at the import smoke test, before any underworld3 code loaded:
UCX is trying an unsupported verbs transport on the runner's Azure MANA network interface; whether a run flakes depends on which VM type the job lands on. Rerun on a fresh runner passes (measured, #526).
Suggested immunisation: pin the transport in the workflow env for the test job, e.g.
(or
UCX_NET_DEVICES=lofor the serial import smoke alone). Filed so the reruns stop being folklore.Underworld development team with AI support from Claude Code