Repository navigation
Conversation
A tree stop signals every PID that `collect_process_tree` returns for the root: `terminate_verified` walks once to snapshot the tree it will wait on and again to pick the signal targets. Since the managed-service stop in `rocmd` was routed through `terminate_verified` (#499), that walk decides the kill set for both `rocm services stop` and `rocmd`. A wrong walk either leaves a GPU worker running or signals a process that is not the service's. The walk is now a separate `walk_process_tree` over a parent-to-children table, so it can run against synthetic tables instead of the live `/proc`; `collect_process_tree` still builds the table from `/proc` and hands it over, with no change in behaviour. The properties generate tables of one to six PIDs over a six-PID alphabet, so shared parents, cycles and self-parents come up often. A consistent process tree has no cycle, but the table is built from one `stat` read per PID at different moments, so a cycle from a PID reused mid-scan is rare but cannot be ruled out, and the walk must terminate on it. The properties check the walk against an independent breadth-first oracle (exact reachable set, root first, no PID listed twice) and that every member after the root comes after its parent, as the walk now documents. Each walk runs under a time bound, so a walk that spins on a cycle fails with its input instead of hanging the job. A companion test asserts the generator keeps reaching each of those shapes, including a root that lies on a cycle of two or more PIDs, so the suite cannot go vacuous. 4096 cases by default; `PROPTEST_CASES` overrides that. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. — N/A, test-only.docs/architecture.md. — N/A.Why
A tree stop signals every PID that
collect_process_treereturns for the service's root PID:terminate_verifiedwalks once to snapshot the tree it will wait on and again to pick the signal targets. Since #499 routedrocmd's managed-service stop throughterminate_verified, that one walk decides the kill set for bothrocm services stopandrocmd. A wrong walk either leaves a GPU worker running or signals a process that is not the service's, and until now the walk had no test of its own: the existing tree tests only use small real process trees, where cycles and other odd tables never occur.What changes
collect_process_treeintowalk_process_tree, which takes an already-built parent-to-children table.collect_process_treestill reads/procand hands the table over. The walk's doc now states its ordering: root first, every other PID after its parent, no PID twice.crates/rocm-core/src/process_tree_properties.rs, Unix-only, mounted as a child module ofprocess.rslikeexamine_driver_proptests.rs):ppid == 0shape occurs), with unconstrained parents, so shared parents, cycles and self-parents come up in a large share of cases. A consistent process tree has no cycle, but the table is built from onestatread per PID at different moments, so a cycle from a PID reused mid-scan is rare but cannot be ruled out, and the walk must terminate on it. A self-parent cannot come from/procand is included as the one-node case of the same cycle.walk_matches_reachability_oracle: the walk's PID set equals an independent breadth-first reachability oracle; the root leads; no PID is listed twice.parents_are_listed_before_their_children: every member after the root comes after its parent, as the walk documents.generator_reaches_the_interesting_shapesguards against a vacuous suite. Over 1024 deterministic draws the generator reaches a shared parent in 567, a self-parent in 443, a cycle of two or more PIDs in 238, and a root lying on such a cycle (the shape where the walk'sseencheck matters) in 116; the floors are about half of each.therock_version_proptests.rsuses for cheap pure functions;PROPTEST_CASESoverrides it, as inexamine_driver_proptests.rs.No new dependencies:
proptestis already a rocm-core dev-dependency, soCargo.lock,MANIFEST.mdand the third-party notices are untouched.Can the properties fail?
Each mutant was applied to the walk (or the generator), the suite run, and the original restored:
seencheck (a PID signalled twice)root + 1(a stranger adopted)seencheckppid >= pid(no cycles)cycle of two or more PIDs: 0/1024, floor 120The walk itself passes every property on
main; no defect was found in it.Scope
/procside of the walk (read_parent_pidand the table assembly incollect_process_tree); the test'schildren_ofmirrors that table builder rather than exercising it. A parse property forread_parent_pid(comm names containing)or spaces) would be a natural follow-up.rocmd's formerps-based descendant walk is not tested here because fix(rocmd): verify process identity before stopping a managed service #499 removed it; fix(rocmd): verify process identity before stopping a managed service #499 also ships the tests that a recorded PID whose start time no longer matches is refused, and that its subtree survives (stop_managed_service_refuses_a_pid_whose_recorded_identity_no_longer_matches,stop_managed_service_leaves_the_subtree_of_a_refuted_pid_running).AGENTS.md §3
Test-only apart from a private refactor: no CLI output, exit code, file layout or runtime selection changes, so no Gherkin scenario is needed.
Verification
On Linux (WSL2):
cargo test -p e2e-cucumber -p rocm -p rocm-core -p rocm-dash-daemon -p rocm-engine-aim -p rocm-engine-lemonade -p rocm-engine-protocol -p rocm-engine-vllm -p rocmd -p xtask(the setcargo xtask affectedselects): exit 0, 3501 passed.cargo clippy --locked --workspace --all-targets -- -D warnings: exit 0.cargo fmt --all -- --check: exit 0.prek run --from-ref origin/main --to-ref HEAD: all hooks pass except the hawkeye license-header hook, which could not run here because the locally installed hawkeye is older than the versionlicenserc.tomltargets; the new file carries the same SPDX header as its siblings, and theLicense header check (hawkeye)CI job uses the pinned version.Not verified locally: Windows. The new code is
cfg(not(windows))throughout, so a Windows build does not compile it.