Skip to content

test(e2e): bind test nodes to port 0 instead of a random fixed port - #281

Merged
Nic-dorman merged 2 commits into
mainfrom
nic/v2-1894-ant-node-e2e-harness-binds-test-nodes-to-random-fixed-ports
Nov 5, 2026
Merged

Nic-dorman merged 2 commits into
mainfrom
nic/v2-1894-ant-node-e2e-harness-binds-test-nodes-to-random-fixed-ports

Conversation

@Nic-dorman

@Nic-dorman Nic-dorman commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Linear issue

Closes V2-1894

Every e2e test node now binds port 0. Later nodes bootstrap from the addresses the bootstrap nodes actually bound (listen_addrs()). This is the same change as ant-client#248, where the race was found and reproduced. The two retries that worked around it are removed.

The race

  • Fixed ports: each test node bound 127.0.0.1 on a port the harness picked, a random base in 20000-60000 plus the node's index. That range overlaps the OS ephemeral range (Linux 32768-60999, macOS and Windows 49152-65535).
  • The colliding socket: every node and client starts a UPnP gateway search when its transport comes up (saorsa-transport upnp.rs, enabled by default, in local mode too). The search binds 0.0.0.0:0, igd-next's default, and holds that socket for up to 2 s. A CI runner has no gateway, so it uses the full 2 s.
  • Why the bind fails: a 127.0.0.1:P bind while another socket holds 0.0.0.0:P fails with "address in use". When a search socket happened to get the port of a node not yet created, that node failed with Failed to create dual-stack network nodes: Failed to create transport.
  • Reproduced on dev2, in ant-client's harness: holding 0.0.0.0:<base+3> gave exactly CI's error string. Details are in ant-client#248, "Flake 2".

What goes

  • start_node's 4 attempts on the same port (eb049e3). They fell inside one 2 s search window, so they rarely helped. On ant-client#248, a run failed all 4.
  • start_network's bring-up on a fresh port range (627f2eb: with_fresh_ports, NETWORK_START_ATTEMPTS). Its doc says it was also for Windows runners whose reserved port ranges move between runs, where a fixed port can be refused outright. Port 0 covers that case too: the OS never assigns a reserved port or one in use.
  • With them: TestNetworkConfig::base_port, TEST_PORT_RANGE_MIN/MAX, the port-overflow check in TestNetwork::new, and TestNode::port. Nothing outside the harness read them.
  • The test a_network_whose_port_is_taken_moves_to_a_fresh_range went with the retry it covered. There is no fixed port left to hold. It's replaced by nodes_listen_on_distinct_os_assigned_ports, which checks what bootstrap now relies on: every node reports a non-zero port, and no two share one.

Also changed

  • src/node.rs unit tests: five built a node on a random port in the same range. Two retried with a fresh port; three didn't retry. They now use port 0, which NodeConfig::port documents as auto-select. One of them, a_finished_store_survives_a_file_store_that_will_not_open, expects build() to fail, so a port collision could have let it pass for the wrong reason.
  • CLAUDE.md: the port-isolation table said tests use 20000-60000. It now says OS-assigned.

Not changed

src/devnet.rs, the devnet behind ant dev start, picks fixed ports the same way (DEVNET_PORT_RANGE 20000-60000). It probably has the same race when it starts many nodes. Not in this PR.

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

  • Wire: none
  • Storage: none
  • API: none. Test code only (tests/e2e/, the #[cfg(test)] module in src/node.rs) and CLAUDE.md.

Semver impact

  • breaking
  • feature
  • fix

Test evidence

On dev2 (Linux, 4 cores), at 0da08c8 (b0df77a plus this change; dc16dda only edits CLAUDE.md):

  • cargo test --lib --features test-utils node::tests: 24 passed.
  • cargo test --test e2e --features test-utils -- --test-threads=1, as CI runs it: 121 passed, 0 failed, 3 ignored, the same as main. nodes_listen_on_distinct_os_assigned_ports passed.
  • cargo +stable clippy --all-targets --all-features -- -D warnings (rustc 1.99.1, as CI): clean. dev2's default rustc 1.95 flags six lints in files this PR doesn't touch (src/replication/pointer.rs, src/storage/chunk_store.rs); 1.99.1 doesn't.
  • cargo fmt --all -- --check: clean.

CI (dc16dda, run 42030617783): green apart from the inherited Security Audit (RUSTSEC-2026-0387, V2-1743). The e2e target passed 121, failed 0, ignored 3 on ubuntu (957 s), macOS (905 s) and Windows (1096 s). Unit tests passed on all three. No "Failed to create node" or "Failed to create transport" in any log, and CI Gate passed.

ant-client#248 made the same change to ant-client's harness and is green on ubuntu and macOS: E2E 69/69 and Merkle E2E 8/8. This PR's CI covers Windows as well.

New dependency

none

ADR

n/a

Mitigation / rollback

Revert. It touches only test code and CLAUDE.md.

🤖 Generated with Claude Code

Nic-dorman and others added 2 commits November 5, 2026 12:20
Test nodes bound 127.0.0.1 on a random port in 20000-60000, which
overlaps the OS ephemeral range. Every node and client starts a UPnP
gateway search that holds a 0.0.0.0:<ephemeral> UDP socket for up to
2 s, and a 127.0.0.1 bind to a port a wildcard socket holds fails with
"address in use", so node creation now and then failed with "Failed
to create transport". ant-client's harness had the same race
(ant-client#248, V2-1852), where it was reproduced.

The e2e harness now binds every node to port 0 and bootstraps later
nodes from the addresses the bootstrap nodes actually bound
(listen_addrs()). The OS never assigns a port that is in use or
reserved, so the two retries that covered the race go:

- start_node's 4 attempts on the same port (they fell inside one 2 s
  search window, so they rarely helped);
- start_network's bring-up on a fresh port range (with_fresh_ports),
  which also covered Windows runners' reserved port ranges.

base_port, TEST_PORT_RANGE_MIN/MAX, the port-overflow check and
TestNode::port go with them. The retry's test is replaced by one that
checks every node reports a distinct, non-zero port.

The node.rs unit tests that built a node on a random port in the same
range (two with retries, three without) now use port 0, which
NodeConfig documents as auto-select.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact base b0df77a and head dc16dda. The change is confined to test harness/unit-test code and documentation. Port-0 binding is paired with deriving bootstrap peers from each started node’s listen addresses; the added test checks assigned ports are non-zero and distinct. No actionable defect found.\n\nCoverage: 2/2 seats reviewed (sol-1 gpt-6.1-sol xhigh; glm-1 z-ai/glm-5.3); both seats agree (two model families).\n\nCI on this head: unit/E2E suites pass on Ubuntu, macOS and Windows; CI Gate and other checks pass. Security Audit fails for the inherited RUSTSEC-2026-0387 / V2-1743 issue; this is a separate merge gate, not introduced by this test-only PR. Focused local Cargo test did not complete within the review command timeout.

@Nic-dorman
Nic-dorman merged commit 34b17c6 into main Nov 5, 2026
17 of 18 checks passed
@Nic-dorman
Nic-dorman deleted the nic/v2-1894-ant-node-e2e-harness-binds-test-nodes-to-random-fixed-ports branch November 5, 2026 13:56
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 6, 2026
Brings in main up to 360d472: OS-assigned ports for e2e test nodes (WithAutonomi#281)
and devnet nodes (WithAutonomi#282), release-key and signature checks (WithAutonomi#276), the
third-party notices and audit-regression CI gates (WithAutonomi#274, WithAutonomi#275), the 10 s
DHT request timeout for test nodes (WithAutonomi#277), and the 0.24.0 release (WithAutonomi#283).

One conflict, in tests/e2e/harness.rs: main removed the network bring-up
retry on a fresh port range together with NETWORK_START_ATTEMPTS and its
`warn` import, and this branch had added two helpers next to them
(POSSESSION_PENALTY_TRUST_RATIO and parseable_single_node_proof). The
resolution keeps both helpers and takes main's removal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants