Skip to content

fix(devnet): bind devnet nodes and loopback browser listeners to OS-assigned ports - #282

Merged
Nic-dorman merged 3 commits into
mainfrom
nic/v2-1896-ant-node-devnet-nodes-and-loopback-webrtc-direct-listeners
Nov 5, 2026
Merged

Nic-dorman merged 3 commits into
mainfrom
nic/v2-1896-ant-node-devnet-nodes-and-loopback-webrtc-direct-listeners

Conversation

@Nic-dorman

@Nic-dorman Nic-dorman commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Round 2 (b17962e): fixes both of Hermes's findings. Ports are now either all OS-assigned or all fixed in disjoint ranges.

  1. Mixed fixed and OS-assigned ports. A loopback browser listener on port 0 could take a later node's fixed native port. The mirror case is fixed too: nodes on port 0 with fixed browser ports (--webrtc-direct-base-port or --host). Any fixed port, native or browser, now gives the rest a fixed range clear of it.
  2. Oversized devnet. allocate_disjoint_range checks the range has room first, and returns DevnetError::Config instead of panicking in gen_range.

New tests: devnet_ports_are_all_os_assigned_or_all_fixed_and_disjoint (each combination) and a_devnet_too_large_for_the_port_range_is_rejected. Both fail against 648444c: the first on its first mixed case, the second panicking inside rand.

On Windows: devnet unit tests 27/27; clippy finds nothing in the changed files; the no-default-features check is clean.

Review range 648444c..288e8cc (288e8cc is an empty commit: GitHub dropped the push event for b17962e, so CI never started).

Linear issue

Closes V2-1896

Devnet nodes, and a loopback devnet's WebRTC Direct listeners, now bind ports the OS assigns. This is the same fix #281 made in the e2e harness, here for src/devnet.rs. That's the devnet behind ant-devnet, ant-sdk's ant dev start and ant-client's LocalDevnet.

The race

  • Fixed ports: each devnet node bound base_port + index in 20000-60000 (devnet.rs:1030, bound at :1136), and base_port == 0 meant "pick a random range". Nodes are dual-stack by default, so each held two fixed sockets, and one failed bind failed the whole start().
  • The colliding socket: that range overlaps the OS ephemeral range. Every node's UPnP gateway search holds a 0.0.0.0:<ephemeral> UDP socket for up to 2 s, so a later fixed-port bind can fail with "address in use". saorsa-core#185 (V2-1895) stops that search for loopback binds.
  • Windows reserved ranges: a fixed port can also land in a range Windows has reserved, and binding it is refused outright.
  • CI evidence: run 38251602766 (10-12, Windows) failed webrtc_direct_devnet::encrypted_hello_and_manifest_never_disclose_verification_rpc with Startup("Failed to create node 0: … Failed to create transport"). That's the devnet's 1 hit in the last 200 ant-node CI runs; the other 6 were in the e2e harness and node.rs tests, fixed by test(e2e): bind test nodes to port 0 instead of a random fixed port #281.
  • Analysis: Projects/ci-reports/devnet-port-race-2026-11-05.md.

What changes

  • Native QUIC: base_port == 0, now the DevnetConfig::default(), means OS-assigned.
    • Each node binds port 0.
    • Once it runs, the devnet records the node's IPv4 bound port in DevnetNode.port. A dual-stack node on port 0 gets a different port for each stack, and bootstrap addresses are IPv4.
    • bootstrap_addrs(), start_regular_nodes and the manifest keep reading DevnetNode.port, so the API is unchanged and ant-client's LocalDevnet compiles as is.
    • An explicit base_port (--base-port N) still gives fixed ports with the same range checks.
  • WebRTC Direct, loopback devnet: webrtc_direct_base_port == 0 now gives each listener 127.0.0.1:0 with no advertised_addr.
    • web_rtc::advertised_addr already reports the bound address for a specified bind IP (source Bind), and the browser endpoints come from the real listener.
    • A loopback listener never asks for a UPnP mapping (maps_port_with_upnp needs an unspecified bind).
  • WebRTC Direct, LAN devnet (--host): it must advertise its port up front, so 0 still allocates a fixed range (allocate_webrtc_direct_range, the previous logic moved into a function). Using an OS-assigned port there is V2-1897.
  • ant-devnet CLI: the help text for --base-port and --webrtc-direct-base-port describes the new default.

Tests

  • a_node_whose_startup_fails_part_way_is_stopped_by_shutdown used to occupy webrtc_direct_base_port before start(). There's no fixed port to occupy now, so it holds a free port in the devnet range itself and passes it as an explicit WebRTC Direct base. Node 0's listener still fails as intended.
  • New: a_default_devnet_binds_os_assigned_ports checks:
    • base_port stays 0;
    • the nodes get distinct non-zero ports;
    • the bootstrap address carries the bootstrap node's bound port.

Compatibility for consumers

  • Manifest: DevnetManifest.base_port stays; it's a required u16 for ant-cli and ant-ffi. It's 0 for a default devnet. Consumers only display it, and ant-sdk's ant dev start print is V2-1898.
  • Browser network_id: becomes local-devnet-0-<created_at>, still unique through the timestamp.

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Proposed T0: this is the local devnet tooling (src/devnet.rs, ant-devnet). Production nodes already default to port 0 (--port, cli.rs:21-23) and are not touched.

Compatibility

  • Wire: none
  • Storage: none
  • API: no signatures change. DevnetConfig::default().base_port is now 0, and base_port == 0 means OS-assigned instead of a random range. The manifest's base_port is 0 for a default devnet.

Semver impact

  • breaking
  • feature
  • fix

Test evidence

On Windows (rustc 1.97.1), at 648444c:

  • cargo test --lib --features test-utils devnet::: 25 passed, including a_default_devnet_binds_os_assigned_ports and the updated startup-failure test.
  • cargo test --test webrtc_direct_devnet --features test-utils: 6 passed, 1 ignored (as on main).
  • cargo check --lib --no-default-features: clean, so no unused import or variable without webrtc-direct.
  • cargo clippy --all-targets --all-features -- -D warnings: no findings in the changed files. Local 1.97 flags doc_markdown and similar in untouched files (replication/pointer.rs, storage/chunk_store.rs, node.rs, legacy_artifacts.rs); CI's stable doesn't.
  • cargo fmt --all -- --check: clean.
    CI (648444c, run 42050987299): all green, Security Audit included.
  • Unit tests: pass on ubuntu, macOS and Windows, including a_default_devnet_binds_os_assigned_ports.
  • Integration targets: webrtc_direct_devnet passes on all three; the five-node WebRTC devnet (Linux only) passes; devnet_lifecycle passes on Linux and macOS.
  • Quick checks: fmt, clippy, the no-default-features check and docs all pass.
  • Log scan: no "Failed to create node", "Failed to create transport", "address in use" or panic anywhere in the logs.

New dependency

none

ADR

n/a

Mitigation / rollback

Revert. Anyone who needs the old fixed ports can pass --base-port / --webrtc-direct-base-port explicitly.

🤖 Generated with Claude Code

…ssigned ports

Devnet nodes bound a fixed port each, base_port + index, in 20000-60000,
and base_port 0 meant "pick a random range". That range overlaps the OS
ephemeral range, so a node could fail with "Failed to create transport"
on a port another socket had just been given (every node's UPnP gateway
search holds a 0.0.0.0:<ephemeral> UDP socket for up to 2 s), or on a
port a Windows runner had reserved. Nodes are dual-stack by default, so
each held two fixed ports, and one failed bind failed the whole start.
Run 38251602766 (Windows) failed webrtc_direct_devnet that way.

base_port 0, now the default, means OS-assigned: each node binds port
0, and once it runs the devnet records the IPv4 port it was given, which
bootstrap_addrs() and the manifest use. An explicit base port still
gives fixed ports and the same range checks. Loopback WebRTC Direct
listeners likewise bind 127.0.0.1:0 and advertise the address they
bound. A LAN devnet (--host) must advertise its browser ports up front,
so it still gets a fixed range (V2-1897).

The startup-failure test now holds a port of its own and passes it as an
explicit WebRTC Direct base. New test: a default devnet's nodes get
distinct non-zero ports, and the bootstrap address carries the
bootstrap node's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES — two blockers are confirmed in the reviewed source.

  1. When native ports are fixed and WebRTC Direct is enabled on loopback with webrtc_direct_base_port == 0, each browser listener receives an OS port while later nodes still need fixed ports from base_port..base_port+node_count. A listener can take a pending native port and make a later node fail to bind. Preserve disjoint allocation for this combination or reserve all fixed native ports before starting ephemeral listeners.
  2. A LAN devnet with WebRTC Direct and node_count large enough that DEVNET_PORT_RANGE_MAX - node_count <= DEVNET_PORT_RANGE_MIN reaches gen_range with an empty range and panics. Reject unsupported capacity with DevnetError::Config before allocation.

CI reported on this head is green, including Windows, macOS, Linux and CI Gate. I did not run the Rust suite locally. Review coverage: 2/2 seats (sol-1 gpt-6.1-sol xhigh; glm-1 z-ai/glm-5.3). The seats disagree: sol-1 identified these blockers; glm-1 approved but also flagged the port-zero/fixed-WebRTC interaction and missing edge-case coverage. I independently verified both blockers against the source. No merge performed.

Comment thread src/devnet.rs Outdated
// A loopback listener on port 0 advertises the address it bound. A
// LAN listener has to advertise its port up front, so it gets a
// fixed range (V2-1897).
if config.webrtc_direct_base_port == 0 && config.advertise_ip.is_some() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For loopback devnets this leaves webrtc_direct_base_port at 0 even when native_range is fixed. start_node starts each node’s native listener and then obtains an OS-assigned WebRTC port before later nodes bind their fixed native ports. That OS port can equal a later node’s pending native port, so startup still fails with an address-in-use race. Keep the WebRTC allocation disjoint from a fixed native range (or reserve all native ports first), and add a regression test for this mixed configuration.

Comment thread src/devnet.rs
let mut rng = rand::thread_rng();
let max_base = DEVNET_PORT_RANGE_MAX.saturating_sub(node_count);
let Some((base_port, max_port)) = native_range else {
return Ok(rng.gen_range(DEVNET_PORT_RANGE_MIN..max_base));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When native_range is None and node_count >= DEVNET_PORT_RANGE_MAX - DEVNET_PORT_RANGE_MIN, max_base is at or below the range start, so gen_range(DEVNET_PORT_RANGE_MIN..max_base) panics. This is reachable for a LAN devnet with default base_port == 0, WebRTC enabled, and a large accepted node_count. Validate available range capacity and return DevnetError::Config instead; add a boundary test.

Nic-dorman and others added 2 commits November 5, 2026 16:09
…vnet the range can't fit

Hermes round 1 on #282:

1. With fixed native ports and loopback WebRTC Direct on port 0, a
   browser listener's OS-assigned port could be one a later node's fixed
   native port still needed. The mirror case had the same race: nodes on
   port 0 with fixed browser ports (an explicit
   --webrtc-direct-base-port, or a LAN devnet).
2. A LAN devnet with WebRTC Direct and a node count too large for
   20000-60000 reached gen_range with an empty range and panicked.

Ports are now either all OS-assigned (a loopback devnet with nothing
fixed) or all fixed in disjoint ranges: any fixed port, native or
browser, gives the rest a fixed range clear of it.
allocate_disjoint_range serves both directions and checks the range has
room first, returning DevnetError::Config instead of panicking.

New tests, both failing against the previous commit: each combination
of fixed and OS-assigned ports resolves to all-OS-assigned or
fixed-and-disjoint, and a devnet too large for the range is a config
error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE. Both blockers from review 5787894527 are fixed: when WebRTC Direct is enabled and either mode requires fixed ports, Devnet::new now resolves both native and browser ranges to fixed, disjoint ranges; range-capacity validation returns DevnetError::Config before random allocation can panic. The added tests cover the mixed configurations and oversized node count.

Both seats agree (two model families): 2/2 reviewed (sol-1 gpt-6.1-sol xhigh; glm-1 z-ai/glm-5.3). CI on this head is green, including Linux, macOS, Windows, CI Gate, Security Audit, and required metadata checks. I did not run Rust tests locally. No merge performed.

@Nic-dorman
Nic-dorman merged commit 4505953 into main Nov 5, 2026
18 checks passed
@Nic-dorman
Nic-dorman deleted the nic/v2-1896-ant-node-devnet-nodes-and-loopback-webrtc-direct-listeners branch November 5, 2026 19:38
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 6, 2026
Brings in main up to 360d472: OS-assigned ports for e2e test nodes (WithAutonomi#281)
and devnet nodes (WithAutonomi#282), release-key and signature checks (WithAutonomi#276), the
third-party notices and audit-regression CI gates (WithAutonomi#274, WithAutonomi#275), the 10 s
DHT request timeout for test nodes (WithAutonomi#277), and the 0.24.0 release (WithAutonomi#283).

One conflict, in tests/e2e/harness.rs: main removed the network bring-up
retry on a fresh port range together with NETWORK_START_ATTEMPTS and its
`warn` import, and this branch had added two helpers next to them
(POSSESSION_PENALTY_TRUST_RATIO and parseable_single_node_proof). The
resolution keeps both helpers and takes main's removal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants