Ask the appliance for its DTLS port before sweeping for it - #98
Merged
Merged
Conversation
This was referenced Sep 20, 2026
Merged
QuiteYellow
marked this pull request as ready for review
September 20, 2026 15:16
probe_dtls_ports read a reply arriving after a ClientHello to port N as proof that a DTLS server listens on N. An OCF stack binds its DTLS socket to port 0 and answers a first flight from that ephemeral port whatever port was addressed (RT-OCF rt_udp.c:152,164; iotivity-lite splits the same way at ipadapter.c:465-467), and the probe's socket filters inbound on host only. So every port dialled looked like a listener. Docker's bridge NAT keys conntrack on the port dialled and had been discarding the mismatched replies, standing in for a correctness check the probe never had. Running the bridge on host networking exposed it in one line: "multiple DTLS listeners answered; configure OCF_PORT", then a confident "discovered DTLS port 5684" for an appliance whose DTLS port is 49155. Measured port by port afterwards, exactly two ports answer on each appliance: the standard CoAPS port 5684, which stock RT-OCF does not bind, and the ephemeral DTLS port. Both reply from the ephemeral one. 5683, 5685, 40000 and the unused band ports stay silent, so the device is not answering indiscriminately. The probe was counting 5684 and 49155 as two listeners. Selection now runs on HostFilteredUdpSocket.observed_reply_port, which already existed and was marked diagnostic only, and DtlsLivenessResult carries it as responder_port. live_ports keeps its meaning, the ports dialled that drew a reply, and responder_ports is the new proven set. Where no source port was observed the old dialled-port selection still applies, so nothing regresses on a platform that does not report it. Validated on two appliances under host networking: correct ports, both connected, no errors. The useful consequence is that one ClientHello to 5684 now returns the real DTLS port, including ports outside the 49152-49160 band the sweep covers.
Docker's bridge NAT keys its conntrack entry on the port dialled, so a reply arriving from any other port reads as an unrelated new inbound flow. Reverse-NAT needs a conntrack entry to match, so the kernel drops that reply at the bridge. An OCF device answers plain-CoAP discovery on 5683 from its own unicast ephemeral socket, so that whole path is unreachable from a bridged container, and with it the standard-port firmware and the non-Samsung OCF devices that path reaches. Symmetric traffic, DTLS to an already-known port, traverses bridge NAT fine, which is why this ran bridged this long without the gap showing. Host mode was tried once before the probe was ready and broke discovery, because the probe took the port it dialled as the listener. That is fixed in the preceding commit, so host mode is safe now. No ports are published: the bridge opens every connection itself, DTLS per appliance and MQTT to the broker on 1883. The comment in the file carries the reasoning, since the next person to tidy this will read network_mode as laziness.
/oic/res discovery parsed eps on the unfiltered read and p.sec/port only on a second /oic/res?rt=oic.r.doxm lookup. A device's spec generation decides which form it emits, and the query has no say in it, so splitting them by lookup cost a round trip to learn what the first answer had already said. Both appliances here are OIC 1.1 (icv core.1.1.0), answer 4.06 to an OCF 1.0 Accept, and carry no eps key anywhere in /oic/res. Their real DTLS port arrives as p.sec/port on the doxm link of the unfiltered directory: dryer port 49155 in a 1727-byte representation, oven 49154 in 1629 bytes, both with_eps 0. So the form the primary read looked for is one no device tested here emits, and the fallback lookup fired every time. Nothing has yet produced an eps entry for that path to parse, on hardware or in a report. The extraction now reads both forms from whichever representation arrives. The doxm narrowing is unchanged and is the point: an eps URI is validated against the address the response came from, where a bare p.port integer carries no address, so it is read only from the one link that structurally has to describe this device's own secure endpoint. A sec-flagged port on any other link stays unread, which a test now pins along with the rt and href checks that establish the link. eps ports lead the returned tuple, since they carry the stronger validation, and _MAX_PORTS still caps the set. The filtered lookup stays for a device whose unfiltered directory advertises nothing usable, where a smaller representation gives the same parse a second chance. An OIC 1.1 directory now resolves in one exchange, asserted end to end by attempts == 1 and by no second request reaching the listener.
Port discovery swept a nine-port band plus 5684 and the descriptor default. That is a guess at a layout, and it misses any device whose DTLS port sits outside the band: vmonkey's TV at 51163 is one, and localthings#321 reads like another. The device already answers the question. /oic/res on the plaintext CoAP port is the unauthenticated path every OCF device has to expose, it is mandatory on all of them, and it names the secure port outright. On both appliances here one such read returns the right port, 49155 and 49154, matching what a ClientHello to 5684 selects. So _resolve_port now reads the directory first and sweeps the band only if that yields nothing. An advertisement stays a candidate and gets proven like any other: both tiers go through the same probe_dtls_ports gate, so a directory serialised before the DTLS bind, or one that has since moved, falls through to the band. That case is real. RT-OCF binds the DTLS socket with port 0 and learns the assignment through getsockname, and a live read here has returned sec: true, port: 0 while a later read of the same device gave a real port. The band is kept for two reasons. 5683 is mandated only as the multicast listen port, and it answers unicast because both reference stacks wildcard-bind that socket, which is a strong convention with no clause behind it. And a device whose plaintext port is not 5683 has no directory for this tier to read. A pinned OCF_PORT and a cached port both skip the read, since neither has anything to learn from an advertisement. Tests cover the ordering, the fall-through, a failed read, the two skips, and ambiguity inside the new tier.
Four paths can pick the port: a pinned OCF_PORT, the cache from an earlier connect, the device's own /oic/res advertisement, and the band sweep. Only one of them logged anything. The port number on its own leaves the useful part out: a swept port means the directory read got nothing out of that device, and that is the reading a bug report turns on. Every path now reports through one line. _accept_port logs the port with the tier that produced it and caches what discovery proved, and the directory read reports its own outcome: the ports it advertises, or the redacted result, whose error code and attempt count separate a silent plaintext port from a device that answered and advertised nothing usable. A cached port going silent says so before rediscovering, where it used to fall through in silence. A pinned port is no longer written to the cache. It is re-read from config on every connect and the cache is only consulted when nothing is pinned, so storing it left _discovered_port meaning two different things. Validated on both appliances, and the log answers the question it was added for: "directory on 5683 advertises 49155", then "DTLS port 49155, advertised by /oic/res on 5683", with the band never swept. The sample output in docs/bridge-demo.md is that run, with the tier vocabulary spelled out underneath.
The dryer answers the bridge's ClientHello with a ClientHello of its own
after the bridge has been away, and OpenSSL, being the client, rejects it. That
surfaced as "session error: session operation failed", a warning, and an
error_count bump, followed by a backoff, for behaviour that is expected and
clears itself in about two seconds.
Captured on the wire with a per-record trace:
TX 160B Handshake epoch=0 seq=0 ClientHello msg_seq=0 frag=0/135 of 135
RX 123B from :49155 Handshake epoch=0 seq=1 ClientHello msg_seq=0 of 98
-> dtls handshake failed at the TLS layer: unexpected message
The appliances run IoTivity classic, which their own device log names
(ocstack.c, cablockwisetransfer.c, ca_adapter_net_ssl.c, plus Samsung's
cloud_manager.c and friends). In that stack CAencryptSsl opens a session
itself when it has a message for an endpoint it holds no peer for, which is
how an appliance comes to dial the bridge: it still holds the fixed local
port from the previous session. GetSslPeer keys the peer table on address and port
with no role, so the bridge's own ClientHello is then stepped into that
client-role context, and the failure path skips the alert for
MBEDTLS_ERR_SSL_BAD_HS_CLIENT_HELLO and removes the peer. The device
therefore answers with silence and drops its half, which is why the next
handshake succeeds. The trace agrees: their ClientHello arrived and no alert
came back.
So the collision resolves itself, and the useful change is to say what it
was. connect() now raises PeerInitiatedHandshakeError when an inbound
epoch-0 ClientHello preceded the failure, logging at info, and the bridge retries after 0.5s without touching error_count or
growing the backoff. Nothing is wrong with the device or the network, and
the health topic keeps meaning what it says.
The detection reads only epoch 0, since a later epoch is encrypted and a
peer's first flight never appears there, and it walks every record because
flights arrive coalesced. PeerInitiatedHandshakeError subclasses
SessionError, so existing handling is unaffected.
Two things deliberately not done. Dropping the peer's ClientHello and
carrying on cannot work: the device sends nothing once it has consumed the
bridge's, so recovery would still wait on OpenSSL's retransmit. Accepting the inbound
handshake has no precedent in any OCF stack read here on UDP, and would
recreate the same role ambiguity.
The docs told readers that a reply after dialling a port meant a DTLS listener was on it, and described port discovery as a sweep of the band. Neither survives this branch: selection now runs on the reply's source port, and the bridge asks /oic/res on 5683 before it sweeps. Also notes that a silent 5683 is worth a second look, since the answer arrives from another source port and a stateful firewall or a bridge NAT will drop it while the device is answering.
Two follow-ups to the peer-initiated handshake classification, both about what happens when the expected case stops being the actual one. A refused handshake was retried at 0.5s with no cap, so an appliance that kept refusing drove a 2 Hz handshake loop indefinitely while error_count stayed at zero and the health topic reported a healthy bridge. Measured with the loop harness: 200 consecutive collisions gave 200 immediate retries. That is also the pattern that has been observed to precede an appliance going silent for minutes. Four consecutive collisions still take the fast path; past that a collision counts as a fault and takes the jittered backoff. A session that runs clears the count, so an appliance that collides once per outage keeps the fast path for good. The fixed DTLS source port moves from 49700 to 26849. Host networking binds it on the host, and 49700 sits inside the default net.ipv4.ip_local_port_range of 32768-60999, where an unrelated process can hold it and the bind fails EADDRINUSE for one appliance after a restart. Under bridge networking the bind was isolated in the container's namespace, which is why the old base was safe before and is not now. 26849-26864 was picked at random from the registered range, carries no name in /etc/services, and is clear of the common dev and IoT ports. A reply whose source port went unrecorded no longer falls back to the port dialled. The real socket records that port as it accepts the datagram, so the fallback was unreachable. Its failure mode was the problem: it restored the rule this work replaced, which answers confidently and wrongly, since an OCF stack replies from its own port whatever was addressed. It reports unreachable now. A regression in the socket layer surfaces as a failed connect, where before it would have opened a session against the wrong endpoint. Two tests that selected on the dialled port go with it. Each already has a counterpart written for the responder rule. Three comments corrected alongside them. probe_dtls_ports selects on the reply source port, so the note calling that port diagnostic no longer describes what it does. The ClientHello scan reads only the first handshake message in each record, so a coalesced one costs the fast retry, and the docstring now says so. And reading both advertised port forms from one answer also means eps is accepted from any link in the filtered doxm lookup, which widens the source without widening what is trusted.
Both comments explaining the fixed DTLS source port credited RFC 6347 section 4.2.8 eviction: a device that sees a re-handshake on the old 5-tuple treats the peer as rebooted and discards the stale association, which otherwise lingers until a device-side timer reaps it. That mechanism belongs to the tinydtls lineage. The firmware these appliances run has no such path and no reaping timer. What it has is a peer table with no cap, no idle timeout and no LRU, keyed on address and port, so the real argument for one fixed port is the opposite shape: a client that reconnects from a fresh ephemeral port every time leaves the old entry live and piles up mbedTLS contexts on an embedded device. Reconnecting over an entry the device still holds stays cheap, because a ClientHello is not application data it can read, so it destroys the stale peer. Measured on hardware: one extra ClientHello and about 1.1s on the dryer, nothing on the oven. The dtls_session comment also credited the oven result to RT-OCF, which is not the stack that oven runs. Both now say "the firmware these appliances run", naming a stack only where a source claim needs one. Nothing about the behaviour changes. The fixed port stays, and the oven's 2026-07-26 verification stands; what it stands on is different.
The comment explaining a peer-initiated handshake cited "IoTivity classic ca_adapter_net_ssl.c:1520 CAencryptSsl". Both halves mislead. 1520 is the upstream number. These appliances run TizenRT's iotivity_1.2-rel fork. There CAencryptSsl sits at 2010 and its InitiateTlsHandshake call at 2040. Anyone who opened the tree the comment is about landed hundreds of lines away in something unrelated. The comment now also gives numbers for the two other symbols it leans on, GetSslPeer at 1074 and SSL_CHECK_FAIL at 215, whose BAD_HS_CLIENT_HELLO test is at 247. Naming the family is the second problem, because upstream predicts handshake behaviour these appliances do not show: it sets VERIFY_REQUIRED, and it carries no ECDHE-ECDSA-AES128-GCM-SHA256, yet the appliances complete a session on an empty client certificate, using exactly that suite. Read upstream here and you get the wrong answer, so the comment names the tree, and the pin.
QuiteYellow
force-pushed
the
feat/directory-first-discovery
branch
from
September 20, 2026 18:39
a3d6b5f to
92a5624
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Discovery now reads the device's own
/oic/resdirectory first and takes the secure port from it. That resource is mandatory on every OCF device and must be reachable without DTLS, so one exchange yields the port. Two other fixes come with it: a probe bug that let a wrong port pass as proven, and a handshake collision that was reported as a fault.Six commits, each standing alone. Validated on a dryer and an oven over several stop/start cycles.
The probe was counting the wrong port
probe_dtls_portsread "a reply arrived after dialling port N" as proof a DTLS server listens on N. An OCF stack binds its DTLS socket to port 0 and answers a first flight from that ephemeral port whatever port was addressed, and the probe's socket filters inbound on host only, so every port dialled looked live. Docker's bridge NAT had been discarding the mismatched replies and standing in for a correctness check the probe never had.Selection now runs on the port a reply came from (
responder_port). Measured port by port, exactly two ports answer on each appliance here: the standard CoAPS port 5684 and the ephemeral DTLS port, both replying from the ephemeral one. TizenRT'siotivity_1.2-relfork, which these appliances run, binds 5684 as its secure multicast socket onINADDR_ANY(caipadapter.c:206at pine590f30ab), so a unicast ClientHello addressed there lands on that socket and is answered from the ephemeral secure one. The old code called those two separate listeners and reported5684for an appliance whose port is49155.Ask before sweeping
_resolve_portnow runs pinnedOCF_PORT→ cached port →/oic/reson 5683 → band sweep, and every tier's answer goes through the sameprobe_dtls_portsgate, so an advertisement is only dialled once it has answered one. That matters: a live read here has returnedsec: true, port: 0, because the stack binds port 0 and learns the assignment afterwards.The band stays. 5683 is mandated only as the multicast listen port; it answers unicast because every OCF stack read here, classic included, wildcard-binds that socket, which is a strong convention with no clause behind it. And a device whose plaintext port is not 5683 has no directory for this tier to read. On the bench the fallback earned its keep unprompted: after one dropout the dryer's plaintext port stayed silent while its DTLS port recovered, and the sweep is what found it. Blocking the route to 5683 makes that path deterministic rather than opportunistic: both appliances log
no_ocf_responseafter four attempts and resolve on the sweep, so a dead directory costs one bounded read and falls through._ports_from_linksalso now reads both advertised forms from one answer. Both appliances here are OIC 1.1, answer 4.06 to an OCF 1.0Accept, and carry noepskey anywhere, so the real port arrives only asp.sec/porton the doxm link. The old split looked forepson the first read andp.sec/portonly on a second, filtered one, so every such device paid a wasted round trip. The doxm narrowing is unchanged and is the point: anepsURI is validated against the address the response came from, where a barep.portinteger carries no address, so it is read only from the link that has to describe the device's own secure endpoint. Nothing here has produced anepsentry, on hardware or in a report, so that path stays unexercised.Say which tier chose the port
Four paths can pick it and only one logged anything. Each now names itself (
configured OCF_PORT,cached from an earlier connect,advertised by /oic/res on 5683,found by sweeping the OCF band), because a swept port means the directory read got nothing out of that device, and that is the detail a bug report turns on.A peer that opens its own handshake is not a fault
After the bridge has been away, the dryer answers its ClientHello with a ClientHello of its own, and OpenSSL, being the client, rejects it. That read as
session error: session operation failed, a warning, anerror_countbump and a backoff, for something that clears itself in about two seconds.These appliances run TizenRT's
iotivity_1.2-relfork, which their own device log names. There,CAencryptSslopens a session itself when it has a message for an endpoint it holds no peer for. That is how an appliance comes to dial the bridge: it still holds the fixed local port from the previous session.GetSslPeerkeys the peer table on address and port with no role, so the bridge's ClientHello is stepped into that client-role context, and the failure path skips the alert forMBEDTLS_ERR_SSL_BAD_HS_CLIENT_HELLOand removes the peer. The device answers with silence and drops its half, which is why the next handshake succeeds. A per-record trace confirmed the prediction: the peer's ClientHello arrived and no alert came back.connect()now raisesPeerInitiatedHandshakeErrorwhen an inbound epoch-0 ClientHello preceded the failure, logging at info, and the bridge retries after 0.5s without touchingerror_countor growing the backoff.Dropping the peer's ClientHello and carrying on was tried and rejected: the device sends nothing once it has consumed the bridge's, so recovery would still wait on OpenSSL's retransmit. It was armed on hardware for two cycles and never fired.
The demo bridge moves to host networking
Bridge NAT keys its conntrack entry on the port dialled, so a reply from any other port reads as an unrelated new inbound flow and the kernel drops it. That silently rules out the 5683 path, which is how standard-port firmware and non-Samsung OCF devices are reached. Symmetric DTLS to a known port traverses NAT fine, which is why this ran bridged for so long with the gap staying hidden. Safe only because of the responder-port fix, so the commit order matters.
Notes
docs/api.mdis regenerated;PeerInitiatedHandshakeErroris intools/api_contract.pybecause callers catch it by name to drive the retry.docs/appliance-compatibility.md,docs/ocf-pki-laundry.mdanddocs/bridge-demo.mdare corrected in the last commit. They told readers that a reply after dialling a port proved a listener on it, and described discovery as a sweep of the band. The compatibility page now also says that a silent 5683 is worth a second look, since the answer comes from another source port.n=2, and no standard-port or newer-PKI device has been tested.Since the first push
Review of the branch found five things. The first three are fixed in
86ab57d, the fourth ina448a5b, the fifth ina3d6b5f.The peer-initiated retry had no cap. An appliance that kept refusing drove a 2 Hz handshake loop for as long as it kept refusing, with
error_countat zero throughout, so the health topic reported a healthy bridge. 200 consecutive collisions through the loop harness gave 200 immediate retries. Four still take the fast path, and past that a collision counts as a fault and takes the jittered backoff; a session that runs clears the count.The fixed DTLS source port moves from 49700 to 26849. Host networking binds it on the host, and 49700 sits inside the default
net.ipv4.ip_local_port_rangeof 32768-60999, where an unrelated process can hold it and the bind failsEADDRINUSE. Under bridge networking that bind was isolated in the container's namespace, which is why the old base was fine until this commit and is not after it.A reply whose source port went unrecorded fell back to the port dialled. That fallback could not be reached through the real socket, and its only behaviour was to restore the rule this branch replaced, so it now reports
unreachable. Two tests that selected on the dialled port went with it, each already having a counterpart written for the responder rule.Both comments explaining the fixed DTLS source port credited RFC 6347 §4.2.8 eviction, where a re-handshake on the old 5-tuple makes the device discard a stale association that a device-side timer would otherwise reap. That mechanism is tinydtls's. The firmware these appliances run has neither the path nor the timer; what it has is a peer table with no cap, no idle timeout and no LRU, so the argument for one fixed port is the opposite shape. Reconnecting from a fresh ephemeral port each time leaves the old entry live and piles up mbedTLS contexts. Reconnecting over an entry the device still holds destroys the stale peer, since a ClientHello is not application data it can read, and that was measured at one extra ClientHello on the dryer and nothing on the oven. The
dtls_sessioncomment also credited the oven result to RT-OCF, which is not the stack that oven runs. The behaviour is unchanged either way.The comment on the peer-initiated handshake cited
IoTivity classic ca_adapter_net_ssl.c:1520 CAencryptSsl. 1520 is the upstream number, and in the fork these appliances runCAencryptSslis at 2010 with itsInitiateTlsHandshakecall at 2040, so following that citation into the right tree landed hundreds of lines away.GetSslPeer(1074) andSSL_CHECK_FAIL(215, with theBAD_HS_CLIENT_HELLOtest at 247) now carry numbers too. The family label was the other half of the problem: upstream setsVERIFY_REQUIREDand carries noECDHE-ECDSA-AES128-GCM-SHA256, yet these appliances complete a session on an empty client certificate using exactly that suite, so upstream answers this question wrongly. Two mentions of the stack in this description are corrected the same way.825 tests pass, which corrects the count above.
Since the rebase
Rebased onto main. The fork-naming pass merged there in the meantime reached the same
dtls_sessioncomment this branch rewrites, so the two collided; the resolution keeps this branch's text. That comment used to close on two corrections, and main has since made the RT-OCF half itself, so the note now states only the RFC 6347 §4.2.8 one.Two comments this branch had left misplaced are tidied with it. The jitter note in
run_foreversat above the immediate-retry branch, several lines away from the wait it describes. The new paragraph on the fixed source port ran straight into the one explaining why the base sits below 32768.871 tests pass, 27 of them added here. The counts earlier in this description predate the rebase.