DTLS 1.3: accept a fragmented first ClientHello when cookies are disabled

[rizlik]

Description

First PR in a series to implement DTLS server that can handle bounded fragmented Client Hello.
This first PR allow to switch return routability check completely by disabling cookies.
A server with cookies disabled is compatible with other libraries that fragments ClientHellos to accomodate big PQC keyshares.

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m4-dtls13

• FLASH: .text +384 B (+0.2%, 188,348 B / 1,048,576 B, total: 18% used)

linuxkm-standard

• Data: __patchable_function_entries +40 B (+0.1%, 50,000 B)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• stm32-sim-stm32h753

[Copilot]

🟡 Changes recommended

Unresolved moderate DTLS handling issues and critical test build-guard failures block approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds configurable DTLS cookie handling so servers can process fragmented initial DTLS 1.3 ClientHellos when cookies are disabled.

Changes:

• Adds cookie policy APIs and shared secret lifecycle handling.
• Updates DTLS accept paths and callback behavior.
• Adds DTLS 1.2/1.3 tests and documentation.

File summaries

File	Change	Final findings
wolfssl/ssl.h	Public cookie APIs and callback documentation	None
wolfssl/internal.h	Internal state and helper declarations	None
tests/api/test_tls13.c	TLS 1.3 API coverage	None
tests/api/test_dtls13.h	DTLS 1.3 test registration	None
tests/api/test_dtls13.c	DTLS 1.3 handoff and fragmentation tests	None
tests/api/test_dtls.h	Shared DTLS test declarations	None
tests/api/test_dtls.c	DTLS cookie, fragmentation, and policy tests	Critical (1 vote): Missing server-build guards cause compilation/link failures. Two tests also require both client and server support.
src/tls13.c	TLS 1.3 cookie and accept handling	Moderate (1 vote): Stateful processing must account for disabled HRR-cookie support.
src/ssl.c	Callback state reset	None
src/ssl_api_hs.c	Generic accept state handling	Moderate (1 vote): No-HRR-cookie builds can reject valid no-cookie DTLS 1.3 ClientHellos and must retain DTLS 1.2 HVR downgrade behavior.
src/ssl_api_dtls.c	Cookie policy and callback implementation	None
src/internal.c	DTLS cookie initialization	Moderate (1 vote): Reset chGoodCbDone when initializing a reused server connection.
doc/dox_comments/header_files/ssl.h	Public API documentation	None

Review details

Suppressed comments (3)

src/internal.c:2440

• This initializer is also used by wolfSSL_set_accept_state(), so a reused server object can enter a new handshake with chGoodCbDone still set from its previous no-cookie connection. DtlsNoCookieChGood() will then suppress the callback for the new ClientHello. Reset this per-connection bit alongside the cookie policy when initializing the server side.
  src/ssl_api_hs.c:931
• In a build with WOLFSSL_SEND_HRR_COOKIE disabled, DTLS 1.3 has no HRR-cookie path, but InitSSL_DtlsServer() still sets sendCookie to 1. This leaves the generic accept path stateless before parsing the version, so DoClientHelloStateless() rejects a valid DTLS 1.3 no-cookie ClientHello instead of transitioning to stateful processing; generic methods also need to retain the DTLS 1.2 HVR downgrade behavior.

        if (!ssl->options.sendCookie)
            ssl->options.dtlsStateful = 1;

src/tls13.c:17781

• This test assumes sendCookie means that a DTLS 1.3 cookie mechanism is available. With WOLFSSL_SEND_HRR_COOKIE disabled, InitSSL_DtlsServer() still initializes this bit to 1 for every DTLS server, so this path leaves dtlsStateful false and sends the first no-cookie ClientHello through DoClientHelloStateless(), which rejects it (and cannot reassemble a first fragment). Please make the stateful decision account for the compile-time absence of HRR-cookie support; the generic wolfSSL_accept() path needs the same correction.

        if (!ssl->options.sendCookie)
            ssl->options.dtlsStateful = 1;

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.