Skip to content

server: fix data races found by the nightly ThreadSanitizer job#9

Merged
philljj merged 1 commit into
wolfSSL:mainfrom
Frauschi:tsan_server_races
Jul 16, 2026
Merged

server: fix data races found by the nightly ThreadSanitizer job#9
philljj merged 1 commit into
wolfSSL:mainfrom
Frauschi:tsan_server_races

Conversation

@Frauschi

Copy link
Copy Markdown
Contributor

What & why

The nightly Sanitizers workflow's ThreadSanitizer job has been failing on every threaded roundtrip since the TSan build actually started running (the earlier -Werror fix let it get far enough to execute the tests). Root cause: two genuine data races.

1. stopping flag — src/server.c

WolfCertServer.stopping was a plain volatile int, written by wolfcert_server_stop() (from another thread in the test harness, or from the wolfcert-server CLI's SIGINT/SIGTERM handler) and polled by the accept loop in wolfcert_server_run(). volatile establishes no happens-before, so TSan flags it. Separately, stop() close()d listen_fd while the accept loop still read it — a second race hidden behind the first.

2. port handoff — tests/integration/test_tls_http.c

srv_thread published the ephemeral listener port into a main-thread stack struct that main spin-polled without synchronization.

The fix

  • stoppingwolfSSL_Atomic_Int (wc_port.h), accessed via WOLFSSL_ATOMIC_LOAD / WOLFSSL_ATOMIC_STORE. Gives the cross-thread handoff a happens-before edge and an async-signal-safe store for the CLI.
  • The accept loop now poll()s the listener on a short timeout and re-checks the flag itself. stop() only sets the flag and never touches listen_fd (closed by free() after the serving thread is joined). This removes the second race and drops the reliance on shutdown()/close() waking a blocked accept(), which is not portable to BSD/macOS.
  • test_tls_http's port field → wolfSSL_Atomic_Int as well.
  • Hardening: the accept loop retries transient per-connection errors (ECONNABORTED / ECONNRESET) instead of tearing the listener down over a single misbehaving peer.

Embedded / portability

wc_port.h's atomics degrade to volatile int under WOLFSSL_NO_ATOMICS, so the client/embedded path is unchanged — on a no-atomics target the field is literally volatile int again, as before. The change is confined to the server subsystem (WOLFCERT_ENABLE_SERVER) and one integration test.

Verification

  • Local: 12/12 roundtrip|tls_http pass clean under -fsanitize=thread (halt_on_error=1); full 24-test suite green.
  • CI: Sanitizers workflow dispatched on this branch — ThreadSanitizer, ASan+UBSan, and valgrind all ✅ (run 29482351976).

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #9

Scan targets checked: wolfcert-bugs, wolfcert-src

No new issues found in the changed files. ✅

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes ThreadSanitizer-reported data races in the server shutdown path and in an integration test’s cross-thread port publication, improving correctness of threaded operation under TSan and real concurrency.

Changes:

  • Replace WolfCertServer.stopping with wolfSSL_Atomic_Int and use WOLFSSL_ATOMIC_LOAD/STORE for cross-thread stop signaling.
  • Make the accept loop poll() with a short timeout instead of blocking in accept(), and stop closing listen_fd from wolfcert_server_stop().
  • Fix the test_tls_http port handoff race by publishing/polling the port via wolfSSL_Atomic_Int.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/integration/test_tls_http.c Uses wolfSSL_Atomic_Int for the server port handoff between threads to satisfy TSan.
src/server.c Reworks the accept loop to poll and uses atomic stop signaling; removes cross-thread listen_fd shutdown/close.
src/internal.h Updates WolfCertServer to store the stop flag as wolfSSL_Atomic_Int.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/server.c

@philljj philljj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one nit on poll(2) event handling.

@philljj philljj assigned Frauschi and unassigned wolfSSL-Bot Jul 16, 2026
The nightly Sanitizers workflow's ThreadSanitizer subjob failed on every
threaded roundtrip once the TSan build started running (the prior -Werror
fix let it get far enough to execute). Two distinct data races:

- wolfcert_server_run()/wolfcert_server_stop(): `stopping` was a plain
  volatile int, written by the stopping thread (or the wolfcert-server CLI
  signal handler) and polled by the accept loop. volatile carries no
  happens-before, so TSan flags it; stop() also closed listen_fd while the
  accept loop still read it. Make `stopping` a wolfSSL_Atomic_Int (wc_port.h)
  and have the accept loop poll() the listener on a short timeout so it
  re-checks the flag itself. stop() now only sets the flag and never touches
  listen_fd (free() closes it after the serving thread is joined). This also
  drops the reliance on shutdown()/close() waking a blocked accept(), which
  is not portable to BSD/macOS.

- test_tls_http: srv_thread published the ephemeral port into a main-thread
  stack struct that main spin-polled without synchronization. Make that field
  a wolfSSL_Atomic_Int as well.

The wc_port atomics degrade to volatile int under WOLFSSL_NO_ATOMICS, so the
client/embedded path is unchanged. The accept loop also retries transient
per-connection errors (ECONNABORTED/ECONNRESET) instead of tearing the
listener down over a single misbehaving peer.

Verified: the 12 roundtrip|tls_http tests pass clean under -fsanitize=thread
(halt_on_error=1); the full 24-test suite stays green.
@Frauschi
Frauschi force-pushed the tsan_server_races branch from 671f516 to c750799 Compare July 16, 2026 16:22
@Frauschi
Frauschi requested a review from philljj July 16, 2026 16:23
@Frauschi Frauschi assigned wolfSSL-Bot and unassigned Frauschi Jul 16, 2026
@philljj
philljj merged commit 3dbb65e into wolfSSL:main Jul 16, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants