Skip to content

ci: fix nightly Sanitizers + Interop failures#8

Merged
JacobBarthelmeh merged 3 commits into
wolfSSL:mainfrom
Frauschi:ci_fixes
Jul 15, 2026
Merged

ci: fix nightly Sanitizers + Interop failures#8
JacobBarthelmeh merged 3 commits into
wolfSSL:mainfrom
Frauschi:ci_fixes

Conversation

@Frauschi

Copy link
Copy Markdown
Contributor

Summary

The first scheduled runs of the nightly Sanitizers and Interop workflows both failed. This branch fixes them. CI-only — no library or source changes, independent of the SCEP work.

Changes

  • Sanitizers / ThreadSanitizer — wolfSSL master no longer compiles under -fsanitize=thread: GCC's -Wtsan fires on wc_port.h's C11 atomic_thread_fence() and wolfSSL's default -Werror makes it fatal. Append -Wno-error=tsan to the full-tsan CFLAGS. Verified on real GCC (fails without, builds + installs with).
  • Interop / openssl — the cross-check drove EST over plaintext http://, which the client rejects (EST mandates TLS, RFC 7030). Now stands the server up behind TLS (throwaway CA + --trust). Also: renamed openssl_cms.shopenssl_pkcs7_xcheck.sh (it never used openssl cms); enroll across the full key-type matrix (RSA, ECC P-256/384/521, Ed25519, Ed448, and ML-DSA when OpenSSL ≥ 3.5) and check each cert + private key with OpenSSL; pinned to ubuntu-26.04 (OpenSSL 3.5.5) so ML-DSA is covered.
  • Interop / micromdm/scep — the D2 case is marked KNOWN-FAIL (same tolerant pattern D1 already uses). It fails on an upstream wolfSSL PKCS#7 bug (wc_PKCS7_VerifySignedData returns SIG_VERIFY_E: micromdm's SignerInfo digestAlgorithm omits the NULL params while its RSA signature covers a NULL-present DigestInfo, so wolfSSL rebuilds the wrong comparison DigestInfo). Not fixable in wolfCert. D2 self-heals to PASS once the wolfSSL fix reaches master; any other failure still fails loudly.

Verification

  • TSan: A/B build on GCC — unpatched fails at wc_port.h, patched builds clean.
  • openssl: openssl_pkcs7_xcheck.sh passes end-to-end locally across all key types (incl. ML-DSA on OpenSSL 3.6).
  • micromdm root cause proven at the byte level; upstream wolfSSL write-up available.

Frauschi added 3 commits July 15, 2026 13:43
wolfSSL master's wc_port.h defines XFENCE() as C11 atomic_thread_fence(),
which GCC rejects under -fsanitize=thread with the -Wtsan diagnostic
"atomic_thread_fence is not supported with -fsanitize=thread". wolfSSL's
default GCC build enables -Werror, so this aborts the full-tsan wolfSSL
build (in random.c/kdf.c via ForceZero's XFENCE) before any test runs.

Append -Wno-error=tsan to the full-tsan CFLAGS to demote that diagnostic
back to a warning. It precedes wolfSSL's trailing -Werror on the command
line, and an explicit -Wno-error= survives a later bare -Werror, so the
build completes with the warning still visible. Verified on GCC: the
unpatched build fails at wc_port.h:1941, the patched build compiles and
installs cleanly.
The openssl interop test drove EST over plaintext http://, which
wolfcert-client refuses (EST mandates TLS, RFC 7030), so the job failed
on its first scheduled run. Stand wolfcert-server up behind TLS with a
throwaway CA + server cert (SAN localhost/127.0.0.1) that the client
trusts via --trust, exercising wolfCert's real TLS transport.

Also:
- Rename openssl_cms.sh -> openssl_pkcs7_xcheck.sh: the test never used
  `openssl cms`; it cross-checks with x509/pkcs7/pkey/req/verify.
- Enroll across the full client key-type matrix (RSA, ECC P-256/384/521,
  Ed25519, Ed448, and ML-DSA when OpenSSL >= 3.5) rather than only
  ecc:256. For each: verify the issued cert chains to the CA, the emitted
  private key parses, and its public half matches the certificate.
- Pin the job to ubuntu-26.04 (OpenSSL 3.5.5) so the ML-DSA path is
  covered; the script probes for support and degrades to classical/EdDSA
  on older OpenSSL rather than failing.
- Encode/decode base64 via `openssl base64` so the script is portable
  (the base64(1) CLI differs between GNU and BSD/macOS).
The D2 case (wolfcert-client enroll against micromdm scepserver) fails
because wolfSSL's wc_PKCS7_VerifySignedData rejects micromdm's CertRep
with SIG_VERIFY_E: micromdm's SignerInfo digestAlgorithm omits the SHA
NULL params while its RSA signature covers a NULL-present DigestInfo, so
wolfSSL rebuilds the comparison DigestInfo from the wrong encoding and it
never matches. This is an upstream wolfSSL PKCS#7 bug, not fixable in
wolfCert.

Wrap D2 in the same tolerant `set +e` / rc-check pattern D1 already uses
so the interop job stays green on this known gap. It self-heals to PASS
once the wolfSSL fix reaches master (the enroll then succeeds), while a
different failure (wrong subject, broken chain) still fails the job
loudly. The otherwise-hidden wolfcert-client.log "detail" line is now
surfaced in the KNOWN-FAIL output.
@JacobBarthelmeh
JacobBarthelmeh merged commit d462dc4 into wolfSSL:main Jul 15, 2026
21 checks passed
@Frauschi
Frauschi deleted the ci_fixes branch July 16, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants