Add CI that builds and runs every example#598
Open
aidangarske wants to merge 70 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a comprehensive GitHub Actions CI system to build and (where possible) run every example in this repository against wolfSSL master and the latest stable tag(s), using a manifest-driven matrix so failures are isolated to per-example jobs.
Changes:
- Introduces manifest-driven CI workflows (host/emulated/cross) plus a nightly triage workflow that retries, classifies flakes vs real failures, and can file/update issues.
- Adds Python harness/scripts to generate matrices, enforce “coverage” (no buildable example directory left unaccounted for), run examples with output assertions, and collect artifacts for triage.
- Updates many examples/Makefiles/scripts/fixtures so they build/run reliably under CI (exit statuses, paths to certs, linking flags, regenerated CRL/certs, etc.).
Reviewed changes
Copilot reviewed 62 out of 64 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| X9.146/Makefile | Makefile header tweak for X9.146 examples |
| x509_acert/README.md | Fix build/run instructions and cert paths |
| x509_acert/Makefile | Make OpenSSL prefix optional; adjust lib paths |
| uefi-static/Makefile | Make libgcc/gnu-efi paths less distro-specific |
| tls/server-tls13.c | Ensure successful exit status after shutdown |
| tls/server-tls13-certauth-clienthello.c | Ensure successful exit status after shutdown |
| tls/server-tls-posthsauth.c | Ensure successful exit status after shutdown |
| tls/client-tls13-resume.c | Load client cert/key and CA separately |
| tls/client-tls.c | Load client cert/key and CA separately |
| staticmemory/memory-bucket-optimizer/tester/Makefile | Add include/lib paths for wolfSSL |
| staticmemory/memory-bucket-optimizer/optimizer/Makefile | Add include/lib paths for wolfSSL |
| staticmemory/Makefile | Correct Makefile title/comment |
| signature/sigtest/Makefile | Make OpenSSL path optional and clearer |
| SGX_Linux/sgx_t.mk | Update SGX C++ runtime library name |
| scripts/run-all-examples.sh | Add local manifest-driven runner script |
| psa/Makefile | Fix include path and guard empty PSA_INCLUDE |
| pq/stateful_hash_sig/xmss_example.c | Remove redundant include |
| pq/stateful_hash_sig/lms_example.c | Remove redundant include |
| pq/ml_kem/README.md | Switch build instructions to make |
| pq/ml_kem/ml_kem.c | Update include/header usage; simplify main signature |
| pq/ml_kem/Makefile | Add Makefile for ML-KEM example |
| pq/ml_dsa/ml_dsa.c | Update to FIPS 204 ctx APIs; fix exit code |
| pq/ml_dsa/Makefile | Fix clean target artifacts list |
| pkcs7/scripts/runall.sh | Fix ordering + ensure failures exit non-zero |
| pkcs7/scripts/openssl-verify.sh | Gate on runall, add failure accounting, improve checks |
| pkcs7/Makefile | Correct Makefile title and zlib comment |
| pkcs11/Makefile | Correct Makefile title/comment |
| pk/srp/Makefile | Fix WOLFSSL_INSTALL_DIR prefix |
| pk/enc-through-sign-rsa/rsa-public-decrypt-app.c | Normalize return code from length-returning API |
| embedded/Makefile | Adjust linux-specific target list handling |
| ecc/ecc-params.c | Handle negative lengths; normalize exit status |
| ebpf/tls-uprobe-trace/Makefile | Add wolfSSL include/lib flags |
| dtls/server-dtls-rw-threads.c | Fix cert paths for CI execution |
| dtls/client-dtls-rw-threads.c | Fix cert paths for CI execution |
| custom-io-callbacks/file-server/file-server.c | Fix server return codes / cleanup flow |
| custom-io-callbacks/file-server/check.sh | Make server/client orchestration bounded and reliable |
| custom-io-callbacks/file-client/file-client.c | Fix client return codes |
| crypto/keywrap/Makefile | Add Makefile for keywrap example |
| crypto/aes-modes/aes-xts.c | Fix XTS key halves to be distinct |
| certvfy/Makefile | Add include/lib flags for wolfSSL |
| certs/crl/regenerate-crl.sh | Add script to regenerate CRL fixture |
| certs/crl/crl.pem | Update CRL fixture to unexpired version |
| certmanager/README.md | Fix example invocation name |
| certmanager/certverify.c | Ensure successful exit status |
| certmanager/certloadverifybuffer.c | Update embedded cert fixtures |
| certgen/Makefile | Add Makefile header/comment |
| certgen/csr_w_ed25519_example.c | Normalize return code from length-returning API |
| can-bus/Makefile | Add include/lib flags for wolfSSL |
| can-bus/generate_ssl.sh | Cap CN length to X.509 limits |
| btle/common/btle-sim.c | Ignore SIGPIPE to avoid FIFO-close crashes |
| .gitignore | Ignore new binaries/pycache; fix typo |
| .github/workflows/nightly.yml | Add scheduled nightly orchestration |
| .github/workflows/examples.yml | Add host-tier matrix workflow + coverage gate |
| .github/workflows/emulated.yml | Add emulated-tier workflow (TPM/SGX/CAN/UEFI) |
| .github/workflows/cross-build.yml | Add cross-build workflow (ESP32/Pico/etc.) |
| .github/workflows/ci-triage.yml | Add triage workflow for nightly results |
| .github/workflows/_resolve-wolfssl.yml | Add reusable wolfSSL ref resolution workflow |
| .github/scripts/run_example.py | Add example harness (build/run/assert/output/artifacts) |
| .github/scripts/manifest.py | Add manifest loader + matrix generator + coverage checker |
| .github/scripts/issue_store.py | Add issue identity/body/state machine + scrubbing |
| .github/scripts/ci_triage.py | Add triage logic (retry, classify, issues, AI advisory) |
| .github/examples-manifest.yml | Add source-of-truth manifest for CI coverage/matrix |
| .github/actions/setup-wolfssl/action.yml | Add composite action to build/cache/install wolfSSL |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
aidangarske
force-pushed
the
ci-matrix
branch
5 times, most recently
from
July 15, 2026 16:34
56d67ec to
f0f3a18
Compare
aidangarske
force-pushed
the
ci-matrix
branch
18 times, most recently
from
July 16, 2026 22:52
360330d to
0ffccd5
Compare
aidangarske
force-pushed
the
ci-matrix
branch
17 times, most recently
from
July 17, 2026 02:23
50ffcaf to
74965b0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
hashfailing in the nightly aidangarske/wolfssl-examples#2CI failures caught and fixed:
Fixtures that expired 2025-09-11 and silently broke examples
1cb122034a591fcertloadverifybuffer.cembedded certs expired whilecerts/*.pemwere regenerated99cc7baclient-ecc-cert.dermissed in that refresh, so it disagreed with its own.pemWrong exit codes — examples that "passed" by returning garbage
a1383dftls/servers returned awolfSSL_writebyte count frommaincc37913certverifyreturnedWOLFSSL_SUCCESS(=1) after succeeding79d1f67ml_dsa -pprinted its table thenexit(EXIT_FAILURE)297705b-1unconditionally07ea19fecc-paramsreflected only the last of six calls8aea08fcsr_w_ed25519_exampleandrsa-public-decrypt-appreturned byte lengthsExamples that never worked
e24538cclient-tls13-resumesent no client cert, soserver-tls13rejected it and it never resumed666b6a4aes-xtsused identical XTS key halves, which wolfSSL rejects per IEEE 1619166e6c4Checks that could not fail by construction
bee99c9runall.shbareexitreturned the precedingecho's status88d5659openssl-verify.shnever exited non-zero, and its signedData verifies never workedESP32 — unbuildable by anyone, on any supported IDF
cbe7358USE_MY_PRIVATE_CONFIGwas committed enabled, forcing an include of one developer's private headerc0cd3a2CONFIG_ESP_WIFI_SSIDwas undeclared0dbb03fclient-dtls13.ccalledXSTRCPY, which wolfSSL does not definea26f443ctxcollided withlibnet80211.aunder-fno-commonf0f3a18PERIPH_*_MODULEAPIesp32_aes.cusesEverything else
24237ebclose_notifylast53670e36338d67server-dtls-nonblockingnever settimeval.tv_usecand never re-armed it, soselectpolled with 0 and burned the DTLS retransmit budget in ms259ed1amemory-bio-dtlsuse-after-free:wolfSSL_set_bio(ssl, NULL, NULL)frees the BIOs it was called to protect, so the client tore down BIOs the server was still readingddcb1c4BIO_s_memis a byte stream, so DTLS records end mid-read and were dropped as truncated datagrams; the dtls profile now setsWOLFSSL_DTLS_RECORDS_CAN_SPAN_DATAGRAMSdb1c0b26eb6a907e33bfc/5676d1fSO_REUSEADDRand could not rebind after a restarta29961b064492f4483350generate_ssl.shcommon name length limit117b793/e1d98311dc1e79And three bugs in wolfSSL itself
this work, now merged.
pico.cusesBAD_FUNC_ARGwith no declaring include. Fixed on master (
c35ade061), in no release yet, sorpi-picotests master only.(
9b1aad457) is in no release yet, sosignedData-streamtests master only.The other 8 pkcs7 targets still test both.
Found by running examples nobody had run
608cf8focsp-stapling's responder cert was issued by the root, butserver1by intermediate1 — RFC 6960 only accepts a delegate issued by the same CA, so wolfSSL rejected every response (BAD_OCSP_RESPONDER). Swapped to wolfSSL's purpose-builtocsp-responder-int1cert.b99c579softhsm2.shran 8 binaries and returned only the last exit code — 7 could fail and it reported success. It also hardcoded/usr/local/lib/softhsm, so it was broken for any distro package.8b67f8epsa/Makefileread$(PSA_WOLFSSL_INSTALL_DIR), a name that appears nowhere in its own README — which documentsPSA_LIB_PATH. Following the docs linked nothing and died on ~20 undefinedpsa_*symbols.87c3211pk/ecc/ecc_sign.cexportedr/swithmp_to_unsigned_bin(minimal bytes) but verify reads them at fixedcurveSzoffsets. A leading-zeror/s(~1 in 256) shifted the signature → intermittent-229. Caught as a flake: identicalv5.9.2-stabletag, one run passed, the next failed.5f04377mynewt/client-tls-mn.ccast pointers throughint— fine on the 32-bit native BSP of 2018,-Werrorfatal on today'snative64.104aeeauefi-librarytested RSA with a 1024-bit key; wolfSSL'sRSA_MIN_SIZEdefaults to 2048. Moved to wolfSSL's owncerts/client-key.der. (Did not fix the UEFI-262— see below.)9a06ba8mynewt/jenkins.shhardcoded a wolfSSL master clone, so a stable leg would have silently tested master.Real bugs found, recorded, not yet fixed
uefi-librarywc_CheckRsaKeyfails-262. Key size is ruled out (2048 gives the same).rsa.ccollapses every error intoRSA_KEY_PAIR_E, hiding the cause; the driver API doesn't exportwc_RsaSSL_Sign/VerifyInlineto instrument it. Needs a Linux gnu-efi box.mynewtsettings.hsetsXMALLOC_OVERRIDEand definesXFREEas a macro,types.hstill declares itextern— the macro expands into the declaration.pkcs11pkcs11_testfails-262on a label-generated EC key (gen_ec_keys_label→wc_ecc_verify_hash); the same sequence without a label passes. Deterministic on both refs. Runs as its own step so the other 7 stay asserted.