Skip to content

pkcs7: add RSA-PSS support for SignedData#9742

Open
sameehj wants to merge 1 commit intowolfSSL:masterfrom
sameehj:pkcs7-rsa-pss
Open

pkcs7: add RSA-PSS support for SignedData#9742
sameehj wants to merge 1 commit intowolfSSL:masterfrom
sameehj:pkcs7-rsa-pss

Conversation

@sameehj
Copy link
Contributor

@sameehj sameehj commented Feb 5, 2026

Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData encoding and verification.

This change enables SignerInfo.signatureAlgorithm to use id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1, salt length), as required by RFC 4055 and CMS profiles.

Key changes:

  • Add RSA-PSS encode and verify paths for PKCS7 SignedData
  • Encode full RSASSA-PSS AlgorithmIdentifier parameters
  • Decode RSA-PSS parameters from SignerInfo for verification
  • Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo)
  • Fix certificate signatureAlgorithm parameter length handling
  • Add API test coverage for RSA-PSS SignedData

This resolves failures when using RSA-PSS signer certificates (e.g. -173 invalid signature algorithm) and maintains backward compatibility with RSA PKCS#1 v1.5 and ECDSA.


Testing

  • API: New test test_wc_PKCS7_EncodeSignedData_RSA_PSS (guarded by HAVE_PKCS7, WC_RSA_PSS, RSA, filesystem, SHA-256). Uses certs/rsapss/client-rsapss.der and client-rsapss-priv.der; encodes SignedData and optionally round-trip verifies.
  • CI: os-check.yml updated with build --enable-pkcs7 CPPFLAGS=-DWC_RSA_PSS.
  • Manual: Encode/verify with RSA-PSS signer cert; existing RSA and ECDSA SignedData tests unchanged.

Checklist

  • added tests
  • updated/added doxygen (PKCS7_RSA_PSS page; cryptocb RSA-PSS notes; pkcs7.h EncodeSignedData \ref)
  • updated appropriate READMEs (examples/configs/README.md; user_settings_pkcs7.h with WC_RSA_PSS)
  • Updated manual and documentation (doxygen; main README/ChangeLog at merge time)

@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 6 times, most recently from c4749c5 to 38bcb07 Compare February 6, 2026 10:15
@sameehj
Copy link
Contributor Author

sameehj commented Feb 6, 2026

retest this please

@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 16 times, most recently from a4ff167 to 2f8e307 Compare February 10, 2026 17:55
@sameehj
Copy link
Contributor Author

sameehj commented Feb 11, 2026

retest this please

@sameehj
Copy link
Contributor Author

sameehj commented Feb 12, 2026

retest this please

@sameehj sameehj requested review from SparkiDev and dgarske February 12, 2026 13:50
@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 2 times, most recently from cb9f6e4 to d4d412b Compare February 13, 2026 06:40
Copy link
Contributor

@dgarske dgarske left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also needs conflict resolved with rebase. Thanks

word32 rsapssOidSz = sizeof(sigRsaSsaPssOid);
/* MGF1 OID 1.2.840.113549.1.1.8 */
static const byte mgf1Oid[] = { 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x08 };
byte tmpBuf[128];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use a macro or enum that already exists for the 128? Also if its really that big you should have a WOLFSSL_SMALL_STACK option. We do have some helper macros for that... see WC_ALLOC_VAR_EX

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defined RSA_PSS_ALGOID_TMPBUF_SZ macro for the 128-byte buffer size. Added WOLFSSL_SMALL_STACK support using XMALLOC/XFREE for heap allocation of the temp buffer.

XMEMSET(pkcs7, 0, sizeof(wc_PKCS7));
pkcs7->isDynamic = (isDynamic != 0);
#if defined(WC_RSA_PSS) && !defined(NO_RSA)
pkcs7->pssSaltLen = -1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why -1 for the defaults on these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI, dropped!

* Use MAX_ENCRYPTED_KEY_SZ (512) to cover RSA keys up to 4096-bit. */
byte digest[MAX_ENCRYPTED_KEY_SZ];
RsaKey key[1];
DecodedCert stack_dCert;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you just use DecodedCert dCert[1];?

@dgarske dgarske requested a review from SparkiDev February 17, 2026 21:04
/* ASN tag for trailerField. */
#define ASN_TAG_RSA_PSS_TRAILER (ASN_CONTEXT_SPECIFIC | ASN_CONSTRUCTED | 3)
#endif
/* RSASSA-PSS-params EXPLICIT context tags (constructed, per X.680/X.690).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SparkiDev please review these changes specifically. Thanks


ExpectIntEQ(wc_InitRng(&rng), 0);
ExpectNotNull(pkcs7 = wc_PKCS7_New(HEAP_HINT, testDevId));
ExpectIntEQ(wc_PKCS7_Init(pkcs7, HEAP_HINT, INVALID_DEVID), 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wc_PKCS7_New already calls wc_PKCS7_Init internally, so this zeroes the struct again, discarding testDevId and replacing it with INVALID_DEVID.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Removed the redundant wc_PKCS7_Init after wc_PKCS7_New; testDevId is now preserved.

* Uses info->pk.rsa (in/inLen = digest, out/outLen = signature,
* key, rng). With WOLF_CRYPTO_CB_RSA_PAD, info->pk.rsa.padding
* supplies hash and salt length. */
ret = wc_RsaFunction(info->pk.rsa.in, info->pk.rsa.inLen,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wc_RsaFunction is the raw RSA private key operation (no PSS padding applied). A developer reading this as example code for WC_PK_TYPE_RSA_PSS would implement a callback that performs a raw unpadded RSA operation, producing a cryptographically invalid signature. The example should use wc_RsaPSS_Sign_ex (or at minimum include a prominent comment that PSS padding must be applied before/after the raw operation).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Example now uses wc_RsaPSS_Sign_ex (with PSS padding) instead of raw wc_RsaFunction.

* @return 0 on success.
* Uses manual (non-template) ASN.1 parsing for both WOLFSSL_ASN_TEMPLATE and
* non-template builds. The ASN template approach (rsaPssParamsASN /
* GetASN_Items) was removed because the EXPLICIT constructed context tags
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it work then.
The code was there and it needs to be made to work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't at some point, now it works

@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 2 times, most recently from db85770 to 1289ea8 Compare February 18, 2026 13:20
@dgarske dgarske requested review from SparkiDev and dgarske February 18, 2026 18:57

#include <wolfssl/wolfcrypt/aes.h>

#if defined(__clang__)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@douzzer thoughts on this? I am not sure I like adding a pragma here. Since you are working in aes.c right now curious your thoughts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to drop, this is usually triggered with -pedantic, let's see if the CI complains.

#define ASN_TAG_RSA_PSS_SALTLEN (ASN_CONTEXT_SPECIFIC | 2)
/* ASN tag for trailerField. */
#define ASN_TAG_RSA_PSS_TRAILER (ASN_CONTEXT_SPECIFIC | 3)
#define ASN_TAG_RSA_PSS_HASH (ASN_CONTEXT_SPECIFIC | 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to loose the original comments and formatting?

@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 4 times, most recently from dd416f8 to 2379563 Compare February 19, 2026 13:15
@dgarske dgarske self-requested a review February 19, 2026 18:00
@dgarske
Copy link
Contributor

dgarske commented Feb 19, 2026

Jenkins retest this please. FIPS v3 test failed history lost.

}

saltLen = pkcs7->pssParamsPresent
? pkcs7->pssSaltLen : RSA_PSS_SALT_LEN_DEFAULT;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make[2]: warning: -j9 forced in submake: resetting jobserver mode.
wolfcrypt/src/pkcs7.c: In function ‘wc_PKCS7_RsaPssVerify’:
wolfcrypt/src/pkcs7.c:4471:45: error: ‘RSA_PSS_SALT_LEN_DEFAULT’ undeclared (first use in this function)
 4471 |                       ? pkcs7->pssSaltLen : RSA_PSS_SALT_LEN_DEFAULT;
      |                                             ^~~~~~~~~~~~~~~~~~~~~~~~
wolfcrypt/src/pkcs7.c:4471:45: note: each undeclared identifier is reported only once for each function it appears in

https://cloud.wolfssl-test.com/jenkins/job/wolfSSL/job/PRB-fips-repo-and-harness-test-v3-part1/7210/

@sameehj
Copy link
Contributor Author

sameehj commented Feb 20, 2026

retest this please

@sameehj sameehj force-pushed the pkcs7-rsa-pss branch 2 times, most recently from 4304044 to 5db768e Compare February 20, 2026 12:28
Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData
encoding and verification.

This change enables SignerInfo.signatureAlgorithm to use
id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1,
salt length), as required by RFC 4055 and CMS profiles.

Key changes:
- Add RSA-PSS encode and verify paths for PKCS7 SignedData
- Encode full RSASSA-PSS AlgorithmIdentifier parameters
- Decode RSA-PSS parameters from SignerInfo for verification
- Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo)
- Fix certificate signatureAlgorithm parameter length handling
- Add API test coverage for RSA-PSS SignedData

This resolves failures when using RSA-PSS signer certificates
(e.g. -173 invalid signature algorithm) and maintains backward
compatibility with RSA PKCS#1 v1.5 and ECDSA.

Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
@sameehj
Copy link
Contributor Author

sameehj commented Feb 20, 2026

retest this please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments