Repository navigation
fix: keep current key pair when key and certificate mismatch - #16
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
Security-sensitive reload ordering needs deterministic concurrency coverage and final human validation.
1 open finding
What changed in this PR
Protects certificate rotation by retaining the current KeyManager when a replacement RSA key and certificate do not match.
Changes:
- Serializes reloads and adds certificate identity and expiry logging.
- Adds reload tests and documents warning recovery.
- Advances the development version and adds locking annotations.
| File | Description |
|---|---|
| src/test/java/org/cloudfoundry/security/FileWatchingX509ExtendedKeyManagerTest.java | Tests matching, mismatched, and empty certificate reloads and chain expiry. |
| src/main/java/org/cloudfoundry/security/FileWatchingX509ExtendedKeyManager.java | Serializes reloads, checks replacement pairs, and logs certificate details. |
| README.md | Documents rotation behavior, warnings, and recovery. |
| pom.xml | Sets 1.22.0-SNAPSHOT and adds compile-time locking annotations. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ac0e468 to
e2557a9
Compare
- Check the private key matches the leaf certificate before replacing the KeyManager; keep the current one otherwise, also for an empty certificate file. Non-RSA keys are accepted with an info message. - Serialize reloads so the shared KeyManagerFactory is never initialized concurrently. - Log certificate serial, subject and chain expiry (UTC) on load; warn when the chain has expired. - Remove the commented-out private key watcher and document why only the certificate file is watched. - Document certificate rotation and mismatch recovery in the README. - Bump revision to 1.22.0-SNAPSHOT.
- Warn when the first loaded pair mismatches; there is no current pair to keep, so it is installed as before - Hint at the likely cause in the reload mismatch warning - Link the Diego executor source for the key-before-certificate order - Replace files atomically in the watcher test, fail within 30s
e2557a9 to
dd59b75
Compare
jcvrabo
left a comment
There was a problem hiding this comment.
I would argue that mismatched errors should be logged as errors instead of warnings (keeping a soon-to-be-expired certificate will keep valid for about 1 hour and assuming that the issue will remain, the app will never really have a valid certificate again).
I would also wonder if in case the app starts without a valid key/certificate pair there is any point in actually loading a (useless) certificate/key pair. Since that is an edge case which would come from a deeper problem with the diego cell, I would just log the error (as error, not a warning and not throwing exceptions as the app might not even need the certificate to operate) and leave the keymanager uninitialized.
- On first load, don't install a mismatched pair or an empty certificate file; delegate methods return null, so other TLS connections keep working and the next matching rotation installs it - Log mismatches and expired chains as SEVERE (ERROR via SLF4J) - README: describe the new first-load behaviour and error levels Refs cloudfoundry#16 (review by jcvrabo)
|
@jcvrabo thanks, both adopted in faa3593:
Compared with 1.21.1 this is never worse:
An opt-in health signal (for example a JMX MBean that an app health check could read, triggering a restart) could be a follow-up issue. |
There was a problem hiding this comment.
🟡 Changes recommended
An in-flight watcher log publication can race with iteration of LogRecorder’s backing ArrayList, causing intermittent ConcurrentModificationException failures in the test.
1 open issue
1 resolved since last review
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed because the tests do not verify the expired-chain diagnostic or its severity, despite its documented role in operational alerting.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
The watcher thread can still publish a log record while the test reads the recorded records, which could throw ConcurrentModificationException.
74e3e2a to
fabe771
Compare
getClientAliases can return null or an empty array, so the helper no longer indexes it blindly. It retries a bounded number of times, because the watcher can swap the key manager between the alias and chain calls, and then fails with a descriptive message.
There was a problem hiding this comment.
🟡 Changes recommended
Replacing the watched certificate can trigger an asynchronous reload after currentAlias is captured, causing intermittent assertion failures because the explicit update() does not wait for the watcher callback.
1 open issue
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed because the unchecked fallback can accept mismatched RSA/non-RSA key and certificate pairs, bypassing safeguards that preserve the current KeyManager.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
|
On the Copilot note about the unchecked non-RSA fallback in matches(): left as is on purpose. The Diego executor only generates RSA keys (credmanager.go uses *rsa.PrivateKey), and key and certificate are created together, so a mixed RSA/non-RSA pair would need an algorithm switch in Diego to coincide with the rename window during rotation. If Diego ever moves to EC or post-quantum keys, both change together and the existing INFO log ("not an RSA key ... the security provider probably needs an update") signals that a check for the new algorithm should be added. |

Ties up the loose ends of #8 after #12. Before replacing the KeyManager, the private key is checked against the leaf certificate (RSA, the type the Diego executor issues). On a reload with a mismatch or an empty certificate file the current pair is kept; on the first load an invalid pair is not installed, so no client certificate is presented but other TLS connections keep working. Both cases, and an expired chain, are logged as errors (JUL
SEVERE). Reloads are serialized, each load logs the certificate serial, subject and chain expiry, and the commented-out key watcher is replaced by a note on why only the certificate is watched. Supersedes #9 and #10.Note: a mismatch is not retried and does not fail fast; see the new "Certificate rotation" section in the README for alerting and recovery.
Draft: planned for 1.22.0, after a 1.21.1 release with the recent dependency updates, as these changes need more review and testing.
Note: if a reload happens between choosing a client alias and looking up its chain or key, the lookup returns null. This is a pre-existing, very small window. It is logged at SEVERE with the prefix
Should not happen:, so we can see whether it ever occurs. If it does, a possible fix is to tag aliases with a generation number and keep the previous KeyManager for a short grace period. That is a heavier change, so it is left out of this PR.