docs: record that lldap's restic mount is hard NFS, and correct a bad finding - #380
Merged
Conversation
… finding An untracked gitops/workloads/lldap/restic-pv.yaml was spotted on the H4 and recorded in BACKLOG as a live PV that urgently needed committing. It was not. kubectl get pv lldap-restic returns NotFound: the file was an unapplied draft, and it has since vanished from the working tree. The claim came from reading its contents without checking whether the object existed -- the same mistake as trusting a green timer, in miniature. But the draft existed for a real reason, now recorded as 1.8. The lldap backup CronJob mounts the restic repo with an inline nfs: volume, and inline NFS volumes cannot carry mountOptions -- that is a PV-only field. So it mounts hard, and an unreachable NFS export blocks the pod indefinitely instead of erroring. That is the 2026-08-02 incident where lldap-backup hung for 3d15h. It was bounded, not fixed: activeDeadlineSeconds: 1800 kills the job after 30 minutes and LabBackupJobFailed pages when it does, so a backup can no longer stall silently for days. The stall itself remains. Fixing it means a PV plus PVC and switching the volume type, which is a change to a backup path that has run reliably since the deadline was added -- recorded as a decision, not scheduled.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An untracked gitops/workloads/lldap/restic-pv.yaml was spotted on the H4 and
recorded in BACKLOG as a live PV that urgently needed committing. It was not.
kubectl get pv lldap-restic returns NotFound: the file was an unapplied draft,
and it has since vanished from the working tree. The claim came from reading its
contents without checking whether the object existed -- the same mistake as
trusting a green timer, in miniature.
But the draft existed for a real reason, now recorded as 1.8. The lldap backup
CronJob mounts the restic repo with an inline nfs: volume, and inline NFS volumes
cannot carry mountOptions -- that is a PV-only field. So it mounts hard, and an
unreachable NFS export blocks the pod indefinitely instead of erroring. That is
the 2026-08-02 incident where lldap-backup hung for 3d15h.
It was bounded, not fixed: activeDeadlineSeconds: 1800 kills the job after 30
minutes and LabBackupJobFailed pages when it does, so a backup can no longer
stall silently for days. The stall itself remains. Fixing it means a PV plus PVC
and switching the volume type, which is a change to a backup path that has run
reliably since the deadline was added -- recorded as a decision, not scheduled.