Skip to content

fix: match AWS Secrets Manager ARNs against ExternalSecret/PushSecret friendly names - #366

Closed
zachbernstein-sdx wants to merge 1 commit into
external-secrets:mainfrom
zachbernstein-sdx:fix/aws-secretsmanager-arn-reference-matching
Closed

fix: match AWS Secrets Manager ARNs against ExternalSecret/PushSecret friendly names#366
zachbernstein-sdx wants to merge 1 commit into
external-secrets:mainfrom
zachbernstein-sdx:fix/aws-secretsmanager-arn-reference-matching

Conversation

@zachbernstein-sdx

Copy link
Copy Markdown

Problem

AwsSqs notification-source events set SecretIdentifier directly from CloudTrail's requestParameters.secretId (internal/listener/sqs/listener.go):

secretEvent := &events.SecretRotationEvent{
    SecretIdentifier:  event.Detail.RequestParameters.SecretId,
    ...
}

For PutSecretValue calls this is the secret's full ARN, including the random 6-character suffix Secrets Manager appends to every secret (e.g. arn:aws:secretsmanager:us-east-1:051826739313:secret:platform/ai-gateway/service-secrets-78YXTj). This isn't an edge case — both AWS's own built-in rotation Lambdas and manual PutSecretValue calls made via the console/CLI with the ARN as --secret-id populate requestParameters.secretId with the ARN.

The ExternalSecret/PushSecret destination handlers' default References() implementations (internal/handler/externalsecret/handler.go#L152-L179, and the analogous code in internal/handler/pushsecret/handler.go) compare this raw identifier against spec.dataFrom[].extract.key / spec.data[].remoteRef.key / spec.data[].match.remoteRef.remoteKey / spec.selector.secret.name using strict string equality.

Those fields conventionally hold the secret's friendly name — the standard, documented way to configure an ExternalSecret against a SecretsManager ClusterSecretStore (dataFrom.extract.key: platform/ai-gateway/service-secrets). Because the friendly name never string-equals the ARN, the comparison never succeeds for AWS-sourced events. The destination is silently skipped ("skipping object as its not referenced") even though the listener, auth, and event-delivery pipeline are all working correctly — in practice, any ExternalSecret backed by AWS Secrets Manager and referenced by friendly name never gets reloaded via the AwsSqs source.

(We noticed the CRD/config docs describe a matchStrategy with Contains/RegularExpression operations that could work around this, but confirmed in both the v0.1.0 and v0.2.0 release tags that matchStrategy is still a stub in processor.go — logged and ignored — so it isn't currently usable as a workaround.)

Fix

Add internal/util.SecretIdentifierMatches (backed by SecretIdentifierAliases), which expands an AWS Secrets Manager ARN into its canonical friendly name (per AWS's documented ARN suffix behavior) before comparing, while still matching non-ARN identifiers by strict equality as before — including the case where a spec already stores the full ARN, so existing configurations are unaffected. Wire this into the _references implementations in both the ExternalSecret and PushSecret handlers, replacing the raw == checks.

This is scoped narrowly (only identifiers that look like arn:aws*:secretsmanager:...:secret:<name>-<6-char-suffix> are expanded; everything else falls through to the original exact-match behavior), so it shouldn't change behavior for any other notification source or provider.

Testing

  • New table-driven unit tests in internal/util/secretid_test.go covering standard/GovCloud/China partitions, friendly names that themselves contain hyphenated 6-char segments, non-ARN identifiers, and non-SecretsManager ARNs.
  • New/extended unit tests in internal/handler/externalsecret/handler_test.go and a new internal/handler/pushsecret/handler_test.go verifying an ExternalSecret/PushSecret configured with a friendly name is now considered referenced when the event identifier is the full ARN.
  • Ran the full suite locally, including the internal/controller envtest integration specs (go test ./internal/controller/... via setup-envtest) — all green, no regressions.

Happy to adjust naming/placement of the helper or expand test coverage if maintainers have a preference — opened as draft to get early signal on whether this approach (alias-expansion inside the handler's reference check) matches how you'd want this solved vs., e.g., normalizing at the AwsSqs listener level or finishing the matchStrategy wiring instead.

… friendly names

AwsSqs notification source events carry event.Detail.RequestParameters.SecretId
verbatim from CloudTrail, which for PutSecretValue calls is the secret's full
ARN (including the random 6-character suffix Secrets Manager appends), not its
friendly name. Both AWS's own built-in rotation Lambdas and manual
PutSecretValue calls made via the console/CLI with the ARN as --secret-id
populate this field with the ARN.

The ExternalSecret/PushSecret destination handlers' default References()
implementations compared this raw identifier against
spec.dataFrom[].extract.key / spec.data[].remoteRef.key /
spec.data[].match.remoteRef.remoteKey / spec.selector.secret.name using strict
string equality. Since those fields conventionally hold the secret's friendly
name (the common, documented way to configure an ExternalSecret against a
SecretsManager ClusterSecretStore), the comparison never succeeded for
AWS-sourced events. The destination was silently skipped
("skipping object as its not referenced") even though the listener, auth and
event delivery pipeline all worked correctly - so, in practice, any
ExternalSecret backed by AWS Secrets Manager and referenced by friendly name
never got reloaded.

Add internal/util.SecretIdentifierMatches (backed by
SecretIdentifierAliases), which expands an AWS Secrets Manager ARN into its
canonical friendly name before comparing, while still matching a raw
identifier by strict equality otherwise (including when a spec already uses
the full ARN, preserving existing behavior). Wire this into both the
ExternalSecret and PushSecret handlers' _references implementations.

Covered by new unit tests in internal/util and the externalsecret/pushsecret
handler packages, plus the existing internal/controller envtest suite (still
green).
@zachbernstein-sdx
zachbernstein-sdx deleted the fix/aws-secretsmanager-arn-reference-matching branch July 27, 2026 19:55
@zachbernstein-sdx

Copy link
Copy Markdown
Author

Superseded by #367

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant