Unify weekly + per-PR security scanning into a single workflow

[vikrantpuppala]

Summary

Consolidates security scanning into one workflow with one job (securityScan.yml). Replaces vulnerabilityCatcher.yml. Adds OSV-Scanner alongside the existing OWASP check, plus cyclonedx SBOM generation so OSV scans the actually-resolved local dependency tree.

This bundles the work that was originally split across (now-closed) PRs #1458 (suppressions + email-notification fix) and #1459 (per-PR gate). One unified workflow file, one set of suppression rules, one place to look.

Design: single job, terminal steps gated by event

Triggers: pull_request, schedule (weekly cron), workflow_dispatch
   │
   ├─ shared: checkout, JDK, JFrog OIDC, maven config
   ├─ shared: NVD database cache (saves ~2 min)
   ├─ shared: mvn package (generates cyclonedx aggregate SBOM)
   ├─ shared: OWASP dependency-check (continue-on-error)
   ├─ shared: install + run OSV-Scanner
   ├─ shared: Collect findings (writes job summary; sets outputs)
   │
   ├─ if pull_request + findings ⇒ fail the job
   └─ if schedule|dispatch + findings ⇒ send email + fail the job

No duplicated steps across jobs. Same scan logic for every trigger; only the notification mechanism differs.

What changed

File	Change
.github/workflows/vulnerabilityCatcher.yml	Removed. Superseded by securityScan.yml.
.github/workflows/securityScan.yml	New. Single job with event-gated terminal steps. Includes NVD database caching for faster runs.
owasp-suppressions.xml	8 new entries for documented CPE/ecosystem false positives — Arrow R-only (CVE-2024-52338), gRPC-Go-only (CVE-2026-33186), protobuf Python-only (CVE-2026-0994), 5 libthrift non-Java bindings. Each entry has a justification comment block.
osv-scanner.toml	New. Mirrors the OWASP suppressions in TOML format. Keep the two files in sync when adding/removing entries.
pom.xml	cyclonedx-maven-plugin 2.9.1 added, bound to package phase, skipNotDeployed=false. Emits an aggregate target/bom.json for OSV to read.

Why two scanners

Scanner	Source	Catches	Misses
OWASP dependency-check	NVD (CPE-based)	Anything with an NVD CPE entry. Reuses existing owasp-suppressions.xml and failBuildOnCVSS=7 threshold.	GHSA-only advisories without an NVD CPE (e.g., CVE-2025-66566 in lz4-java — the #1455 finding).
OSV-Scanner v2.3.8	OSV.dev (purl-based; federates GHSA + NVD + PyPA + RustSec)	The GHSA-only gap. Already turned up a new finding: bouncycastle CVE-2026-5598 (severity 8.9) in bcprov-jdk18on 1.79, invisible to OWASP.	Findings without a CVSS score (rare for high-severity).

Why cyclonedx SBOM

OSV-Scanner's Maven resolver consults deps.dev's published-artifact metadata. For the project's own GA, that means it sees the previously-published 3.3.3 pom — not whatever bumps are sitting on main. Result: in CI, OSV would keep showing yesterday's released state instead of the PR branch's actual dependency tree. Producing a CycloneDX aggregate SBOM at build time captures the actually-resolved local tree.

Fixes from #1458 included here

The weekly scan has been failing every Sunday for 7+ consecutive weeks (since early April 2026) without sending a notification email. Root cause: the OWASP maven plugin step exits non-zero when CVEs are found (because <failBuildOnCVSS>7</failBuildOnCVSS> is set in jdbc-core/pom.xml:357), short-circuiting the rest of the job. The subsequent steps never ran.

The new workflow has continue-on-error: true on the OWASP step and routes finding-detection through an explicit Collect findings step that reads the OWASP JSON report directly. Whether the job ultimately fails (and whether email goes out) is decided after that step, not by maven's exit code.

Findings on main today

This workflow, run against current main, reports unsuppressed findings that will be addressed in separate follow-up PRs:

1. `org.bouncycastle:bcprov-jdk18on@1.79` → CVE-2026-5598 (severity 8.9). Fixed in bouncycastle 1.84. OWASP doesn't see this — OSV-only finding.
2. `org.apache.thrift:libthrift@0.19.0` → 4 CVEs (CVE-2026-41603/04/05/43869, max severity 8.2). Cleared by the libthrift 0.19 → 0.23 follow-up (requires regenerating `TCLIService.java` with the matching compiler).

So this PR's own CI is expected to fail on these 2 known findings until the follow-ups land. Aligned with the rollout plan: add the workflow now, mark it required-to-merge in branch protection after the burn-down.

Expected runtime

Each invocation: ~5–7 minutes (vs ~4 min for the old weekly). Breakdown:

• mvn package (with cyclonedx SBOM): ~45–90s
• OWASP dependency-check (with NVD cache hit): ~1–2 min
• OSV-Scanner install + run: ~15s
• Other steps: ~30s

NVD caching cuts ~2 minutes off OWASP's runtime when the cache is warm. The cache key rotates daily so we still get fresh CVE data.

What this PR is NOT

• It is not a required check. Branch protection is unchanged — PRs aren't blocked by the red ❌ yet.
• It does not introduce diff-mode (net-new findings only) gating. Every PR has to ship into a clean codebase; this is the design we agreed on, to force burn-down rather than just "not making it worse."
• It does not include a Phase 2 design (override flags, severity tiers, auto-issue filing). Phase 2 is deferred until we have ~2 weeks of operational experience.

Test plan

• `mvn package -DskipTests -Ddependency-check.skip=true` produces `target/bom.json` (69 components, all expected versions).
• `mvn -pl jdbc-core org.owasp:dependency-check-maven:check` reports 4 unsuppressed CVEs (all libthrift; matches expectation).
• `osv-scanner scan source --config=osv-scanner.toml --format=json target/bom.json` + severity≥7 filter reports 2 findings (bouncycastle + libthrift cluster).
• Findings-step jq logic correctly counts OWASP CVSS>=7 from the JSON report.
• XML validation of `owasp-suppressions.xml`.
• YAML validation of `securityScan.yml`.
• CI run on this PR — expected to fail on the 2 known findings above. The CI failure on this PR is the intended outcome.
• Manual `workflow_dispatch` run post-merge to verify the email path fires.

NO_CHANGELOG=true

This pull request and its description were written by Isaac.

[gopalldb]

🔴 Major #1: pr-scan step ordering risk — OSV depends on SBOM but Build step has no if: always().

• name: Build (generates cyclonedx SBOM)
  run: mvn package -DskipTests -Ddependency-check.skip=true -B

• name: Run OWASP Dependency Check
  id: owasp
  run: mvn -pl jdbc-core org.owasp:dependency-check-maven:check ...

• name: Install osv-scanner
  if: always()

• name: Run OSV-Scanner
  id: osv
  if: always()

If the Build step fails (Maven compile error, JFrog flake), target/bom.json won't exist. OSV-Scanner runs with if: always() and tries to scan a missing file → osv-scanner ... target/bom.json || true swallows
the error → the jq query on an empty/non-JSON file → cryptic failure. Not the end of the world but worth a guard:

• name: Run OSV-Scanner
  id: osv
  if: always()
  run: |
  if [ ! -f target/bom.json ]; then
  echo "::warning::SBOM not present (build likely failed); skipping OSV-Scanner"
  exit 0
  fi
  ...

🔴 Major #2: OWASP failBuildOnCVSS=7 will fail the pr-scan owasp step short-circuiting osv only if osv lacks if: always().

Good news: osv does have if: always(). But the implication is that on findings, the pr-scan step owasp exits non-zero, the job reports failure, AND osv still runs and may report additional findings. So PR
authors get the union of both. Correct behavior.

However: the OWASP step exit code is the only signal for OWASP findings in the PR job. The osv step explicitly filters severity≥7 and emits ::error:: + exit 1. There's no symmetric severity filter for OWASP
— it's gated entirely by the pom.xml failBuildOnCVSS config. If the suppression file is misconfigured (parse error), OWASP may exit 0 with no findings and PRs would silently pass. Add an explicit check that
the OWASP report file exists and that the scan ran to completion:

• name: Verify OWASP ran
  if: always()
  run: test -f jdbc-core/target/dependency-check-report.json

🟡 Minor #3: set -uo pipefail (no -e) in OSV step is intentional but worth a comment.

• name: Run OSV-Scanner
  run: |
  set -uo pipefail
  /tmp/osv-scanner scan source ... || true

The || true swallows OSV's non-zero exit (OSV exits 1 on ANY finding regardless of severity), and the severity≥7 filter does the actual gating. Reasonable. But anyone reading this for the first time will
wonder why -e is missing. One-line comment:

Drop -e: osv-scanner exits 1 on any finding (severity-blind);

we do our own severity filter below.

set -uo pipefail

🟡 Minor #4: jq severity filter uses tonumber? // 0 — silently treats malformed severity as 0 (no finding).

select((.max_severity | tonumber? // 0) >= 7)

If OSV ever changes its severity format to a string ("HIGH"), tonumber? returns null → // 0 → severity 0 → finding ignored. Probably fine since OSV uses CVSS numerics, but the silent fail-open is a footgun.
Consider logging when filter fails:

TOTAL_FINDINGS=$(jq '[.results[].packages[]? | .groups[]?] | length' /tmp/osv-out.json)
echo "OSV-Scanner: $TOTAL_FINDINGS total findings, $COUNT at CVSS>=7"

This way if 50 findings are detected but the severity filter shows 0, ops people will notice the mismatch.

🟡 Minor #5: osv-scanner.toml is keyed by CVE ID; not scoped by package.

[[IgnoredVulns]]
id = "CVE-2026-33186"
reason = "..."

Unlike the OWASP regex which scopes the suppression to a specific artifact, OSV's IgnoredVulns is CVE-id global. If a future dependency picks up the same CVE id legitimately (e.g., a non-Java
grpc artifact gets added later that's actually vulnerable), it'll be silently suppressed.

OSV does support [[PackageOverrides]] for package-scoped overrides — worth using:

[[PackageOverrides]]
name = "io.grpc:grpc-api"
ecosystem = "Maven"
vulnerability.id = "CVE-2026-33186"
ignore = true
reason = "..."

Or at minimum, document this trade-off in the file header so future-readers know suppressions are CVE-id global.

---

Coverage Gaps

🟡 #6: No SBOM for databricks-jdbc-thin (uber jar).

The cyclonedx aggregate BOM is from mvn package of the multi-module reactor. But the published artifact is the shaded uber jar (databricks-jdbc:3.x) which has different effective dependencies (relocated,
some excluded). If a vulnerability is in a transitive that gets excluded by shading, it'll show in the BOM but not actually ship. Conversely, if shading includes something not in the reactor BOM, it'd be
missed.

Not blocking for this PR — but worth a follow-up: also scan the uber jar via osv-scanner scan source .

🟡 #7: PR scan doesn't run on forks (no JFrog OIDC).

• name: Get JFrog OIDC token
  if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository

OK on its own — fork PRs would fail OIDC token mint. But then the subsequent Build (generates cyclonedx SBOM) step has no if: guard and will try to run without Maven settings configured. It might still work
if no JFrog-only artifacts are needed (cache hit), but it's a structural gap. Either:

• Skip the entire pr-scan job for forks: if: github.event.pull_request.head.repo.full_name == github.repository
• Or have a fork-path that uses public Maven Central directly

🟡 #8: SBOM is committed to artifact but not the repo.

The aggregate BOM is generated fresh per CI run. That's fine for the scanner, but: if a customer / security team wants to independently audit the driver's SBOM for a specific release, they have no way to
access it post-build. Consider attaching bom.json to GitHub releases as a release asset.

Not a blocker for this PR — separate "publish SBOM" PR worth filing.

---

Maintainability

🟢 Suppression sync — the PR header acknowledges this:

▎ "Keep the two files in sync when adding/removing entries."

But there's no automation to enforce it. Worth a follow-up CI check that parses both files and validates that every entry in osv-scanner.toml has a corresponding X in owasp-suppressions.xml and
vice versa.

🟡 #9: Duplication between pr-scan and weekly-scan jobs.

The two jobs duplicate ~120 lines (JFrog OIDC + Maven settings + checkout + JDK setup). Use a reusable composite action or a shared setup-and-build job that both depend on. Today the duplication is fine; if
a third trigger gets added (e.g., release scan), the maintenance cost compounds.

[vikrantpuppala]

@gopalldb thanks for the thorough review — really helpful framing. Quick map of where each point lands given two changes that happened since you wrote it:

Already addressed before your review landed (force-push):

• (9) duplication between pr-scan and weekly-scan jobs: I collapsed the two jobs into a single `security-scan` job in commit `4c4864c` (shared steps; terminal steps gated by `github.event_name`). Same workflow file, no duplication. Worth a re-read from the top.

Direction change after your review (next push):
After your review, we decided to drop OWASP entirely from this workflow and use OSV-Scanner as the sole gate. Two reasons:

1. Coverage: OSV's database is a strict superset of NVD's (federates GHSA/NVD/PyPA/RustSec). In testing we found OSV catches two real CVEs that OWASP misses entirely — CVE-2025-66566 in `at.yawk.lz4:lz4-java` and CVE-2026-5598 in bouncycastle — because they're GHSA-only with no NVD CPE entry. Haven't seen the reverse case.
2. Operational cost: OWASP needs the NVD database (~150–700 MB), the NVD_API_KEY secret, the cache machinery, and minutes per run even with warm cache. OSV is a single ~5s call.

The release pipeline at `databricks/secure-public-registry-releases-eng` already explicitly skips OWASP (`-Ddependency-check.skip=true` in every Maven step) and runs its own artifact-level scan via `databricks/gh-action-scan`. So removing OWASP from this repo's CI doesn't disturb release gating.

That mootss your (2) (OWASP silent-success risk) and (9 part 2) (suppression sync between owasp-suppressions.xml and osv-scanner.toml).

Will fix in the next push:

• (1) (SBOM existence guard before OSV runs) — adding `if [ ! -f target/bom.json ]` check at the top of the OSV step.
• (3) (`set -uo pipefail` needs a comment explaining why no `-e`) — adding a one-line comment.
• (4) (severity filter silently treats malformed scores as 0) — adding a "total findings vs filtered findings" log line so mismatches are visible.
• (5) (suppressions are CVE-id global, not package-scoped) — good catch, this gets more important when OSV is the sole gate. Refactoring `osv-scanner.toml` to use `[[PackageOverrides]]` per entry, scoped by ecosystem+name.
• (7) (fork PR maven config) — actually a non-issue in practice: all our direct deps are public Maven Central artifacts (verified by walking `jdbc-core/pom.xml`). JFrog is just a mirror/proxy for speed; without it, Maven falls through to Central and resolves fine. I'll add a clarifying comment in the workflow so future-readers don't have to re-derive this.

Acknowledged as follow-ups (out of scope for this PR):

• (6) (SBOM is from `mvn package` reactor, not the shaded uber jar — shading drops/relocates deps so the SBOM may diverge from what actually ships). Real concern. Filing as a separate issue to scan the uber JAR directly post-build.
• (8) (SBOM not published anywhere customer-facing). Filing as a separate issue to attach `bom.json` to GitHub releases.

Pushing the OSV-only + review-fix commit shortly.

[vikrantpuppala]

Quick correction on (5) (suppression scoping) from my earlier reply: I intended to use `[[PackageOverrides]]` for per-package CVE-id scoping, but OSV-Scanner v2.3.8 actually doesn't support that intersection. The `PackageOverrideEntry.Vulnerability` struct in `internal/config/config.go` has only an `ignore` boolean field, no per-CVE id. So `[[PackageOverrides]]` blanket-ignores all vulns on the matched packages (way too broad), and `[[IgnoredVulns]]` stays the only viable per-CVE option (which is global across all packages).

Kept the `[[IgnoredVulns]]` approach but added a detailed comment block in `osv-scanner.toml`'s header documenting the trade-off and the acceptable-risk reasoning -- if a future Maven dep ever legitimately picks up one of the CVE IDs we've suppressed, the suppression would silently apply to it too. Mitigated because every entry we have is ecosystem-mismatched (the Go/Python/R/C_glib binding of a library, not the Java binding), so a legitimate Java affectation would itself be a notable advisory event we'd want to revisit.

If you'd prefer we eat the broader `[[PackageOverrides]]` blanket on `org.apache.thrift:libthrift` (since that's the only package family with multiple suppressed CVEs in our config), I can do that -- but for the cross-family ones (Arrow, gRPC, protobuf) it'd still be one CVE per family so the distinction collapses anyway.