Skip to content

perf(depscan): skip a scheduled rescan when OSV answers the same - #2685

Merged
Makisuo merged 2 commits into
mainfrom
perf/dep-scan-skip-unchanged
Nov 16, 2026
Merged

Makisuo merged 2 commits into
mainfrom
perf/dep-scan-skip-unchanged

Conversation

@Makisuo

@Makisuo Makisuo commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The scheduled dependency scan runs every 12 h (VcsScheduledSyncService). It queues one dependency-scan (trigger schedule) per active repository on its tracked or default branch. Almost every one of them repeats work it did 12 h earlier.

Production, burst of 2026-11-15 11:30-13:30Z: 3,784 scheduled scans across 3,784 repositories in 33 orgs. p50 1.72 s, 6,979 s of DependencyScanner.scan in total. 1,064 of them (28%) found no lockfile and still ran the whole path. That is about 11k scans a day, and about 97% are at the same commit as 12 h before.

Where the time went (11:50-12:50Z, 3,228 scans, 5,987 s):

Span Total
osv.querybatch 2,584 s
getCommit (2 per scan) 2,309 s
getRecursiveTree 1,012 s
DependencyScanStore.completeScan 300 s
blob reads ~0 (parse cache by blob sha)

Cause

  • DependencyScanner.ts skipped an unchanged scan only for trigger === "push", by comparing lockfile blob shas. That check cannot be used for schedule: it would hide newly published advisories.
  • The "commit already scanned" check only applied with an explicit commitSha.
  • So every scheduled scan did all of this: two resolveRef calls (the head, then the head-moved check), the recursive tree listing, OSV querybatch, the vulnerability record reads, startScan and completeScan.

Fix

A vulnerability record never changes for one id@modified. The same querybatch answer for the same packages therefore means the same matches and the same findings.

  1. dependency_scans.advisory_fingerprint (nullable text, migration 20261116082407_dependency_scan_advisory_fingerprint). completeScan stores it only when the scan has no incomplete reason, so a scan that never reconciled is never skipped on.
  2. PackageAdvisoryCatalog is split into query (querybatch: id and modified per package) and resolve (record reads). advisories is query then resolve. fingerprintOf is a SHA-256 over the sorted packageKey -> [id@modified] of every queried package, plus PARSER_VERSION.
  3. A commit's lockfile selection (selected, skipped, and the manifests each yarn lockfile reads) is a serialisable plan. It is cached in the edge cache by provider:externalRepoId:commitSha. A commit is immutable. A truncated tree is never cached.
  4. New branch for trigger === "schedule" on the default branch head, after the first resolveRef. It reads the latest completed scan of the head. If that scan has the same commit, no error and a fingerprint, the branch loads the plan (cached), reads the packages (parse cache) and calls catalog.query.
    • Same fingerprint: returns Unchanged and annotates maple.depscan.outcome=unchanged, maple.depscan.skip_reason=advisories_unchanged (no_lockfiles when the commit has none). No tree fetch, no second resolveRef, no startScan or completeScan.
    • Different fingerprint: runs the full scan and reuses the answer it already fetched. OSV is called once.
    • Any precheck failure: logs it and falls back to the full scan, which records the failure on its row as before.
  5. Manual, push and production scans are unchanged. The 12 h schedule is unchanged.

Security invariant: OSV is still asked on every scheduled scan of a repository that has packages. A new advisory id, or a new modified time on an existing one, changes the fingerprint and runs a full scan that stores the match and opens the finding. Tests cover both cases. A commit with no packages has nothing to ask OSV about, so it skips without a call.

Possible follow-up: no checked_at column in this PR. "Last scanned" stays at the time of the commit's last full scan, even though advisories are rechecked every 12 h.

Before / After

Metric (one unchanged scheduled scan) Before After Change
resolveRef (GitHub getCommit) 2 1 -50%
fetchTreeFiles (getRecursiveTree) 1 0 (plan cached by commit) -100%
OSV querybatch 1 1 flat
OSV record reads per vuln (cache hits) 0 -100%
startScan + completeScan (Postgres) 1 + 1 0 -100%
Unchanged commit with no lockfiles: OSV calls 0 0 flat
Metric (production, per 12 h burst) Before After (expected)
Scheduled DependencyScanner.scan total ~7.0k s ~3.0k s
getRecursiveTree count ~3.8k about -95%
getCommit count ~7.6k about -50%
osv.querybatch count ~3.8k flat

Method:

  • The production BEFORE numbers come from production traces, DependencyScanner.scan and child spans, 2026-11-15 11:30-13:30Z (the 12:00 UTC burst).
  • The local per-scan counts come from DependencyScanner.test.ts. It runs the real scanner wiring on PGlite (createTestDb) and the in-memory edge cache, with a stub provider and a stub OSV that count calls. The "Before" row is the first scheduled scan, which still takes the full path. The "After" row is the second scheduled scan of the same commit.
  • The production AFTER numbers are expectations. The volume only exists in the 00/12 UTC scheduled burst, and a preview cannot reproduce it, so this PR has no preview label.

How to verify after deploy

From the second burst after deploy (the first fills the new column):

SELECT SpanAttributes['maple.depscan.trigger'] trig, SpanAttributes['maple.depscan.outcome'] outcome, SpanAttributes['maple.depscan.skip_reason'] reason, count() c, round(quantile(0.5)(Duration)/1e6) p50ms, round(sum(Duration)/1e9) sumS FROM traces WHERE $__orgFilter AND $__timeFilter(Timestamp) AND ResourceAttributes['deployment.environment']='production' AND SpanName='DependencyScanner.scan' GROUP BY 1,2,3 ORDER BY c DESC

Expect:

  • About 97% of schedule rows are unchanged / advisories_unchanged (or no_lockfiles).
  • The scheduled sumS drops from ~7.0k s to ~3.0k s per burst.
  • getRecursiveTree count drops about 95% and getCommit about 50%.
  • The osv.querybatch count stays flat. That proves advisories are still checked every 12 h.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Devin Review

Summary by CodeRabbit

  • Improvements
    • Scheduled scans can detect new or modified advisories even when the default branch’s commit has not changed.
    • When the commit and its advisory information are unchanged, a scheduled scan can skip the full scan.
    • Advisory changes trigger a full scan. Incomplete, unsuccessful, or otherwise ineligible scans do not qualify for the skip, allowing a subsequent scan to proceed.

Every 12 h each active repository gets a scheduled dependency scan at its
tracked head. About 97% hit the commit already scanned 12 h before, and
each one still resolved the head twice, listed the tree, asked OSV,
read the records and wrote a new scan with its matches.

A completed scan now stores a fingerprint of the OSV answer it matched
against (sorted package -> id@modified, plus the parser version). A
scheduled scan of the same commit loads the commit's lockfile plan from
the edge cache, reads the cached parses, asks OSV, and returns Unchanged
when the fingerprint matches. A different answer runs the full scan with
the answer already fetched. OSV is still asked on every scheduled scan
of a repository with packages. Manual and push scans are unchanged.
@maple-review-bot

This comment has been minimized.

@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The scanner caches lockfile plans by commit and compares OSV advisory fingerprints during eligible scheduled scans. It skips a scan when the commit and fingerprint match. Completed scans store the fingerprint; incomplete scans store null.

Changes: Scheduled scan advisory fingerprints

Layer / File(s) Summary
Query and fingerprint advisories
packages/backend/src/services/security/PackageAdvisoryCatalog.ts, packages/backend/src/services/security/PackageAdvisoryCatalog.test.ts
The catalog separates advisory queries from record resolution. It computes fingerprints from sorted package keys, advisory IDs, modification times, and parser version. Tests cover order independence and changes to those inputs.
Persist completed-scan fingerprints
packages/db/src/tables/dependency-scan.ts, packages/db/drizzle/20261116082407_dependency_scan_advisory_fingerprint/*, packages/backend/src/services/dependency-scan/DependencyScanStore.ts
The schema adds a nullable fingerprint column. The scan store initializes it to null and saves a supplied fingerprint only when a scan completes. latestScan can include scans of any status when requested.
Plan and precheck scheduled scans
packages/backend/src/services/dependency-scan/DependencyScanner.ts, packages/backend/src/services/dependency-scan/DependencyScanner.test.ts
The scanner caches commit lockfile plans and uses planned files for reads. An eligible scheduled scan queries advisories and returns Unchanged when the fingerprint matches. Otherwise, it reuses the prepared query and reads for a full scan. Tests cover rescans, fallback conditions, and scan modes.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant DependencyScanner
  participant DependencyScanStore
  participant PackageAdvisoryCatalog
  participant OSV
  DependencyScanner->>DependencyScanStore: Read latest scan for the commit
  DependencyScanner->>PackageAdvisoryCatalog: Query advisories for planned lockfiles
  PackageAdvisoryCatalog->>OSV: Request vulnerability IDs and modification times
  OSV-->>PackageAdvisoryCatalog: Return vulnerability references
  PackageAdvisoryCatalog-->>DependencyScanner: Return advisory query
  DependencyScanner->>DependencyScanner: Compare query fingerprint with stored fingerprint
  DependencyScanner->>PackageAdvisoryCatalog: Resolve advisory records when a full scan is needed
  DependencyScanner->>DependencyScanStore: Complete scan with advisory fingerprint
Loading

Possibly related PRs

  • MapleTechLabs/maple#2043: Defines the dependency-scan storage tables and store used to save completed-scan fingerprints.
  • MapleTechLabs/maple#2044: Defines the scanner flow updated here to precheck scheduled scans and compare advisory fingerprints.
Merge Risk: Minimal · up to 42677

Scheduled dependency scans now skip the full rescan when OSV returns the same answer, using a new nullable fingerprint column. No concrete merge-blocking risk was identified, and manual, push and production scans are unchanged.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the change's main purpose: skipping a scheduled dependency rescan when OSV returns the same advisory answer. The raw summary confirms this: a fingerprint of the OSV answer is compared,…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the change's main purpose: skipping a scheduled dependency rescan when OSV returns the same advisory answer. The raw summary confirms this: a fingerprint of the OSV answer is compared, and a match returns Unchanged. The perf() prefix fits the performance intent. The title is specific and does not overstate the change. It is also in scope for the 'perf' scope term, which is accurate for the stated goal of reducing repeated work.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 531e2f5f-d5c5-4dfa-83ea-8ec181c764eb
Commits

Reviewing files that changed from the base of the PR and between 6346519 and 4267722.

Files selected for processing: 3

  • packages/backend/src/services/dependency-scan/DependencyScanStore.ts
  • packages/backend/src/services/dependency-scan/DependencyScanner.test.ts
  • packages/backend/src/services/dependency-scan/DependencyScanner.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

The repository list shows the newest head scan of any status. The
precheck read only the latest completed scan, so a transient failure
after it stayed on the repository until the commit changed. The precheck
now reads the newest head scan whatever its status and skips only when
that scan completed.
@maple-review-bot

maple-review-bot Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge. Skip, fallbacks and the failed-scan regression are each pinned by call-counting tests; the residual assumption is OSV bumping modified when a record is edited.
no findings · tests covered · risk medium · 3/3 new units observable

A scheduled dependency rescan now returns Unchanged without a scan row when the commit and OSV's answer are unchanged, with the lockfile plan cached per commit and a new advisory_fingerprint column. Every skip and fallback path is pinned by new tests; safe to merge.

  • DependencyScanner.scan prechecks a scheduled head scan and skips when the OSV fingerprint matches
  • PackageAdvisoryCatalog splits into query (querybatch) and resolve (record reads), with fingerprintOf
  • dependency_scans.advisory_fingerprint stores the OSV answer only on scans that reconciled
  • Commit lockfile selection is cached as a plan keyed by provider, repo and commit sha

Before merge

4267722 · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo merged commit 430a0ce into main Nov 16, 2026
37 checks passed
@Makisuo
Makisuo deleted the perf/dep-scan-skip-unchanged branch November 16, 2026 09:56
@maple-review-bot

Copy link
Copy Markdown

Merged with 2 steps from "Before merge" still open. If they are done, tick them on the review comment; if not, now is the time.

  • Manual · Deploy applies 20261116082407_dependency_scan_advisory_fingerprint; existing rows have a null fingerprint, so every repository full-scans once after rollout
  • Migration · Check it is safe on production data and with the code still deployed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant