Repository navigation
feat: Step Cache API - #1641
Draft
RandomByte wants to merge 8 commits into
Draft
feat: Step Cache API#1641RandomByte wants to merge 8 commits into
RandomByte wants to merge 8 commits into
Conversation
The step-based build tasks in @ui5/project and the standalone step runner in @ui5/builder both flush their per-step writes in key order and must reject two concurrent keys that write the same path. @ui5/builder cannot import @ui5/project, so without a shared module the two runners would drift. Add internal/stepWriteBuffer.js as the single write-buffer contract: assertDistinctWrite guards against a same-path collision across keys, flushWriteBuffer writes the buffered resources in key order. Export it through the package exports map so both runners load the same code. createReader now accepts useGitignore (default false) and forwards it to the adapter, matching createAdapter. Add createMonitor(readerWriter) for the input-recording reader wrapper that MonitoredTaskUtil uses. Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
matz3
force-pushed
the
feat/step-cache-api
branch
from
October 8, 2026 12:05
e31a191 to
143f591
Compare
…caching
A custom or built-in task opts into the step-based API by exporting a static
stepBased flag and a default build(options) factory that returns an array of step
descriptors ({name, keys, each}). Task.getStepBased() reads the flag, replacing the
old getSupportsDifferentialBuildsCallback(). The built-in definitions flip their
cacheable tasks from supportsDifferentialBuilds to stepBased.
StepRunner drives each map step: it resolves the step keys, derives a per-key identity
to select the changed keys for a delta build, buffers writes, and records per-key
invocation data. TaskRunner calls the factory once at plan time to discover step names
for setTasks, then drives StepRunner through per-stage hooks. Each step maps to its own
pipeline stage, so BuildStageCache replaces the per-task BuildTaskCache and stageSignature
defines the four-component stage signature (project resources, dependency resources,
non-resource inputs, root resources).
The build cache also tracks non-resource task inputs. MonitoredTaskUtil wraps the TaskUtil
handed to a task and records getEnv, getTime, and tracked getProject reads, and routes
reader results through a monitored reader so resource reads become cache inputs. TaskInputSet
holds the tracked inputs and their signature. quantizeTime buckets a timestamp so getTime
stays stable within a granularity, while getBuildTime stays an untracked per-run passthrough.
ProjectResources routes tag operations through monitored tag collections so a step's tags are
recorded and replayed.
Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
matz3
force-pushed
the
feat/step-cache-api
branch
from
October 8, 2026 12:09
143f591 to
2592f9c
Compare
The incremental build cache keys its work per step, so a built-in task has to expose its unit of work as a step factory. Convert minify, buildThemes, replaceVersion, replaceCopyright, replaceBuildtime, enhanceManifest, and escapeNonAsciiCharacters to a default build(options) factory that returns step descriptors. generateThemeDesignerResources returns three steps, the others one map step each. replaceCopyright and replaceVersion read the copyright year and version through taskUtil.getTime and taskUtil.getProject, so a changed value invalidates only the affected units on a delta build. Add runSteps.js, a cache-free step runner for @ui5/builder standalone use (outside the @ui5/project build cache). Its BufferedWriter mirrors StepRunner through the shared @ui5/fs/internal/stepWriteBuffer contract. themeBuilderWorker imports themeBuilder.js lazily so TaskRunner can import the module for plan-time step-name discovery without evaluating the less-openui5 graph. Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
generateLibraryPreload is not a step factory, so it reads the experimental UI5_CLI_EXPERIMENTAL_BUNDLE_INFO_PRELOAD flag directly from process.env. The incremental build cache cannot see that read, so a changed flag value does not invalidate the cached bundle. Read the flag through taskUtil.getEnv when a TaskUtil is present, which records the read as a tracked build-cache input. Fall back to the direct process.env read when the task runs without a TaskUtil. Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
Update the incremental-build skill for the step-based build system. architecture.md describes the component map for StepRunner, MonitoredTaskUtil, BuildStageCache, TaskInputSet, stageSignature, and quantizeTime, the step-factory contract, the one-stage-per-step model, and the per-key identity argument. performance-investigation.md adds the step pipeline to the playbook: the running-step log marker, the per-key identity tiering that avoids content hashing each enumerated key, the cheap delta-build step fold, and the step invocation sidecar re-serialization rule. Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
Replace the supportsDifferentialBuilds and changedProjectResourcePaths documentation in CustomTasks.md with the step-based task API: a task opts in through a static stepBased flag and a default build(options) factory that returns scalar and map step descriptors, and the build cache re-runs only the steps and keys whose inputs changed on a delta build. Rewrite the renderMarkdownFiles example as a step-based task with a single map step and a lazily imported processor. Co-authored-by: Merlin Beutlberger <m.beutlberger@sap.com>
… cache
The step-based build API renamed the build-time unit from "task" to "stage"
(one stage per legacy task, one per step of a step-based task), but several
cache identifiers kept the old "task" name while holding stage data. Align
them with what the code actually stores.
- Index cache: the stage-execution-order field `tasks` -> `stages`. It holds
one entry per stage, keyed by stage id, not per task.
- Persistent storage: the `task_metadata` table -> `stage_request_metadata`
(it holds a stage's resource request graphs and non-resource inputs, not
task output), and its accessors readTaskMetadata/writeTaskMetadata ->
readStageRequestMetadata/writeStageRequestMetadata on BuildCacheStorage and
CacheManager.
- In-code identifiers: tasksWithDepRequests -> stagesWithDepRequests,
taskDependencySignatures -> stageDependencySignatures,
#anyTaskHasRootRequests -> #anyStageHasRootRequests, and cache-layer
comments/JSDoc that called a stage a "task".
Genuine task terms are kept: taskName/stepName, the task/{taskName} stage-id
prefix, setTasks, allTasksCompleted, and references to step-based and legacy
build tasks.
The table and field renames change the on-disk cache format. The incremental
build cache is unreleased and obsolete versioned cache directories are ignored
and rebuilt, so no migration or CACHE_VERSION bump is required.
Also updates the incremental-build skill architecture doc to match.
matz3
force-pushed
the
feat/step-cache-api
branch
from
October 8, 2026 13:34
2592f9c to
965a674
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: CPOUI5FOUNDATION-1363