Skip to content

release: draft-edit PROD fix (#2909) + health-gate fixes (#2975) + mission path ordering (#2990) - #3004

Merged
jung-thomas merged 60 commits into
mainfrom
DEV
Nov 3, 2026
Merged

jung-thomas merged 60 commits into
mainfrom
DEV

Conversation

@jung-thomas

Copy link
Copy Markdown
Contributor

DEV→main promotion. 60 commits. Primary driver: restore mission editing in PROD (the 2026-11-02 outage).

Critical PROD fix

  • Use new Draft Delete - cds.fiori.draft_admin_integrity: true in CAP #2909 fix(cap): disable draft_admin_integrity for production — the CAP 10.2 flag was on in PROD with no HANA cascade constraints, orphaning draft rows and making missions un-editable (DRAFT_ALREADY_EXISTS / Not Found). Flag now off for all profiles; preflight rewritten to discover real ADMINSERVICE_*_DRAFTS tables. Already mitigated live in PROD (orphans cleared, env override); this makes the fix permanent in the build.

Deploy-enabling fixes (#2975 readiness gate, uncovered deploying the above)

Also included

Verification (DEV, this release's exact build)

  • Both apps 1/1; /health/ready 200 on tutorials-srv (db:connected) and tutorials-srv-qa.
  • Approuter: homepage 200, /tutorials/ 200, /concepts/ 200.
  • Smoke gate: 7/413 failed — all triaged non-regressions (concepts p50 perf-threshold; concepts-cap + explore-graph checks are content/seed-data-gated, not code). No failure relates to the changes in this release.

Post-merge PROD steps (I will run after approval + PROD deploy)

  1. Deploy PROD from fresh origin/main (blue-green).
  2. Run cleanup-draft-orphans.cjs --commit on PROD to clear any orphans created in the gap while the old build still had the flag on.
  3. Unset any residual PROD env overrides (already done: CDS_CONFIG removed).

🤖 Generated with Claude Code

jung-thomas and others added 30 commits November 2, 2026 09:58
…2533)

Vite emits each island's CSS sidecar into the JS output tree under
assets/ (hugo/static/js/assets/<name>-<hash>.css). Those bundles ship
at /js/assets/... , never /css/... . The asset-retention smoke gate
derived the probe kind from the file extension (.css -> /css/), so every
assets/-prefixed .css entry 404'd on a perfectly healthy deploy -- a
false positive that flapped the post-deploy gate.

Fix: assets/-prefixed bundles are always probed under /js/ regardless
of extension; only top-level Hugo-fingerprinted .css lives under /css/.
Verified against live PROD: all 627 manifest bundles now serve 200.
…-kind-2533

fix(smoke): probe assets/ island CSS under /js/ not /css/ (#2533)
…er runbook

The scheduler-troubleshooting runbook still described the ~10-min self-heal
tick as 'alert-only' and 'the reaper is the recovery, the alert is just the
signal'. Since #2975 that tick also calls the documented cds.flush() on every
pass to drain the 'processor idle, queue healthy' stall mode (CAP 10.2's
in-memory runner does one boot flush then only re-arms off an event-driven
chain; when that chain doesn't carry forward after a deploy, ready status=NULL
rows sit unconsumed until the ~1h fallback). Fold that signature and its active
recovery into the runbook, per the #2975 acceptance criteria:

- Periodic self-heal bullet now lists all four passes (reap / re-seed / flush /
  canary) and reframes passes 1-3 as the active recovery, canary as last-resort.
- New 'idle-runner drain via cds.flush()' subsection: the signature, the 1h
  fallback / event-chain mechanism, PROD 2026-11-02 + 2026-11-01 proof, the
  fail-open flush, and the open CF readiness-gate (health-check-http) follow-up.
- Manual-restart fallback note now reflects reap+re-seed+flush all ran.
…ale (#2974)

QA channel fetches private *-Contribution repos over github.com only. A
wrong-host token (github.tools.sap bridged by the deploy-watcher WSL wrapper)
or an expired one makes discovery 401 and fail SOFT — falling back to the
on-disk/HANA cache and collecting per-slug 404s into an error list that never
throws. The step exits 0 and the deploy ships a QA channel built from stale
cache minus the private slugs (the 1.42.0 symptom: 31 failed QA fetches,
degraded /tutorials-qa/).

Add qaDiscoveryIsDegraded() and gate it in main(): when channel=qa and
discovery degraded to 'disk'/'hana', exit non-zero so deploy-mta.cjs Step 2.7
aborts. Opt out with ALLOW_STALE_QA=true for a genuine github.com outage.
Prod is unaffected (cache fallback tolerated by design).

The companion deploy-watcher SKILL.md token-bridge fix (bridge a github.com
token, not github.tools.sap) lives in ~/.claude/skills/ (not this repo).
…lush-2975

docs(ops): document #2975 idle-runner cds.flush() recovery in scheduler runbook
fix(fetch): fail loud on degraded QA discovery (#2974)
…leanup (#2983)

The __scheduler_selfheal__ JobLastRun row is a dead orphan from a previous
self-heal implementation. The current armSchedulerSelfHeal (scheduler-reaper.js)
is a setInterval timer, not a runWithLock job, so it never writes a JobLastRun
row and the orphan sits permanently stale — an operator footgun that read as a
false 'scheduler is dead' signal after the #2975 PROD deploy.

- Runbook: document that self-heal liveness is verified via scheduled jobs
  advancing since boot, NOT a self-heal JobLastRun row; exclude the orphan from
  any JobLastRun-sorted overdue view.
- scripts/cleanup-selfheal-orphan.cjs: guarded, idempotent one-time cleanup
  (dry-run default, --delete to remove; no-op if absent). Uses cds.ql, no raw SQL.

Verified: cleanup CQL tested on in-memory SQLite against the real JobLastRun
schema — dry-run preserves rows, --delete removes only the orphan, re-run no-ops.
…retention-comment-2982

docs(asset-retention): correct stale comment about assets/ island CSS (#2982)
…han-2983

docs(scheduler): clarify self-heal liveness + orphan JobLastRun cleanup (#2983)
- CRITICAL: replace `cds.db?.run()` optional-chain with an explicit
  `if (cds.db)` guard so an absent DB leaves dbOk=false instead of
  silently resolving to true (await undefined never throws)
- IMPORTANT: add db:'connected' + timestamp assertions to integration
  test so a regression dropping either field is caught end-to-end
)

Final-review Minor: both the runbook and the design doc misstated the
fastest canary. Verified against srv/jobs/scheduler.js: kg-ondemand-drain
is 1-59/2 (~every 2 min), metrics-rollup is */5. The state-3 'unreachable
synchronously' rationale holds either way; corrected the figure.
…lush-2975

feat(scheduler): /health/ready readiness gate for blue-green deploys (#2975 Fault A)
…ds.entities() (#757/#911)

cleanup-selfheal-orphan.cjs called cds.entities('com.sap.developers.ims')
after a bare cds.connect.to('db') under 'cds bind --exec', with no
cds.load(). The regression-#757 guard (cds-entities-requires-load.test.ts)
failed the DEV unit-tests job, and at runtime the script would throw
'cds.entities is not a function'.

Apply the canonical #911-safe pattern used by the other operator scripts:
  cds.model = cds.linked(await cds.load('*'));
  void cds.model.entities;   // prime getter before connect strips the iterator
  await cds.connect.to('db');

Scoped guard test: 293/293 pass (was 1 failed).
…han-cds-load-2983

fix(scripts): load CDS model in cleanup-selfheal-orphan before cds.entities() (#757/#911)
…yId (#2952)

The App Space track UI used legacyId as the track identifier (id: p.legacyId
in getAppSpaceProgress), but legacyId is only populated for IMS-migrated rows
and is null for AI-generated / event tracks (TechEd Texas). A null id broke
three things at once, all of which looked like a deploy regression but were
data-shape dependent:

  - track -> URL sync: trackUrl() took the delete-param branch for a null id,
    so selecting a track never wrote ?track=<id>; a browser refresh dropped
    back to the track list.
  - Refresh button: the re-open-after-refresh match keyed on the null id, so
    the open track (and its unlocked barcode) never re-rendered.
  - checkpoint barcode gate: keyed on track/item identity.

Fix: emit the always-present CompletionPaths UUID (p.ID) as the track id and
treat it as an opaque string end-to-end in AppSpace.vue (no Number() coercion
in syncTrackFromUrl). legacyId stays reserved for its migration purpose.

Also harden the scanner expired-session redirect: the approuter answers an
expired XSUAA session with 200 + an HTML login document (fetch follows the
302), not a bare 401, so the old status===401 check never fired. Detect the
redirected/HTML-login response and reload so the approuter re-issues auth.

Tests: track-url regression fixture switched to UUID string ids (reproduces
the null/non-numeric id PROD shape).
…ck-id-2952

fix(app-space): key track identity on CompletionPaths UUID, not legacyId (#2952)
… cascade constraints corrupted drafts (#2909)

The CAP 10.2 cds.fiori.draft_admin_integrity flag was enabled for production
(#2909) but the required ON DELETE CASCADE .hdbconstraint files were never
generated/deployed. With the flag ON and no cascade, every mission draft
activate/discard left ORPHANED draft rows (draft row survives, its
DraftAdministrativeData parent is gone). FE then 404s on draftEdit
(DRAFT_ALREADY_EXISTS + 'Not Found'), and the discard affordance never renders
— making missions permanently un-editable in PROD.

Changes:
- package.json: flip cds.fiori.draft_admin_integrity to false (prod + all
  profiles) — reverts to the proven pre-10.2 draft-deletion behavior. The
  runtime CDS_* env override does NOT work on the deployed build (verified),
  so this must be a code change.
- draft-integrity-preflight.js: the DRAFT_TABLES list used non-existent
  com_sap_developers_ims_*_drafts names. The real tables are
  ADMINSERVICE_*_DRAFTS, so the preflight reported 26/26 'missing' and
  validated NOTHING. Rewrote checkDraftCascadeConstraints to DISCOVER the real
  draft tables from SYS.TABLE_COLUMNS at runtime so the check can never drift.
- Updated unit tests for the two-query discovery contract (14 pass).

PROD orphan cleanup (57 rows: 1 MISSIONS_DRAFTS root + 32 COMPLETIONPATHITEMS +
11 COMPLETIONPATHS + 7 MISSIONTAGS + 6 MISSIONCATEGORIES) was done directly in
HANA; live drafts (11 admin rows) preserved.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…ity-2909

fix(cap): disable draft_admin_integrity — missing HANA cascade corrupted mission drafts (#2909)
…2909 (#2993)

Repeatable HANA cleanup for draft rows orphaned by draft_admin_integrity
running without cascade constraints. Discovers real ADMINSERVICE_*_DRAFTS
tables from SYS.TABLE_COLUMNS, deletes rows whose DraftAdministrativeData
parent is gone, preserves live drafts, verifies clean after commit.
--dry-run (default) / --commit. HANA-only. Idempotent.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…anup-script-2993

feat(ops): cleanup-draft-orphans.cjs — sweep orphaned draft rows (#2909/#2993)
PR #2948 corrected URL generation but left already-stored HelpDocs rows on
the old path-derived URLs that 404. This targeted script rewrites the url
column for the 6 architecture-sap-com docs behind the broken links on
/concepts/agent-gateway, keyed by (source, sourceId), to their verified
Docusaurus slug routes (all 200). Dry-run default; idempotent.
…very deploy (#2975)

The #2975 readiness gate probes the DB with a bare `SELECT 1`. HANA rejects
that ("sql syntax error") — it requires `SELECT 1 FROM DUMMY`. So dbOk was
ALWAYS false on HANA, /health/ready stayed 503, and CF's new startup health
gate (also #2975) killed tutorials-srv at 60s on every deploy → crash loop.

The integration test runs on in-memory SQLite (where bare `SELECT 1` works),
so it passed green while the real HANA path was broken — a classic
test-environment gap.

Fix: add dialect-safe `dbPingSql(kind)` to scheduler-ready.js
(`SELECT 1 FROM DUMMY` on hana, `SELECT 1` elsewhere) and use it in both
/health/ready and /health/db (the latter had the mirror bug: unconditional
`FROM DUMMY`, broken on SQLite). Unit tests cover both dialects.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
jung-thomas and others added 29 commits November 2, 2026 14:55
- buildDocUrl: date-expand Docusaurus blog paths (news/YYYY-MM-DD-rest →
  /news/YYYY/MM/DD/rest); the naive path form 404s. Fixes the 7th broken
  archcenter link on /concepts/agent-gateway that #2948 missed.
- 2 fetcher tests: dated-news routing + non-dated fallthrough.
- fix-archcenter-urls.cjs: add the news row; after the url repair, touch the
  linked HelpDocConceptLinks rows (value-neutral) so their managed modifiedAt
  bumps the concept-input-digest — otherwise the concept re-render phase-skips
  and the corrected href never re-bakes into the served BLOB.
…-hana-2975

fix(health): /health/ready 503 on HANA crash-loops every deploy (#2975)
…t enabled (#2993)

Adds scripts/patch-draft-cascade-constraints.cjs which enumerates all
*_drafts.hdbtable files emitted by `npx cds build --production` and
injects a FOREIGN KEY ... ON DELETE CASCADE clause for the
DraftAdministrativeData_DraftUUID column so HANA can enforce cascade-
delete of orphaned draft rows.

Wired into the MTA before-all builder (prod gen tree only; QA tree
emits zero draft tables). The draft_admin_integrity flag remains false
in package.json — this task only makes the constraints deployable.

Verified: patched 46 / skipped 0 against a real production build.
The digest nudge previously keyed off `plan` (rows whose url changed this run)
and was gated behind an early `planned===0` return. On an idempotent re-run
where urls are already correct but the served concept BLOB is still stale, that
skipped the nudge entirely — so the re-bake never triggered. Nudge now targets
every PRESENT target HelpDoc's links (presentIds) and runs on any --commit.
… cascade patch (#2993)

Two review findings:
- Comma now trails PRIMARY KEY(ID) line (trailing-comma style) rather
  than appearing as a bare leading comma on the separator line. Fixed by
  trimEnd()ing the body slice so the comma attaches to the last element.
- Idempotency regex tightened from /ON DELETE CASCADE/i to a pattern
  that keys on DraftAdministrativeData_DraftUUID + ON DELETE CASCADE
  (cross-paren match via [\s\S]*?) so a different column's cascade
  clause cannot be mistaken for our injection.

New test: 'comma trails the previous line — no leading-comma style'
asserts PRIMARY KEY(ID), immediately precedes FOREIGN KEY.
5/5 tests pass; real build: patched 46 / skipped 0.
…legacyId (#2952)

PR #2991 (#2952) changed getAppSpaceProgress to emit the CompletionPaths
UUID (p.ID) as the track id instead of p.legacyId, but left two test files
looking paths up by the old numeric legacyId. The .find(p => p.id === <num>)
calls returned undefined, failing 11 tests (expected undefined to be defined)
on DEV and every branch that merged it.

Switch the lookups to the seeded UUID constants (G_PATH_GROUP/G_PATH_FLAT,
FORK_PATH_ID/LONE_PATH_ID) to match the new opaque-string id contract.

Verified: 11 previously-failing tests now pass; full unit suite green.
…tests-2952

fix(test): key App-Space progress tests on CompletionPaths UUID, not legacyId (#2952)
…rls-data

fix(help-docs): complete the Architecture Center URL repair (#2948 follow-up)
Critical fix: local column in FK clause changed from quoted mixed-case
("DraftAdministrativeData_DraftUUID") to quoted UPPERCASE
("DRAFTADMINISTRATIVEDATA_DRAFTUUID"). The column is declared bare
(unquoted) in the generated .hdbtable, so HANA folds it to uppercase;
a quoted mixed-case reference is case-sensitive and would not resolve.
Confirmed by srv/lib/draft-integrity-preflight.js:93,100 and
scripts/cleanup-draft-orphans.cjs:55 both using the uppercase name.

Idempotency regex updated to match the uppercase token.

Test updated: FK-match assertion now requires "DRAFTADMINISTRATIVEDATA_DRAFTUUID"
(exact uppercase, not /i on the column name) and adds a regression guard
that fails on the mixed-case quoted form.

Docs corrected across runbook, spec, and plan: inline FK is ENFORCED
(NOT VALIDATED is an ALTER TABLE affordance only), so orphan sweep must
come BEFORE deploying the constraint-bearing build. Runbook reordered:
Step 1 = sweep, Step 2 = deploy (with enforced-FK precondition noted).
All "NOT VALIDATED / no pre-clean needed" claims removed.

Real build: patched 46 / skipped 0; PRIMARY KEY(ID), FOREIGN KEY
("DRAFTADMINISTRATIVEDATA_DRAFTUUID") ... ON DELETE CASCADE confirmed.
5/5 tests pass.
…r sites

- packages/content/catalog-data.js: add pathOrder to columns, orderBy('pathOrder','name')
- srv/lib/build-catalog.js: orderBy('mission_ID','pathOrder','name') for cross-mission query
- srv/developer-service.js: orderBy('pathOrder','name') in getEventProgress and getAppSpaceProgress
- srv/lib/branch/mission-detail.js: append orderBy('pathOrder','name') to paths SELECT
- packages/content/catalog-mission-hierarchy.js: update JSDoc comment
- vitest.config.ts: add @tutorials/content alias to resolve worktree src in unit tests
- test/: add pathOrder-ordering assertions in catalog-data, build-catalog-mission-detail,
  and app-space-segments-resolver (NULL pathOrder sorts first on SQLite per observed behavior)
…constraints-2993

fix(db): make draft_admin_integrity safely re-enable-able, keep flag OFF (#2993)
…e sentinel)

Use tagged template literal .orderBy\`coalesce(pathOrder, 999999), name\` at all
five CompletionPaths render sites. Plain .orderBy('pathOrder','name') sorts NULLs
FIRST on SQLite but LAST on HANA — coalesce(pathOrder, 999999) forces NULL-last on
both engines. Tests updated to assert NULL-last ordering.

Fixes the { func: 'coalesce', args: [...] } object form which is not valid as a
fluent .orderBy() positional arg — the orders() function treats plain objects without
.ref as {field: sort} pairs, turning 'func' into a field reference lookup that fails.
The tagged template literal form goes through cds.parse._select and produces the
correct CQL func/args AST.
…QA, aborting MTA deploy (#2975)

The #2975 batch set srv-qa's CF startup health check to http /health/ready
(mta.yaml:199, parity with tutorials-srv) but never added that route to
srv-qa/server.js — it only has /healthz and /health/db. So CF's probe got a
404, srv-qa crash-looped (3 retries), and the ENTIRE MTA deploy aborted even
though tutorials-srv itself started fine. (Observed on the 2026-11-02 DEV
deploy.)

srv-qa has no scheduler, so its readiness is DB-only: ready ⇔ a dialect-safe
HANA ping (SELECT 1 FROM DUMMY — a bare SELECT 1 is a HANA syntax error)
succeeds. Also made /health/db dialect-safe (it hardcoded FROM DUMMY, which
breaks on local SQLite). Unit tests cover both dialects + the down path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…ady-2975

fix(health): add /health/ready to srv-qa — 404 crash-loop aborted MTA deploy (#2975)
…ering-2990

feat(#2990): mission completion-path ordering
…able (#2993)

The #2993 build patch injected FOREIGN KEY ... ON DELETE CASCADE directly into
each generated *_drafts.hdbtable. HDI rejects that (error 8250014: 'this
artifact type must not have references to other objects') — an .hdbtable cannot
carry references. Result: HDI make failed with 96 errors across 46 draft
tables, crashing tutorials-db-deployer and aborting the entire MTA deploy
(observed on the 2026-11-02 DEV deploy).

Fix: generate a standalone *.hdbconstraint artifact per draft table instead
(plugin com.sap.hana.di.constraint, already registered in the generated
.hdiconfig). Format confirmed against SAP-samples (hana-shine-xsa
PO_CONST.hdbconstraint): the bare SQL body with NO ALTER TABLE/ADD verb —

  CONSTRAINT <name>
  ON "<child>" FOREIGN KEY ("DRAFTADMINISTRATIVEDATA_DRAFTUUID")
  REFERENCES "DRAFT_DRAFTADMINISTRATIVEDATA" ("DRAFTUUID") ON DELETE CASCADE

The .hdbtable is left untouched. Verified against the deployed DEV container
that DRAFT_DRAFTADMINISTRATIVEDATA lives in the same HDI container schema with
PK DRAFTUUID, so HDI's ALTER TABLE ADD CONSTRAINT resolves the reference at
deploy time even though the table is CAP-runtime-created, not an HDI artifact.
Generator produces 46 .hdbconstraint files against a real production build; the
draft_admin_integrity flag stays OFF in package.json (this only makes the
constraints deployable). Unit tests rewritten for the new output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…cascade-patch-2993

fix(db): draft cascade FK via .hdbconstraint (was HDI-illegal inline FK) (#2993)
…d delete is off, blocking deploy (#2993)

The #2993 build patch generated ON DELETE CASCADE constraints for draft tables
to support CAP 10.2's optimized draft deletion (dad strategy /
cds.fiori.draft_admin_integrity). But:

1. The constraints are ONLY needed when draft_admin_integrity is ON. It is
   OFF (#2992) and staying off — orphan cleanup is handled by
   scripts/cleanup-draft-orphans.cjs (#2994) instead. So the constraints
   provide zero value right now.
2. Both delivery forms failed HDI: inline-in-.hdbtable (error 8250014), then
   standalone .hdbconstraint (error 8212001 — the ON <table> clause needs the
   base .hdbtable in the same make graph, which a delta deploy doesn't
   provide). Making it work is unresolved HDI-delta research.
3. It aborts tutorials-db-deployer on every deploy, blocking urgent PROD fixes
   (the #2909 draft-edit fix + #2995/#3000 health-gate fixes) from reaching DEV
   and then prod.

Remove the build-time patch entirely: the .deploy/mta.yaml builder line, the
script, and its unit test. Re-enabling the dad strategy later is tracked by
#2993 and requires solving the HDI make-graph dependency first (likely a
CAP-native constraint mechanism, not a hand-rolled patch).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…atch-unblock

revert(db): remove draft cascade-constraint patch blocking deploy (#2993)
@jung-thomas
jung-thomas merged commit 79d1010 into main Nov 3, 2026
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant