Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
d22df3a
fix(smoke): derive asset probe path from output tree, not extension (…
jung-thomas Nov 2, 2026
820eed3
Merge pull request #2981 from sap-tutorials/worktree-fix-smoke-assets…
jung-thomas Nov 2, 2026
d4663b5
docs(ops): document #2975 idle-runner cds.flush() recovery in schedul…
jung-thomas Nov 2, 2026
833f48e
fix(fetch): fail loud on degraded QA discovery instead of shipping st…
jung-thomas Nov 2, 2026
3afdd1e
Merge pull request #2984 from sap-tutorials/worktree-docs-scheduler-f…
jung-thomas Nov 2, 2026
e061bcd
Merge pull request #2985 from sap-tutorials/fix-qa-stale-fetch-2974
jung-thomas Nov 2, 2026
d49150e
docs(asset-retention): correct stale comment — pipeline DOES emit ass…
jung-thomas Nov 2, 2026
f64db5f
docs(scheduler): clarify self-heal liveness + add orphan JobLastRun c…
jung-thomas Nov 2, 2026
12643cb
Merge pull request #2986 from sap-tutorials/worktree-fix-stale-asset-…
jung-thomas Nov 2, 2026
19c63da
Merge pull request #2987 from sap-tutorials/worktree-fix-selfheal-orp…
jung-thomas Nov 2, 2026
cd9013a
docs(plan): scheduler readiness gate design + implementation plan (#2…
jung-thomas Nov 2, 2026
ee7010a
feat(scheduler): readiness latch for blue-green health gate (#2975)
jung-thomas Nov 2, 2026
d8a7390
feat(scheduler): computeReadiness helper + flip latch at end of CronS…
jung-thomas Nov 2, 2026
f5e8add
feat(scheduler): anonymous /health/ready readiness probe (#2975)
jung-thomas Nov 2, 2026
d21c618
fix(scheduler): guard absent cds.db; assert full response shape (#2975)
jung-thomas Nov 2, 2026
86e732c
feat(deploy): gate blue-green on /health/ready for tutorials-srv + sr…
jung-thomas Nov 2, 2026
9566e18
docs(ops): document shipped /health/ready readiness gate (#2975)
jung-thomas Nov 2, 2026
c48837f
docs(ops): correct fastest-canary cadence in readiness-gate notes (#2…
jung-thomas Nov 2, 2026
eaab06e
Merge pull request #2988 from sap-tutorials/worktree-docs-scheduler-f…
jung-thomas Nov 2, 2026
2705c3a
fix(scripts): load+link CDS model in cleanup-selfheal-orphan before c…
jung-thomas Nov 2, 2026
81c5a28
Merge pull request #2989 from sap-tutorials/worktree-fix-selfheal-orp…
jung-thomas Nov 2, 2026
b7f7087
fix(app-space): key track identity on CompletionPaths UUID, not legac…
jung-thomas Nov 2, 2026
c834018
Merge pull request #2991 from sap-tutorials/worktree-fix-appspace-tra…
jung-thomas Nov 2, 2026
48164bf
fix(cap): disable draft_admin_integrity for production — missing HANA…
jung-thomas Nov 2, 2026
d997777
Merge pull request #2992 from sap-tutorials/worktree-fix-draft-integr…
jung-thomas Nov 2, 2026
71207c2
feat(ops): cleanup-draft-orphans.cjs — sweep orphaned draft rows from…
jung-thomas Nov 2, 2026
2d4f419
Merge pull request #2994 from sap-tutorials/worktree-draft-orphan-cle…
jung-thomas Nov 2, 2026
dbab4e7
fix(help-docs): repair stored Architecture Center URLs (#2948 follow-up)
jung-thomas Nov 2, 2026
058a536
fix(health): /health/ready dbOk always false on HANA — crash-looped e…
jung-thomas Nov 2, 2026
4cbef20
docs(#2990): mission path ordering design + plan
jung-thomas Nov 2, 2026
cab7e58
docs(2993): design + implementation plan for safe draft_admin_integri…
jung-thomas Nov 2, 2026
8fea106
feat(#2990): add pathOrder to CompletionPaths schema
jung-thomas Nov 2, 2026
4b1f37e
fix(help-docs): route dated news posts + nudge concept re-render
jung-thomas Nov 2, 2026
7bd3bd3
Merge pull request #2995 from sap-tutorials/worktree-fix-health-ready…
jung-thomas Nov 2, 2026
be38da7
feat(db): build-time ON DELETE CASCADE patch for draft tables, not ye…
jung-thomas Nov 2, 2026
2bcf77f
feat(#2990): editable Order column + SortOrder on CompletionPaths adm…
jung-thomas Nov 2, 2026
bc78b3b
fix(script): nudge ALL target links, not just changed rows
jung-thomas Nov 2, 2026
e02afbe
fix(db): trailing-comma DDL form + tighter idempotency guard in draft…
jung-thomas Nov 2, 2026
32e74ec
fix(test): key App-Space progress tests on CompletionPaths UUID, not …
jung-thomas Nov 2, 2026
14f372c
Merge pull request #2997 from sap-tutorials/worktree-fix-appspace-id-…
jung-thomas Nov 2, 2026
0925d3c
Merge pull request #2996 from sap-tutorials/worktree-fix-archcenter-u…
jung-thomas Nov 2, 2026
4b3866f
test(drafts): flag-agnostic gate — Mission discard leaves zero child …
jung-thomas Nov 2, 2026
77280e0
feat(#2990): auto gap-number pathOrder on new CompletionPaths
jung-thomas Nov 2, 2026
3aeec23
feat(#2990): preserve pathOrder in copyMission and seed it in promote…
jung-thomas Nov 2, 2026
0aa246e
docs(ops): runbook for safely re-enabling draft_admin_integrity (#2993)
jung-thomas Nov 2, 2026
7beacdf
fix(docs): correct issue cross-references in draft-integrity-reenable…
jung-thomas Nov 2, 2026
dcf5e1b
fix(db): uppercase FK local column + enforced-FK docs correction (#2993)
jung-thomas Nov 2, 2026
d61f64b
feat(#2990): order mission completion paths by pathOrder at all rende…
jung-thomas Nov 2, 2026
1f1341f
docs(spec): remove stale (NOT VALIDATED) parenthetical from Goals sec…
jung-thomas Nov 2, 2026
f522aa9
Merge pull request #2999 from sap-tutorials/worktree-draft-integrity-…
jung-thomas Nov 2, 2026
286f8d6
fix(#2990): sort NULL pathOrder last on both HANA and SQLite (coalesc…
jung-thomas Nov 2, 2026
a18bbf2
feat(#2990): backfill CompletionPaths.pathOrder alphabetically per mi…
jung-thomas Nov 2, 2026
a0fb331
fix(health): add /health/ready to srv-qa — missing route 404-crashed …
jung-thomas Nov 2, 2026
aea5a81
Merge pull request #3000 from sap-tutorials/worktree-fix-qa-health-re…
jung-thomas Nov 2, 2026
c575f3d
build(#2990): stage cds build artifacts for pathOrder (csn + hdbmigra…
jung-thomas Nov 2, 2026
1f07641
Merge pull request #3001 from sap-tutorials/worktree-mission-path-ord…
jung-thomas Nov 2, 2026
cd6ee06
fix(db): emit draft cascade FK as .hdbconstraint, not inline in .hdbt…
jung-thomas Nov 3, 2026
f386769
Merge pull request #3002 from sap-tutorials/worktree-unblock-disable-…
jung-thomas Nov 3, 2026
6f117e6
revert(db): remove draft cascade-constraint patch — unneeded while da…
jung-thomas Nov 3, 2026
e0b502a
Merge pull request #3003 from sap-tutorials/worktree-remove-cascade-p…
jung-thomas Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .deploy/mta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,10 @@ modules:
# srv/developer-service.js, srv/lib/ip-rate-limit.js, srv/lib/chat-rate-limit.js.
# See docs/developers/operations/mta-deployment.md "Scaling Constraints" and TODO §22.
instances: 1
health-check-type: http
health-check-http-endpoint: /health/ready
health-check-invocation-timeout: 10
timeout: 180
properties:
NODE_ENV: production
EXPOSE_CAP_UI: false
Expand Down Expand Up @@ -187,6 +191,10 @@ modules:
disk-quota: 1024M
buildpack: nodejs_buildpack
instances: 1
health-check-type: http
health-check-http-endpoint: /health/ready
health-check-invocation-timeout: 10
timeout: 180
build-parameters:
builder: custom
commands:
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,4 @@ The load-bearing few. **Full detail for every relocated item → [tutorials-ims-
- **External channels subsystem** — `Channels` entity (`db/channels.cds`) is the source of truth; re-ingest via `npm run seed-channels`; directory at `/channels` (baked by `fetch-channels` in `build:all`); verb-lane fill via `npm run promote-channels`; community items never land in `START_HERE`. → [channels.md](docs/developers/reference/channels.md).
- **TechEd sessions subsystem (#2312)** — new `TechEd*` entities (`db/external/teched.cds`, NOT reused `CommunityEvents`), loaded via a **`using` side-effect** in `db/external-content.cds` (don't delete the import); RainFocus ingest (`POST https://events.rainfocus.com/api/sessions`, headers `rfapiprofileid`/`rfwidgetid` per venue from `RAINFOCUS_{TE26,TEV26}_PROFILE`/`_WIDGET` env — live-confirmed defaults baked in, IDs may rotate; speakers embedded in `participants[]`, tracks in `attributevalues[attribute=="Track"]`, offset-paginated); idempotent seed via `scripts/seed-teched.cjs`; public `/build/teched` feed; corrected fetcher + delta cron jobs shipped in #2332; `/teched/` page + island, KG projection, embeddings, and Devtoberfest cross-linking are later units. → [teched.md](docs/developers/reference/teched.md).
- **Signed provenance envelope (#2245)** — `PROVENANCE_ENVELOPE_ENABLED` (DB config, default OFF, DEV-first); two anonymous Express endpoints (`/content/tutorials/:slug/provenance` and `/.well-known/tutorial-provenance/jwks.json`); signing key = `PROVENANCE_SIGNING_KEY` credstore secret (Ed25519 PKCS8 PEM); rotate via `/admin-ui/#secrets`. Fail-open. → gotchas.md "Signed provenance envelope".
- **Optimized draft deletion `draft_admin_integrity` ON (#2909, CAP 10.2)** — `cds.fiori.draft_admin_integrity: true` in package.json; draft discard cascade-deletes draft `CompletionPathItems`/`GroupPathItems` at the DB level, so rollup reconcile now fires via `after('DISCARD','Missions.drafts'/'Groups.drafts')` (the `.drafts` target is required), NOT the per-row child `after('DELETE')` hooks (those stay for direct programmatic deletes). Deleting an active entity right after `draftActivate` can 403 ("a draft exists") until the residual draft is discarded. Constraint migration deploys to HANA (`.hdbconstraint`, `NOT VALIDATED`) BEFORE the flag serves. → gotchas.md "Optimized draft deletion".
- **Optimized draft deletion `draft_admin_integrity` ON (#2909, CAP 10.2)** — `cds.fiori.draft_admin_integrity: true` in package.json; draft discard cascade-deletes draft `CompletionPathItems`/`GroupPathItems` at the DB level, so rollup reconcile now fires via `after('DISCARD','Missions.drafts'/'Groups.drafts')` (the `.drafts` target is required), NOT the per-row child `after('DELETE')` hooks (those stay for direct programmatic deletes). Deleting an active entity right after `draftActivate` can 403 ("a draft exists") until the residual draft is discarded. Constraint migration deploys to HANA (`.hdbconstraint`, `NOT VALIDATED`) BEFORE the flag serves. → gotchas.md "Optimized draft deletion". Re-enable runbook: docs/developers/operations/draft-integrity-reenable.md.
3 changes: 3 additions & 0 deletions app/admin-annotations.cds
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,7 @@ annotate AdminService.CompletionPaths with {
name @Common.Label: 'Title';
description @Common.Label: 'Description' @UI.MultiLineText;
slug @Common.Label: 'Slug';
pathOrder @Common.Label: 'Order';
};

annotate AdminService.CompletionPaths with @UI: {
Expand All @@ -233,7 +234,9 @@ annotate AdminService.CompletionPaths with @UI: {
{ Value: description }
]
},
PresentationVariant: { SortOrder: [ { Property: pathOrder } ] },
LineItem: [
{ Value: pathOrder, Label: 'Order' },
{ Value: name },
{ Value: slug }
]
Expand Down
6 changes: 5 additions & 1 deletion db/last-dev/csn.json
Original file line number Diff line number Diff line change
Expand Up @@ -1298,6 +1298,10 @@
"type": "cds.String",
"length": 255,
"@cds.persistence.name": "SLUG"
},
"pathOrder": {
"type": "cds.Integer",
"@cds.persistence.name": "PATHORDER"
}
},
"$tableConstraints": {
Expand Down Expand Up @@ -4522,7 +4526,7 @@
"flavor": "inferred",
"sap.changelog.enhanced": true,
"minified": true,
"build": "CDS Build v10.1.0"
"build": "CDS Build v10.2.2"
},
"$version": "2.0"
}
1 change: 1 addition & 0 deletions db/schema.cds
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,7 @@ entity CompletionPaths : cuid, LegacyKeyed {
name : String(255);
description : String(1000);
slug : String(255);
pathOrder : Integer;
items : Composition of many CompletionPathItems on items.path = $self;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
== version=3
== version=4
COLUMN TABLE com_sap_developers_ims_CompletionPaths (
ID NVARCHAR(36) NOT NULL,
legacyId INTEGER,
mission_ID NVARCHAR(36),
name NVARCHAR(255),
description NVARCHAR(1000),
slug NVARCHAR(255),
pathOrder INTEGER,
PRIMARY KEY(ID)
)

== migration=4
-- generated by cds-compiler version 7.0.1
ALTER TABLE com_sap_developers_ims_CompletionPaths ADD (pathOrder INTEGER);

== migration=3
-- no-op: description column already added in migration=2

Expand Down
230 changes: 230 additions & 0 deletions docs/developers/operations/draft-integrity-reenable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
# Safely Re-enabling `draft_admin_integrity` on Production

**Status:** DEFERRED — This runbook documents the SAFE re-enable procedure for `cds.fiori.draft_admin_integrity`. The flag is currently **disabled** (`false` in `package.json`). Re-enable ONLY after all green checks — another outage is unacceptable. This runbook exists so it CAN be done safely when the decision is made; it is NOT an instruction to do it now.

**Context:**
- Issue #2909: introduced the flag (CAP 10.2) + missing cascade constraints → PROD outage.
- Issue #2968: added the preflight check `srv/lib/draft-integrity-preflight.js`.
- Issue #2992: disabled the flag for all profiles and fixed the preflight (it had queried non-existent `com_sap_developers_ims_*_drafts` names instead of the real `ADMINSERVICE_*_DRAFTS` discovered from `SYS.TABLE_COLUMNS`).
- Issue #2994: added the orphan cleanup sweep `scripts/cleanup-draft-orphans.cjs`.
- Issue #2993 (this work): added the build-time FK injection `scripts/patch-draft-cascade-constraints.cjs`, the gate test `test/admin-drafts-cascade-gate.test.js`, and this runbook.

**Key facts:**
- The inline FK (`FOREIGN KEY ... ON DELETE CASCADE` inside the `COLUMN TABLE (...)` body) is created **ENFORCED** by HANA — `NOT VALIDATED` is an `ALTER TABLE` affordance only and cannot be specified inline. This means orphaned draft rows **must be swept BEFORE the constraint-bearing HDbtable is deployed**, or HANA will reject the deploy.
- `CDS_*` env-var overrides do NOT reach `cds.fiori.draft_admin_integrity` on deployed builds — only the baked `package.json` value counts.
- Preflight is HANA-only and fail-open; SQLite unit tests do not require cascade FKs.

---

## Prerequisites

- [ ] Git branch is up-to-date with `origin/main` (or `origin/DEV` for DEV environments).
- [ ] You have `cf login` credentials for the target environment (DEV first, then PROD).
- [ ] Node.js 22+ and `npm` are installed locally.
- [ ] `@sap/cds-dk` is globally available (`cds --version`).

---

## Step 1: Sweep Orphaned Drafts (Dry-Run First)

The inline FK is ENFORCED — HANA will reject deploying the constraint if orphaned draft rows already exist in the schema. Sweep orphans **before** deploying the constraint-bearing build.

**Locally, with credentials bound to the target environment:**

1. Set up the service binding (one-time):
```bash
npm run bind:setup # if this is your first hybrid run
```

2. Run a dry-run to inspect any orphans:
```bash
npx cds bind --exec -- node scripts/cleanup-draft-orphans.cjs --dry-run
```

3. **Expected output:** Zero orphans found:
```
Scanning for orphaned drafts...
Found 0 orphaned draft rows. (All clean.)
```

4. **If orphans are found**, inspect them (the script prints details) and decide whether to proceed. If you expect orphans from the #2909 outage:
- Commit/note the findings.
- Run WITHOUT `--dry-run` to delete:
```bash
npx cds bind --exec -- node scripts/cleanup-draft-orphans.cjs --commit
```
- Verify the count matches the dry-run scan.
- Re-run dry-run to confirm zero before proceeding.

---

## Step 2: Deploy with Cascade Constraints (Flag Still `false`)

The Task 1 patch `scripts/patch-draft-cascade-constraints.cjs` already injects `ON DELETE CASCADE` foreign keys into the 46 draft tables in the generated `gen/db/src/gen/AdminService.*_drafts.hdbtable` files.

**On the target environment (DEV or PROD), after Step 1 confirms zero orphans:**

1. Ensure the MTA includes the latest `cds build --production` output with the patched constraints.
2. Deploy the MTA as usual (e.g., `npm run deploy -- --env dev` or via CI/CD):
```bash
npm run build:all
cd .deploy && mbt build && cf deploy mta_archives/*.mtar -e ../deploy/dev.mtaext -f
```
3. The FK is enforced — a clean schema (zero orphans from Step 1) is the precondition for the deploy to succeed.
4. Verify deployment success and CAP service startup.

**Note:** The constraints land on every build already (while `draft_admin_integrity` is `false`). This step ensures they have deployed to the target environment HANA schema with a clean baseline.

---

## Step 3: Verify Preflight Check

After deployment, the CAP service logs startup messages. Look for the preflight result from `srv/lib/draft-integrity-preflight.js`.

**Expected log:**
```
[draft-integrity-preflight] verified (N/N)
```
where N is the count of verified draft tables (typically 46 for Admin/Mission/Group/Author services + compositions).

**If you see a WARN instead:**
```
[draft-integrity-preflight] ⚠ missing: <table-names>
```
**STOP.** The constraints are incomplete or not deployed. Do not proceed to Step 4. Investigate:
- Is the MTA's `gen/db` module serving the patched HDbtable files?
- Did the HANA HDI deployment succeed (check `cf logs <app> --recent`)?
- Run `cds build --production` locally and verify `gen/db/src/gen/AdminService.*_drafts.hdbtable` files contain the `ON DELETE CASCADE` clause.

---

## Step 4: Run the Gate Test (Mission Draft Lifecycle)

The gate test `test/admin-drafts-cascade-gate.test.js` verifies that Mission draftEdit + discard leaves zero orphaned child drafts.

**Locally:**
```bash
npx vitest run test/admin-drafts-cascade-gate.test.js
```

**Expected:** All tests pass (green), including:
- Draft creation, edit, and discard lifecycle.
- Zero orphaned `Missions.drafts` rows after discard.
- Zero orphaned `CompletionPathItems.drafts` rows after parent discard.

**If the test fails,** re-examine Step 1–3. Do NOT proceed to Step 5 until this test is green.

---

## Step 5: Flip the Flag in `package.json`

**ONLY after Steps 1–4 are green on the target environment**, update the flag:

1. Edit `package.json`:
- Line ~461: find the `"cds": { "fiori": { ... } }` block and set `"draft_admin_integrity": true`.
- Line ~486: find the `"[development]": { "cds": { "fiori": { ... } } }` block and set `"draft_admin_integrity": true`.

2. Example (before):
```json
"cds": {
"fiori": {
"draft_admin_integrity": false
}
}
```
Example (after):
```json
"cds": {
"fiori": {
"draft_admin_integrity": true
}
}
```

3. Commit and push the change:
```bash
git add package.json
git commit -m "feat(config): enable draft_admin_integrity globally (#2993)"
git push origin <branch>
```

4. Run a fresh build with the flag ON:
```bash
npm run build:all
```

5. Deploy to the target environment:
```bash
cd .deploy && mbt build && cf deploy mta_archives/*.mtar -e ../deploy/dev.mtaext -f
```

---

## Step 6: Smoke Test – Mission Discard on Deployed Environment

After the re-deployed service is running with the flag `true`, verify the behavior end-to-end:

1. **Via Admin UI** (`/admin-ui/#/admin/missions`):
- Create or open an existing Mission draft.
- Edit it (add a step, change the title, etc.).
- Click **Discard** (or equivalent button).
- Expected: Discard succeeds; no orphaned rows left in `Missions.drafts` or child composition tables.

2. **Verify no errors in the server logs:**
```bash
cf logs <app> --recent | grep -i error
```

3. **If discard fails or errors appear,** rollback the flag to `false`, redeploy, and investigate the error. Escalate to the team.

---

## Rollback Plan (Emergency)

If any step fails or the deployed service encounters errors:

1. Revert the flag to `false` in `package.json` (Steps 1–2):
```json
"draft_admin_integrity": false
```

2. Commit, push, and redeploy:
```bash
git add package.json
git commit -m "revert(config): disable draft_admin_integrity (#2993)"
git push origin <branch>
npm run build:all
cd .deploy && mbt build && cf deploy mta_archives/*.mtar -e ../deploy/<env>.mtaext -f
```

3. The enforced constraints remain in the schema but the flag controls whether the runtime relies on them. With the flag `false` the app-level reconcile path runs instead, so the deployed constraints are dormant but harmless.

4. Investigate the root cause before attempting re-enable again. Involve the team.

---

## Environment Order

**Always test DEV first, then PROD.**

- **DEV:** Lower risk; catch any logic gaps before production.
- **PROD:** Only after DEV smoke test passes for at least 2–4 hours without incident.

---

## Related Issues

- **#2909:** Introduced the flag (CAP 10.2) + missing cascade constraints → PROD outage.
- **#2968:** Added the preflight check `srv/lib/draft-integrity-preflight.js`.
- **#2992:** Disabled the flag for all profiles and fixed the preflight (queried non-existent `com_sap_developers_ims_*_drafts` instead of real `ADMINSERVICE_*_DRAFTS` from `SYS.TABLE_COLUMNS`).
- **#2994:** Added the orphan cleanup sweep `scripts/cleanup-draft-orphans.cjs`.
- **#2993:** Added the build-time FK injection `scripts/patch-draft-cascade-constraints.cjs`, the gate test `test/admin-drafts-cascade-gate.test.js`, and this runbook.

---

## Contacts & Escalation

For questions or issues:
- Check [docs/developers/reference/tutorials-ims-gotchas.md](../reference/tutorials-ims-gotchas.md) for known draft-related gotchas.
- Review the related issues linked above for context.
- Escalate failures to the team via Slack or the issue tracker.
Loading
Loading