Add a repeatable way to publish chosen self-host apps

content_docs/self-host-apps/data-workflow.md

@@ -12,7 +12,7 @@ recurring discovery
   -> provider refresh
 ```
 
-Production is canonical. Local is a disposable mirror for development and QA.
+Production is canonical. Local is a disposable mirror for development and QA. Claim, review, release, estimate and single-app upsert examples target development explicitly. For authorized production work on homelab, replace `--environment development` with `--environment production --confirm stack127-production`. These commands ignore `NODE_ENV` and inherited `DATABASE_URL` and reject `--database-url`. Discovery and scheduled refresh commands keep their existing environment behavior.
 
 ## Discovery
 
@@ -31,12 +31,14 @@ npm run self-host-apps:discover-app-catalogs -- --catalog all --limit 5000 --wri
 ```bash
 npm run self-host-apps:dedupe-candidates -- --dry-run
 npm run self-host-apps:dedupe-candidates
-npm run self-host-apps:select-candidate-batch -- --limit 100 --out <batch.json>
+npm run self-host-apps:select-candidate-batch -- --environment development --limit 100 --out <batch.json>
 npm run self-host-apps:collect-github-evidence -- --file <batch.json>
 ```
 
 Selection uses `FOR UPDATE SKIP LOCKED` and marks claimed rows `enriching`, so two workers do not take the same work. Reviewers use `references/review-format.md`. Put supporting links in `sources` and `data_source_urls`; use `technical_fact_status` for checked-but-unpublished technical facts. There is no field-source table.
 
+Before a review commits, claiming the same IDs again is safe. Release an abandoned claim with `npm run self-host-apps:release-candidate-batch -- --environment development --file <claimed.json> --to needs_review`; production requires the production environment and confirmation above. After the review commits, resume at estimates without reclaiming or reapplying the review. Estimates and repository refreshes can be re-run.
+
 The legacy `self-host-apps:reset-categories` command is retired and exits without database access. It previously replaced all category links using imported labels and keyword matches. Existing categories need source review too. Correct them per app with `apply-qa-overrides.mjs`, `review_scope: "partial"`, expected categories, and official evidence URLs. Do not infer purpose from incidental words such as image, model, or server.
 
 The review applier validates and moves data. Publication requires an official identity, a managed icon in `stack127-media`, one to three controlled categories, a verified self-hosting classification, install documentation, and at least one source URL. Open and source-available apps need a verified repository and license. Closed-source apps need an official website and publisher identity, but no repository.
@@ -59,7 +61,7 @@ Estimate RAM and storage for every published app for a typical small self-hosted
 
 ```bash
 npm run self-host-apps:propose-resource-estimates -- --limit 5000 --out <resource-estimates.json>
-npm run self-host-apps:apply-resource-estimates -- --file <reviewed-estimates.json> --apply-reviewed
+npm run self-host-apps:apply-resource-estimates -- --environment development --file <reviewed-estimates.json> --apply-reviewed
 ```
 
 An estimate record needs a range, kind, confidence, a short internal basis, and a link to the official project or publisher page. Use `review.method: "app_estimate"` for a practical estimate. Keep published requirements as source evidence for the estimate, not as a separate public RAM or storage label. Do not use a package memory cap as an app requirement.

scripts/self-host-apps/README.md

@@ -21,10 +21,10 @@ Discovery stores raw evidence. It cannot publish or assign final categories, sum
 
 ## Review movement
 
-- `select-candidate-batch.mjs` atomically claims queued and name-review rows.
+- `select-candidate-batch.mjs` atomically claims queued and name-review rows by rank, or exact IDs with `--ids` / `--ids-from <reviewed.json>`. `--match <name-or-repository>` is read-only lookup.
 - `collect-github-evidence.mjs` collects official API evidence without editing apps.
 - `apply-candidate-review.mjs` validates reviewed decisions and writes them transactionally.
-- `release-candidate-batch.mjs` releases an abandoned claim.
+- `release-candidate-batch.mjs` releases only abandoned enriching claims, through the shared environment and production confirmation gate.
 
 `apply-candidate-review.mjs` requires source links for published records. Unknown values stay `null` or `unknown`; checked-but-unpublished facts use `technical_fact_status`.
 
@@ -39,4 +39,6 @@ Discovery stores raw evidence. It cannot publish or assign final categories, sum
 
 Production is canonical. `sync-prod-to-local.mjs` rebuilds the disposable local mirror. `sync-local-to-prod.mjs` is an exceptional guarded upsert, requires `--confirm-reviewed-push`, and does not draft missing production rows unless `--draft-missing` is also explicit.
 
+For targeted additions, follow “Add one app on request” in `stack127_docs/self-host-apps-data.md`. Do not use `sync-local-to-prod` for single apps. Editorial commands (including claim release) use `--environment development` (default) or `--environment production --confirm stack127-production`, including production previews. GitLab/Codeberg and install-option refreshes accept `--slug`; scheduled refresh callers keep their existing environment behavior.
+
 See `content_docs/self-host-apps/data-workflow.md` and `scripts/self-host-apps/scripts.json`.

scripts/self-host-apps/apply-candidate-review.mjs

@@ -16,6 +16,8 @@ import { buildInstallOptions } from "./lib/install-options.mjs";
 import { installMethods, installOptionsToRow } from "./lib/streamlined-storage.mjs";
 import { isManagedMediaUrl } from "../stack127/media/storage.mjs";
 
+import { databaseOptions } from "./lib/environment.mjs";
+
 const decisions = new Set(["publish", "duplicate", "reject", "needs_review"]);
 
 function reviewRecords(input) {
@@ -100,7 +102,7 @@ async function ensureCategories(client, categories) {
   }
 }
 
-export async function upsertApp(client, record, expectedAppId) {
+export async function upsertApp(client, record, expectedAppId, dryRun = false) {
   const existing = await client.query("select id from public.self_host_apps where slug = $1", [record.slug]);
   if (existing.rows[0] && expectedAppId && existing.rows[0].id !== expectedAppId) {
     throw new Error(`${record.slug} belongs to a different app than app_id`);
@@ -109,6 +111,8 @@ export async function upsertApp(client, record, expectedAppId) {
     throw new Error(`${record.slug} already exists; provide app_id after confirming the merge`);
   }
 
+  if (dryRun) return existing.rows[0]?.id ?? null;
+
   const values = appColumns.map((column) => record[column]);
   const updates = appColumns
     .filter((column) => column !== "slug")
@@ -281,23 +285,26 @@ export async function upsertInstallOptions(client, appId, options, catalogSource
   );
 }
 
-async function candidateById(client, id) {
-  const result = await client.query("select * from public.self_host_app_candidates where id = $1 for update", [id]);
+async function candidateById(client, id, dryRun) {
+  const result = await client.query("select * from public.self_host_app_candidates where id = $1" + (dryRun ? "" : " for no key update"), [id]);
   if (!result.rows[0]) throw new Error(`Missing candidate: ${id}`);
   return result.rows[0];
 }
 
-async function applyReview(client, review) {
+async function applyReview(client, review, dryRun = false) {
   const decision = review.decision ?? "publish";
   if (!decisions.has(decision)) throw new Error(`Unsupported decision: ${decision}`);
   if (!review.candidate_id) throw new Error("candidate_id is required");
-  const candidate = await candidateById(client, review.candidate_id);
+  const candidate = await candidateById(client, review.candidate_id, dryRun);
   if (candidate.status !== "enriching") {
     throw new Error(`Candidate ${candidate.id} is ${candidate.status}; claim it before applying a review`);
   }
 
   if (decision === "duplicate") {
     if (!review.app_id) throw new Error("duplicate decisions require app_id");
+    const target = await client.query("select id from public.self_host_apps where id = $1", [review.app_id]);
+    if (!target.rows[0]) throw new Error(`Missing duplicate app: ${review.app_id}`);
+    if (dryRun) return { candidate_id: candidate.id, decision, app_id: review.app_id };
     await preserveCandidateSources(client, candidate, review.app_id);
     await client.query(
       `
@@ -313,6 +320,7 @@ async function applyReview(client, review) {
 
   if (decision === "reject" || decision === "needs_review") {
     if (!review.reason) throw new Error(`${decision} decisions require reason`);
+    if (dryRun) return { candidate_id: candidate.id, decision };
     const candidateStatus = decision === "reject" ? "rejected" : decision;
     await client.query(
       `
@@ -331,6 +339,16 @@ async function applyReview(client, review) {
     throw new Error(`${validation.record.slug}: ${validation.errors.join("; ")}`);
   }
 
+  for (const slug of validation.record.categories) {
+    if (!getSelfHostAppCategory(slug)) throw new Error(`Unknown category: ${slug}`);
+  }
+  if (dryRun) {
+    const sourceRows = catalogSourceRows(candidate, review);
+    const options = Array.isArray(review.install_options) ? review.install_options : buildInstallOptions(validation.record, sourceRows);
+    if (options.length) installOptionsToRow(options, { catalogSources: sourceRows, checkedAt: new Date().toISOString() });
+    const appId = await upsertApp(client, validation.record, review.app_id ?? null, true);
+    return { candidate_id: candidate.id, decision, app_id: appId, slug: validation.record.slug };
+  }
   await ensureCategories(client, validation.record.categories);
   const appId = await upsertApp(client, validation.record, review.app_id ?? null);
   await replaceCategoryLinks(client, appId, validation.record.categories);
@@ -361,24 +379,31 @@ async function applyReview(client, review) {
   return { candidate_id: candidate.id, decision, app_id: appId, slug: validation.record.slug };
 }
 
+export async function applyReviews(client, reviews, dryRun = false) {
+  const results = [];
+  const ids = reviews.map((review) => review.candidate_id);
+  if (new Set(ids).size !== ids.length) throw new Error("Repeated candidate_id in review batch.");
+  await client.query(dryRun ? "begin read only" : "begin");
+  try {
+    // Lock in ID order, the same order claims and dedupe use, so batches in any order can't deadlock with them.
+    if (!dryRun) {
+      await client.query("select id from public.self_host_app_candidates where id = any($1::uuid[]) order by id for no key update", [ids.filter(Boolean)]);
+    }
+    for (const review of reviews) results.push(await applyReview(client, review, dryRun));
+    await client.query(dryRun ? "rollback" : "commit");
+    return results;
+  } catch (error) {
+    await client.query("rollback");
+    throw error;
+  }
+}
+
 async function main() {
   const args = parseArgs();
+  const databaseUrl = databaseOptions(args);
   if (!args.file) throw new Error("Pass --file with reviewed candidate decisions.");
-  const input = await readJson(args.file);
-  const reviews = reviewRecords(input);
-  const results = [];
-
-  await withClient(async (client) => {
-    await client.query("begin");
-    try {
-      for (const review of reviews) results.push(await applyReview(client, review));
-      if (args["dry-run"]) await client.query("rollback");
-      else await client.query("commit");
-    } catch (error) {
-      await client.query("rollback");
-      throw error;
-    }
-  }, args["database-url"]);
+  const reviews = reviewRecords(await readJson(args.file));
+  const results = await withClient((client) => applyReviews(client, reviews, Boolean(args["dry-run"])), databaseUrl);
 
   console.log(JSON.stringify({ dry_run: Boolean(args["dry-run"]), results }, null, 2));
 }