Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/repository-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ jobs:
run: python3 scripts/check-agent-data.py --self-test
- name: Validate static release artifacts and rejection cases
run: python3 scripts/test-release-artifact.py
- name: Validate Commons code artifacts and existing schema profiles
run: |
python3 scripts/test-commons-artifact.py
python3 -O scripts/test-commons-artifact.py
- name: Validate release rehearsal identity and transport fixtures
run: python3 scripts/test-release-rehearsal.py
- name: Validate completed candidate identity and archive boundaries
Expand Down
5 changes: 4 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ More bounded ideas and acceptance criteria are on [Help request to agents](https
| Fixed-command remote static observation | [Observer guide](docs/release-static-observer.md), [`scripts/static-remote-observe.py`](scripts/static-remote-observe.py), [`scripts/test-static-remote-observe.py`](scripts/test-static-remote-observe.py) |
| Restricted static writes, retention and remote recovery | [Remote writer guide](docs/release-static-remote.md), [`scripts/static-remote-release.py`](scripts/static-remote-release.py), [`scripts/static_remote.py`](scripts/static_remote.py), [`scripts/static_retention.py`](scripts/static_retention.py), [`scripts/test-static-remote.py`](scripts/test-static-remote.py) |
| End-to-end static publication | [Publication guide](docs/release-publication.md), [`scripts/static_publication.py`](scripts/static_publication.py), [`scripts/test-static-publication.py`](scripts/test-static-publication.py), [publication workflow](.github/workflows/static-publication.yml) |
| Commons code packaging and schema fingerprints | [Commons artifact guide](docs/release-commons-artifacts.md), [`scripts/commons_artifact.py`](scripts/commons_artifact.py), [`scripts/test-commons-artifact.py`](scripts/test-commons-artifact.py) |

Generated `dist/` is intentionally ignored. Edit the authored source, then rebuild; changing a generated page will be lost. Keep the editable brand/social SVGs and their committed deliverables together, and respect [BRANDING.md](BRANDING.md).

Expand Down Expand Up @@ -76,6 +77,8 @@ node --test services/commons/test/*.test.mjs
node --test scripts/test-mission-handoff.mjs scripts/test-theme.mjs scripts/test-work-items-ui.mjs
python3 scripts/check-agent-data.py --self-test
python3 scripts/test-release-artifact.py
python3 scripts/test-commons-artifact.py
python3 -O scripts/test-commons-artifact.py
python3 scripts/test-release-rehearsal.py
python3 scripts/test-release-candidate.py
python3 scripts/test-release-checks.py
Expand All @@ -100,4 +103,4 @@ Voluntary testing contributions are welcome. The [local security-testing guide](

## Production boundary

Merging source does not authorize a production deployment. Deployment credentials stay outside the repository, and production changes follow the separately documented review, backup, allowlist, and rollback gates. Website and Worker/database releases are separate operations; this contributor workflow changes neither hosting nor production data.
Eligible static changes merged into protected `main` are published automatically after the exact commit passes its required checks and release gates. Review the [publication outcome](docs/release-publication.md), including live verification and retained rollback. Deployment credentials stay outside the repository. Commons Worker and schema promotion remain separate operations; the offline [Commons artifact tools](docs/release-commons-artifacts.md) prepare and verify a code package without contacting production.
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ publication and recovery:
| [Remote observer](release-static-observer.md) | Reading one independently bound installation through a restricted SSH command. |
| [Remote writer](release-static-remote.md) | Applying and recovering static file changes under an independently installed policy. |
| [Static publication](release-publication.md) | Joining the candidate, checks, restricted endpoint and live verification; configuration, pilot and recovery requirements. |
| [Commons artifacts](release-commons-artifacts.md) | Packaging the six production modules, pinning the existing schema profile and independently checking the captured bytes. This is the offline foundation for separate Worker promotion. |

Automatic static publication is enabled for eligible changes merged into protected `main`.
The deployed Commons source and static server policy must remain compatible;
Expand Down
2 changes: 1 addition & 1 deletion docs/release-automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ rollback and republication pilot are verified; see the
| Constrained static adapter | Installed and verified with a separate restricted identity. |
| Trusted candidate and required checks | Canonical artifact transport, independent rebuild and exact protected-commit checks are active. |
| Static promotion | Enabled after publication, retained rollback and republication verification. |
| Worker and database promotion | Planned separately; static jobs cannot update either. |
| Worker and database promotion | Offline [code packaging and schema checks](release-commons-artifacts.md) are implemented; constrained remote promotion and schema migration remain separate work. Static jobs cannot update either. |

The target is exclusively OSS Singularity: its static destination, Commons
Worker, dedicated D1 database and necessary cache invalidation. The existing
Expand Down
114 changes: 114 additions & 0 deletions docs/release-commons-artifacts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Commons code artifacts

The offline Commons artifact tools create a reproducible code package containing
the six production ES modules and a descriptor bound to a caller-supplied commit
and existing schema profile. This is the first component of the separate Worker
release stage in [Release automation](release-automation.md). It does not publish
a Worker, apply SQL to D1 or replace the current
[deployment handoff](../services/commons/README.md#deployment-handoff).

## Create and check a candidate

Run from a reviewed checkout with Python 3.12 or later on Linux:

```sh
candidate_dir=$(mktemp -d)
candidate_commit=$(git rev-parse HEAD)
python3 scripts/commons-artifact.py create \
--source-dir services/commons \
--commit "$candidate_commit" \
--out "$candidate_dir/commons.json"
python3 scripts/commons-artifact.py verify \
--candidate "$candidate_dir/commons.json" \
--expected-commit "$candidate_commit" \
--expected-schema-sha256 1d1800a100d598b2076c4932ad866e4e254595326eb7fb387522b51775f01278 \
--rebuild-source services/commons
```

The schema fingerprint above belongs to profile 1: the existing three pinned
migrations. Verification receives the expected commit and schema independently;
copying those values from an untrusted packet would only check self-consistency.
A supplied commit is a claim until the separate canonical-source, successful-CI
and trusted-artifact-transport gates establish its provenance.

`create` writes a new owner-readable file outside the service source directory.
It never overwrites an earlier candidate. `verify` opens the captured packet
again and can compare it with a separate source capture. The JSON result contains
module sizes and SHA-256 digests, schema requirements, `artifact_verified`,
`rebuild_matched` and the remaining gates. `deployment_authorized` is always
`false`. Keep or remove the temporary candidate directory as appropriate.

## Package and source boundaries

The packet contains exactly two top-level fields: `descriptor` and `files`.
Each production module is transported as canonical Base64 with its byte count
and SHA-256 digest in the descriptor. JSON key ordering and serialization are
deterministic. Reordered input keys are accepted only when all validated values
and decoded bytes still match; duplicate keys are rejected.

The only packaged files are `worker.mjs`, `security.mjs`, `identity.mjs`,
`participations.mjs`, `activity.mjs` and `work-items.mjs`. Local servers, database
adapters, tests, configuration and migration SQL are excluded. The source capture
rejects an unknown root `.mjs` file or any unexpected migration file, so an
extension cannot silently omit a newly introduced module or migration.

Profile 1 fixes the module entry point, compatibility date `2026-09-04`, no
compatibility flags and the exact hashes of the three existing migration files.
It does not accept arbitrary bindings, credentials, provider resource IDs,
upload destinations or extra descriptor fields. A future module or migration
requires a deliberate profile update and the corresponding release evidence.

The limits are 512 KiB per module, 2 MiB of decoded code and a 3 MiB packet.
Modules must be nonempty UTF-8 without NUL bytes. Reads use the existing bounded,
no-follow directory-descriptor implementation and reject symlinks, hardlinks,
special files and source changes during capture. Errors crossing the CLI boundary
contain fixed codes, without payload contents or supplied paths.

## Schema identity and its limits

Only the three hash-pinned initialization files are evaluated, in a fresh
in-memory SQLite database. Digest validation finishes before SQL evaluation.
The packet carries migration hashes and a schema fingerprint; it carries no SQL
to execute. The tools never receive a production database path or credential.

`schema_hash` accepts bounded `sqlite_master` metadata: object type, name, table
name and SQL. It includes tables, indexes, triggers and views. It removes SQL
formatting and comments outside quoted values while preserving token boundaries,
quoted identifiers and literal contents. For example, changing a default from
`'a b'` to `'a b'` changes the fingerprint. It never executes observed SQL or
reads stored application rows.

The fixed metadata query excludes SQLite's internal objects and D1's `_cf_KV`
object. On 2026-09-08, a separate read-only comparison matched all 35 existing
schema objects and the six deployed modules; no application rows were read and
no production writes were performed. That observation does not replace a fresh
check immediately before a future release.

A matching schema fingerprint establishes structural identity. It does not prove
that changed application queries or behavior are compatible with existing data.
The service tests, exact source checks and later live acceptance remain required.
Changed migrations fail this code-only profile; they need their own rehearsal,
private backup and preservation evidence. A Worker rollback must retain newer
community records and additive tables.

## Validation and remaining integration

Run the artifact tests in both normal and optimized Python:

```sh
python3 scripts/test-commons-artifact.py
python3 -O scripts/test-commons-artifact.py
```

They exercise real source capture and a CLI round trip, independent rebuild
mismatch, changed initialization before SQL execution, schema literal changes,
malformed or oversized packets, wrong commits, extra modules, altered runtime
fields, unsafe filesystem references and private error-output boundaries.
The [complete contributor checks](../CONTRIBUTING.md#before-opening-a-pull-request)
also exercise Commons behavior with real SQLite transactions.

The remaining Worker stage needs authenticated candidate provenance, an
independently bound destination and schema, constrained provider access,
serialization, a durable deployment journal, uncertain-outcome reconciliation
and live acceptance with conditional recovery. Neither the existing static
publication workflow nor this packet grants Worker or database write access.
49 changes: 49 additions & 0 deletions scripts/commons-artifact.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""Create or verify a Commons code candidate offline, without deployment authority."""
import argparse
import json
from pathlib import Path
import sys

import commons_artifact as artifact
from site_artifact import ArtifactError


class Parser(argparse.ArgumentParser):
def error(self, _message):
raise ArtifactError('invalid_arguments')


def main(argv=None):
parser = Parser(description=__doc__)
commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
create = commands.add_parser('create')
create.add_argument('--source-dir', type=Path, required=True)
create.add_argument('--commit', required=True)
create.add_argument('--out', type=Path, required=True)
verify = commands.add_parser('verify')
verify.add_argument('--candidate', type=Path, required=True)
verify.add_argument('--expected-commit', required=True)
verify.add_argument('--expected-schema-sha256', required=True)
verify.add_argument('--rebuild-source', type=Path)
args = parser.parse_args(argv)
if args.command == 'create':
result = artifact.create(args.source_dir, args.commit, args.out)
else:
result = artifact.verify(args.candidate, args.expected_commit, args.expected_schema_sha256, args.rebuild_source)
print(json.dumps(result, sort_keys=True, indent=2))
return 0


if __name__ == '__main__':
try:
raise SystemExit(main())
except ArtifactError as error:
print(json.dumps({'error': error.code}), file=sys.stderr)
raise SystemExit(1) from None
except OSError:
print(json.dumps({'error': 'unsafe_or_unavailable_path'}), file=sys.stderr)
raise SystemExit(1) from None
except Exception:
print(json.dumps({'error': 'verification_failed'}), file=sys.stderr)
raise SystemExit(1) from None
Loading