Step3 manifest#102
Open
hyperpolymath wants to merge 9 commits into
Open
Conversation
…trees Adds three ADRs for verisimiser and one for verisimdb-data, plus the mechanical file deletions they authorise. - ADR-0001 (octad-ontology): concerns octad is canonical; modalities become Tier 2 overlays. Closes #19; sets up #20; closes #21 wontfix. - ADR-0002 (verification-tree): strip the empty 8-subdirectory tree; Idris2 stubs in src/interface/abi/ are unaffected. Closes #15. - ADR-0003 (justfile-aspirational-recipes): delete recipes that name non-existent clap subcommands. Closes #11 (#10 is the mechanical follow-up). - ADR-0001 (verisimdb-data, repo-purpose): repo carries two explicit purposes (scan store + ABI dogfood). Lands in the data repo commit. Deletes: - examples/SafeDOMExample.res, examples/web-project-deno.json (unrelated template flotsam — closes #12) - root SECURITY.md, root CODE_OF_CONDUCT.md (duplicate; .github/ versions are canonical — closes #13, #14) - verification/ subtree (closes #15 via ADR-0002) Closes #11, #12, #13, #14, #15, #19, #21 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…scaping Two bugs in tests/integration_test.rs caused 2 of 9 integration tests to fail (the unit tests were unaffected). 1. Prefix mismatch — codegen emits identifiers prefixed `verisimdb_` (see src/codegen/overlay.rs). The integration tests asserted substring presence of `verisim_…` which is not a substring of `verisimdb_…`. Replaced 11 occurrences in tests/integration_test.rs. 2. Windows path escaping — test_end_to_end_file_workflow interpolates `schema_path.display()` into a TOML basic string with `"…"`. On Windows the path contains backslashes which TOML treats as escapes, producing a malformed manifest and an unwrap-on-Err. Switched the embedded path to a TOML literal string (single quotes) which suppresses escape interpretation. Verified: cargo test now reports 26 + 26 + 9 = 61 tests, 0 failed. Closes #8 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per ADR-0003. The previous recipe block contained literal `\n` characters where newlines were intended, collapsing three recipes into one syntactically broken rule whose target name embedded `\n`. Even with newlines restored the recipes pointed at clap subcommands that don't exist in src/main.rs. Replaced the block with a comment placeholder noting why it was removed and what to do when the subcommands ship. Closes #10 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 13-lint allow block in both lib.rs and main.rs silenced clippy across
the codebase, making `just lint` (`cargo clippy -- -D warnings`) a hollow
signal. Removed both blocks and fixed every lint clippy surfaced.
Fixes:
- codegen/query.rs:124 — nested format!() flagged by
`clippy::format_in_format_args`. Combined `format!("{}::text",
format!("{}.ctid", t))` into `format!("{}.ctid::text", t)`.
- manifest/mod.rs:309 — `init_manifest` had a dead ternary returning
"false" on both branches (flagged by `clippy::if_same_then_else`).
Replaced with a single binding plus a comment explaining where the
per-backend toggle would go if/when it becomes real.
- main.rs — was re-declaring `mod abi; mod codegen; mod intercept;
mod manifest; mod tier1; mod tier2;` already declared in `lib.rs`,
so each module compiled twice. From the bin's perspective most of
the ABI types (ProvenanceEntry, LineageEdge, TemporalVersion,
AccessPolicy, SidecarConfig, DriftCategory, …) appeared as dead
code. Replaced the six `mod …;` lines with `use verisimiser::{abi,
codegen, manifest};` so the bin consumes the library properly.
This also halves redundant test runs (35 unique tests instead of
61 with duplicates).
Verified:
- `cargo clippy --all-targets -- -D warnings` exits clean
- `cargo test` reports 26 lib + 9 integration tests, 0 failed
Closes #16, #17
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per ADR-0001 the canonical octad is concerns (Data/Metadata/Provenance/Lineage/Constraints/AccessControl/Temporal/ Simulation), not modalities. The previous README led with a modalities table the codebase no longer supported. README.adoc rewrites: - Replace the "Eight Modalities" table with an "Eight Concerns" table whose rows match `OctadDimension` enum, OctadConfig fields, and the emitted sidecar tables. - Reframe the eight cross-modal drift categories under Constraints (they are symptoms observed by Constraints when Data, Metadata, and active Tier 2 overlays disagree). Note explicitly that each category still needs a computable definition. - Tier 1 narrative reorganised around the five Tier 1 concerns (Provenance, Temporal, Constraints, Lineage, AccessControl). - Tier 2 retains modalities but as overlay representations, not as "the octad". - Add a "Related repos" section linking verisimdb-data. - Add an "ABI" section pointing at src/interface/abi/ and src/interface/ffi/ where the Idris2 and Zig stubs actually live. - Cite ADR-0001 and ADR-0002 inline. ROADMAP.adoc rewrites: - Phase 0 marked complete with accurate evidence (ABI types exist in three languages; codegen scaffolding ships). - Phases reordered to match the bottom-up plan: SQLite Tier 1 MVP first (cheapest end-to-end), then PostgreSQL, then multi-backend, then Constraints/Drift, then AccessControl/Lineage, then Tier 2 modality overlays, then Simulation, then VCL-total integration, then production hardening, then ecosystem. - Each phase phrased in concerns/modality terms consistent with ADR-0001. Closes #20 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…r commit The `rust-ci.yml` lane already runs `cargo check + clippy + fmt` (job `check`) and `cargo test --all-targets` (job `test`, depends on `check`). That is exactly what V-L3-A2 asked for — the previous mega-list incorrectly claimed no cargo-test gate existed. Real improvement this commit lands: - Add a `concurrency` block at workflow level so push storms cancel older queued runs instead of queueing for minutes. Group keyed on `github.ref` so concurrent PR pushes serialize per-PR but parallel to main. Verified locally: - `cargo fmt --all -- --check` clean - `cargo clippy --all-targets -- -D warnings` clean - `cargo test` reports 35 tests (26 lib + 9 integration), 0 failed The cargo-test gate becomes a required check when branch protection is configured to require Rust CI / Cargo test — that is a repo settings change, not a workflow change. Closes #9 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…-L2-L1..L2
Step 2 of the bottom-up plan. Brings the Provenance octad concern up to
the claim made in the README: tampering with any audit-relevant field
in a logged entry breaks `verify()`.
V-L1-B1 — docs/theory/provenance-threat-model.adoc:
Four-adversary model (R / SW / SR / SR+CK), per-adversary protection
matrix, the field-coverage and canonical-encoding requirements that
bind V-L2-C1 + V-L2-C2, the append-serialisation requirement that
binds V-L2-L1 + V-L2-L2, anchor/notary future work, open questions
(None vs Some(""), chain_id). Each Step 2 issue cites a section.
V-L2-N1 — deduplicate ProvenanceRecord vs ProvenanceEntry:
Delete src/tier1/provenance.rs::ProvenanceRecord (orphan duplicate
of abi::ProvenanceEntry with its own compute_hash that risked
drifting). tier1/provenance.rs now re-exports the canonical type;
the file is the future home of V-L1-C1's write-path helpers
(sqlite3_update_hook → append_provenance). TOPOLOGY.md updated.
V-L2-C1 — full-field, domain-separated hash:
compute_hash signature changes from (4 strs) to (5 strs + DateTime +
2 Options). New preimage = domain tag b"verisim-prov-v1\0" ||
length-prefixed (previous_hash, entity_id, operation, actor) ||
canonical timestamp (V-L2-C2) || length-prefixed (before_snapshot,
transformation). All seven fields participate. PROV_DOMAIN_TAG
versioning is reserved for a future SHA-256→? migration.
verify(), genesis(), chain() all pass the full field set.
V-L2-C2 — canonical timestamp:
Replace timestamp.to_rfc3339() (multiple valid forms per instant)
with i64_le(timestamp()) || u32_le(timestamp_subsec_nanos()), 12
bytes total. Round-trip unit test asserts two construction paths
that yield the same instant produce the same hash.
V-L2-C3 — positive tamper-detection tests:
Eight new unit tests in abi::tests covering each hash-covered
field (entity_id, actor, before_snapshot, transformation,
operation, previous_hash, timestamp) plus the canonical-encoding
property test plus a 4-entry chain mutation-matrix that asserts
every field mutation on every entry breaks verify(). 9 new test
cases (26 → 35 lib tests).
V-L2-C4 — flip the wontfix test:
tests/integration_test.rs::test_provenance_chain_integrity_multi_step
previously codified the bug ("Actor is not part of hash — tamper to
actor alone is invisible"). Replaced with assertions that
tampering with actor and with before_snapshot both break verify().
V-L2-L1 — chain_head table + write-path serialisation spec:
codegen/overlay.rs emits a new verisimdb_provenance_chain_head
(entity_id PK, head_hash, updated_at) alongside the provenance log.
The write-path lock (SELECT … FOR UPDATE / BEGIN IMMEDIATE on the
head row, INSERT into log, UPDATE head, COMMIT) is specified in
the threat-model doc and the table-generator docstring. The
library function that performs the transaction is V-L1-C1's job;
V-L2-L1 only lands the schema.
V-L2-L2 — UNIQUE INDEX makes forks unrepresentable:
CREATE UNIQUE INDEX IF NOT EXISTS ux_provenance_chain ON
verisimdb_provenance_log(entity_id, previous_hash). Genesis rows
all carry previous_hash='' so the same constraint enforces exactly
one genesis per entity. Two new DDL tests assert presence of both
the UNIQUE INDEX and the chain_head table.
Verified locally:
- cargo fmt --all -- --check clean
- cargo clippy --all-targets -- -D warnings clean
- cargo test reports 35 + 9 = 44 tests, 0 failed
Closes #25, #26, #27, #28, #29, #30, #31, #32
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…init
Step 3 of the bottom-up plan. Three issues in one commit because the
changes interlock on OctadConfig's shape and DatabaseConfig's
effective_backend signature.
V-L2-D1 — explicit `enable_constraints`:
Constraints was a first-class concern in ADR-0001 but was treated
in code as "implied when count > 2". Promoted to an explicit
OctadConfig field with serde default = true. The "+ 1 if count > 2"
arithmetic in enabled_count is gone; the new implementation is a
straight popcount over the six optional toggles plus 2 inherent
dimensions. print_status reads enable_constraints directly.
Two new unit tests assert the count is bounded by 2..=8 across
all 64 toggle combinations and that the arithmetic is exact.
V-L2-E1 — refuse conflicting backend/target_db:
effective_backend was value-based: `backend != "postgresql"` was
used as a proxy for "user-set", which silently picked sqlite when
a user explicitly set `backend = "postgresql"` alongside a legacy
`target-db = "sqlite"`. Replaced with a fall-through match that
errors loudly on conflict and returns `Ok(default)` otherwise.
Signature is now `Result<&str>`; callers in main.rs + print_status
propagate. Four new unit tests cover each branch (conflict,
agreement, modern-only, legacy-only, default).
V-L2-O1 — init_manifest reads Default + adds --force/--name:
Template values are now derived from OctadConfig::default() rather
than hardcoded strings, so flipping a default in code automatically
flows into the generated file. Added --force (overwrite existing)
and --name (override project name) flags to `verisimiser init`.
A regression test asserts the OctadConfig::default() invariant the
template depends on.
Test fixtures in codegen/{overlay,query}.rs and the integration test
add the new enable_constraints field. tests/integration_test.rs's
backward-compat case calls .effective_backend().unwrap() for the
new Result signature.
Verified locally:
- cargo fmt --all -- --check clean
- cargo clippy --all-targets -- -D warnings clean
- cargo test reports 42 lib + 9 integration = 51 tests, 0 failed
(was 35 + 9 = 44; +7 manifest unit tests)
Closes #34, #35, #36
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…rov fix (#67) Step 4 partial. Lands the mechanical DDL-correctness work (V-L2-G1, H1, H2, I1, J1, K1). The bigger architectural items in Step 4 stay filed (V-L2-A1 sqlparser replacement, V-L2-B2 composite-id hashing, V-L2-F1 dialect split) — each needs a dedicated session. V-L2-G1 — identifier validation: Added `validate_identifier` / `must_validate_identifier` to overlay.rs accepting only `^[A-Za-z_][A-Za-z0-9_]*$`. Every user-controlled identifier flowing into `INSERT OR IGNORE INTO verisimdb_metadata VALUES ('{}', ...)` is now validated at codegen time, so a table named `posts'); DROP TABLE x;--` is rejected with a structured error instead of injected. Two new test sets cover 5 safe names and 10 attack strings. V-L2-K1 — provenance latest-per-entity view fixed: The previous greatest-N-per-group subquery had a broken correlation (inner MAX subquery referenced the outer uncorrelated row rather than the alias). Replaced with the canonical ROW_NUMBER() OVER (PARTITION BY entity_id ORDER BY timestamp DESC) = 1 pattern, which works on SQLite 3.25+ and PostgreSQL. The integration test for the view now asserts the new pattern and the absence of the old broken correlation. V-L2-H1 + V-L2-H2 — temporal exactness: - CREATE UNIQUE INDEX (was non-unique partial); enforces exactly one current row per (entity, table) at DB level instead of relying on application-layer discipline. - CHECK valid_to IS NULL OR valid_to >= valid_from. - CHECK version >= 1. V-L2-I1 — lineage self-edges forbidden: CHECK NOT (source_entity = target_entity AND source_table = target_table). Cycle prevention beyond self-edges is V-L1-G1 (runtime concern, separate ADR). V-L2-J1 — closed-set CHECKs and the missing FK: - provenance_log.operation ∈ {insert,update,delete,transform} - lineage_graph.derivation_type ∈ {copy,transform,aggregate,join,filter} - temporal_versions.operation ∈ {insert,update,rollback} - access_policies.access_level ∈ {read,write,admin,deny} - access_policies.active ∈ {0,1} - simulation_branches.status ∈ {active,merged,abandoned} - simulation_deltas.operation ∈ {insert,update,delete} - simulation_branches.parent_branch REFERENCES simulation_branches.branch_id (self-FK; was declared but un-enforced). DDL tests added for every constraint above (7 new test functions). Verified locally: - cargo fmt --all -- --check clean - cargo clippy --all-targets -- -D warnings clean - cargo test reports 49 lib + 9 integration = 58 tests, 0 failed (was 42 + 9 = 51; +7 codegen tests) Closes #39, #40, #41, #42, #43 Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
RSR Quality Checklist
Required
just testor equivalent)just fmtor equivalent)unsafeblocks without// SAFETY:commentsbelieve_me,unsafeCoerce,Obj.magic,Admitted,sorry).envfiles includedAs Applicable
.machine_readable/STATE.a2mlupdated (if project state changed).machine_readable/ECOSYSTEM.a2mlupdated (if integrations changed).machine_readable/META.a2mlupdated (if architectural decisions changed)TOPOLOGY.mdupdated (if architecture changed)CHANGELOGor release notes updatedsrc/interface/abi/andsrc/interface/ffi/consistent)Testing
Screenshots