feat(cli): export and validate OKF v0.2 bundles - #1554
phernandez wants to merge 29 commits into
Conversation
Signed-off-by: phernandez <paul@basicmachines.co>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4215eaf0cc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
|
@codex review All six first-round findings are fixed in 286c68f with focused regressions and replies. Also fixed default filename-title resolution for nested notes. Current validation: 63 OKF unit tests with 100% core module coverage on Python 3.13 and 3.14; four integration tests including pinned upstream timestamp behavior; 53 manual checks; just fast-check and just doctor passed. Please review the current head. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 286c68f1f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0fcf8905b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 64a0be96a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 73a1f83d72
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a16b65f2de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 540b76e0ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 753ee70094
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 207248806d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a53c548afd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9bb9fae004
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a3765de35a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eb82d9f6ba
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45b92bef1b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c46f907fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9bacc52c52
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 961ff851ac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d2f07ab81
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1aee8d2f8f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ec0b373e96
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d0b9b4c519
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e3872755b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d97b053298
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1bb4dc6c2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b71487e413
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5dc3f5076
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: phernandez <paul@basicmachines.co>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f4fc77c30
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Diagnostic(path=str(error.filename), rule="filesystem.read", message=str(error)) | ||
| ) | ||
|
|
||
| for directory, directories, files in os.walk(root, onerror=scan_error, followlinks=False): |
There was a problem hiding this comment.
On Python 3.12, Path.is_symlink() and Path.is_file() can raise errors such as PermissionError for an entry whose metadata becomes unreadable during the walk. Because these predicates are outside the existing read-error handler, bm okf check --json then escapes with an exception instead of returning the documented structured filesystem.read diagnostic. Catch stat failures for each entry and append a diagnostic, as is already done for traversal and content-read failures.
Useful? React with 👍 / 👎.
1 participant
Why
Closes #1550. Basic Memory consumes Markdown links but needs an explicit static OKF producer and filesystem conformance boundary. This adds
bm okf export DESTINATION --project NAME [--replace] [--json]andbm okf check BUNDLE_PATH [--json].What Changed
bm.okf_exportextension.Implementation Details
okf/render.pydefines independent snapshot/file/history values and pure rendering;okf/validation.pyowns structural checks;okf/export.pyhandles the local snapshot and publication. The existing Markdown semantic parser, link normalization/permalink candidates, project scanner, and accepted-change repository are reused. Live Wiki rendering is not used as export output.Export stages beside the destination, validates before publication, and verifies that source files and recorded history did not change. Existing destinations require
--replace. A failed publication restores the previous bundle; if restoration also fails, backup bytes remain outside automatic staging cleanup. Unmarked files at reserved filenames are rejected even without frontmatter. Export reads older databases without migrating them; absent journal schemas yield empty history. PostgreSQL export installs the guarded event-loop policy before async work.Compatibility pins upstream revision
e6d34fd29c1c6c75ec23078e7a8191a9c8209620. The unmodified reference document parser consumes the exported fixture with an asserted two-concept count. The upstream Bitcoin bundle passes validation, indexes all nine concepts, resolves Markdown graph links, and is searchable in Basic Memory. Upstream's Acme log has nonconformant frontmatter and is retained as a negative test. A timestamp round-trip test also preserves the pinned consumer’s stale-after behavior, including ISO T/Z spelling and whitespace-bearing frontmatter fences. No upstream runtime dependency is added. Upstream fixtures retain their original whitespace.Testing
Verified in the dedicated OKF worktree on Python 3.14:
uv run pytest tests/cli/test_man_command.py::test_section_1_pages_document_every_option_of_their_command tests/okf/test_okf.py::test_scalar_permalink_aliases_preserve_authored_metadata -q --no-cov— 3 passed. The man test previously reproducedKeyError: okf-check; both scalar cases previously produced broken links.uv run pytest tests/okf tests/index/test_local_project_scan_parity.py tests/cli/test_ignore_utils.py -q -o addopts='' --cov=basic_memory.okf --cov=basic_memory.ignore_utils --cov-report=term-missing --cov-fail-under=100— 201 passed (152 OKF, 10 scanner, and 39 ignore tests); OKF and ignore-utils modules 100% coverage.uv run pytest tests/cli/test_man_command.py tests/test_man_pages.py test-int/test_okf_integration.py -q --no-cov— 73 passed, including real CLI export and pinned upstream ingestion/retrieval.UV_NO_SYNC=1 just fast-check— passed, including full typecheck, afteruv sync --locked --extra milvusinstalled the optional imports required by the full static check.just doctor— passed.just man-regen— all pages already match their source.The option-documentation test now resolves the nested
okf checkandokf exportcommand paths, matching the generator convention, and checks their options rather than skipping them. Export alias resolution uses canonical frontmatter normalization for numeric and boolean permalinks while preserving authored metadata. Wikilink scanning also uses the canonical escape rule for opening and closing delimiters; regressions cover odd and even backslash runs. Permalink and unique-title identities are resolved before file paths; collision tests compare the result with the canonical ProjectEntityIdentityIndex resolver and preserve explicit relative paths. Duplicate authored permalinks fail with both source filenames before publication; a regression confirms an existing replacement destination stays intact. Rooted wikilinks retain exact paths without suffix inference, and NUL normalization preserves raw source bytes and correct link spans. Project-prefix compatibility candidates are used only for permalink lookup, never to guess filenames. Escaping paths remain literal wikilinks, slash-bearing paths retain canonical filename inference despite duplicate titles, and YAML sets serialize deterministically across process hash seeds while mapping order is preserved. Unordered sets inside title/type fields fail explicitly because their canonical string coercion is hash-dependent. Explicit foreign-project qualifiers remain literal; current-project qualifiers resolve their remainder without a source-relative guess. Exported bodies use deterministic LF line endings; source files are never rewritten. Export uses strict source scanning: per-file stat failures and partial traversal failures abort publication, while ordinary indexing keeps its existing behavior. Notes without authored permalinks receive canonical generated lookup addresses, and staged files use exclusive creation to reject destination filesystem path collisions before publication. Malformed source frontmatter now fails with a repair diagnostic because moved notes can retain an indexed identity absent from disk; network-path wikilinks remain literal to avoid introducing external URLs.The case-insensitive filesystem fixture patches both stat and exists because Python 3.14 bypasses Path.stat in exists; this corrects the Linux 3.14 CI fixture failure without changing production behavior.
Destination containment compares filesystem identities, including differently cased source aliases and ancestors; exact configured project names take precedence over normalized aliases.
Export loads ignore rules in strict read-only mode: missing files use canonical defaults, while read errors abort before publication. Nested resource filename titles participate in canonical title lookup and ambiguity handling.
UUID targets remain literal because indexed external IDs outrank authored aliases and cannot be recovered from a filesystem snapshot. Regressions compare UUID spellings with the canonical bulk resolver and prevent conflicting permalink/title aliases from capturing the link.
Risks / Follow-ups
This is local directory export and structural OKF v0.2 compatibility, not full OKF runtime, attestation, or import support. Cloud projects must first be pulled locally. Export follows project ignore rules and omits symlinks. Markdown concepts require lowercase
.md; other BM Markdown suffixes fail with a rename diagnostic rather than being silently exported as assets. Use a quiescent project: staging/source verification is not a filesystem-wide transaction over concurrent external edits. Concept body observation syntax remains human-readable BM syntax; typed relation metadata preserves semantics that generic Markdown edges cannot carry.Do not merge without explicit authorization.