feat(semantic-layer): metastore scope/ACL support (targeted + org-wide) [AI-3790] - #715
Open
Matovidlo wants to merge 3 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds CLI + service support for the metastore’s new semantic-layer visibility model (PSGO-140): items can be project-private (default), shared with specific projects (targeted grants), or elevated to organization-wide visibility, including new scope management subcommands and scope preservation across DELETE+POST edits.
Changes:
- Add
--scope project|organization|targeted+--target-projectto semantic-layer creation commands, with an interactive picker fallback for targeted scope. - Introduce
semantic-layer scopesubcommands (status/grant/request-elevation/withdraw-elevation/elevate/pending) plus corresponding service/client primitives. - Update metastore envelope schemaVersion to
1.1.0and preserve scope/grants during edit flows.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_semantic_layer_scope.py | Unit tests for scope helpers + service orchestration + edit-preserves-scope regression coverage. |
| tests/test_semantic_layer_scope_cli.py | CLI tests for --scope/--target-project and semantic-layer scope subcommands + permission gating. |
| tests/test_metastore_client.py | Metastore client tests updated for schemaVersion 1.1.0 and new scope/grant endpoints. |
| src/keboola_agent_cli/services/semantic_layer_service.py | Service layer wiring for scope/grants/elevation + passing scope/grants on creates. |
| src/keboola_agent_cli/services/_semantic_layer_scope.py | New helper module: alias→project_id resolution and scope/grant/elevation orchestration logic. |
| src/keboola_agent_cli/services/_semantic_layer_crud.py | Preserve scope/grants across DELETE+POST edits (including rollback). |
| src/keboola_agent_cli/permissions.py | Register permission categories for semantic-layer.scope.* operations. |
| src/keboola_agent_cli/metastore_client.py | Add scope/grant/elevation primitives; bump create envelope schemaVersion to 1.1.0. |
| src/keboola_agent_cli/commands/semantic_layer.py | Wire scope sub-app and add --scope/--target-project to model create. |
| src/keboola_agent_cli/commands/context.py | Update generated context docs for new scope flags and scope subcommands. |
| src/keboola_agent_cli/commands/_semantic_layer_scope.py | New Typer sub-app implementing semantic-layer scope ... CLI surface. |
| src/keboola_agent_cli/commands/_semantic_layer_helpers.py | Add resolve_scope_targets helper with interactive picker / non-interactive fail-fast behavior. |
| src/keboola_agent_cli/commands/_semantic_layer_crud.py | Add --scope/--target-project plumbing to semantic-layer add <kind>. |
| plugins/kbagent/skills/kbagent/SKILL.md | Add command table entries for new semantic-layer scope commands (and sl alias). |
| plugins/kbagent/skills/kbagent/references/semantic-layer-workflow.md | Cross-link to the new scope workflow guidance. |
| plugins/kbagent/skills/kbagent/references/metastore-scope-workflow.md | New workflow doc for sharing/elevation with strong “ask user first” safety rules. |
| plugins/kbagent/skills/kbagent/references/gotchas.md | Add PSGO-140 gotchas (schemaVersion 1.1.0 requirement, replace semantics, 403 vs 404, etc.). |
| plugins/kbagent/skills/kbagent/references/commands-reference.md | Document new scope flags and semantic-layer scope subcommands. |
| plugins/kbagent/agents/keboola-expert.md | Add explicit rule preventing agents from widening scope without user-specified targets. |
| CLAUDE.md | Update command inventory and add scope feature notes + new commands. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+120
to
+126
| if clear and (target_project or remove_target_project): | ||
| formatter.error( | ||
| message="--clear cannot be combined with --target-project/--remove-target-project.", | ||
| error_code=ErrorCode.INVALID_ARGUMENT, | ||
| ) | ||
| raise typer.Exit(code=2) | ||
| result = _handle_service_call( |
Comment on lines
+217
to
+223
| if target_project_ids and scope != "targeted": | ||
| raise KeboolaApiError( | ||
| message=( | ||
| f"target_project_ids is only valid with scope='targeted', got scope={scope!r}." | ||
| ), | ||
| error_code=ErrorCode.VALIDATION_ERROR, | ||
| ) |
Comment on lines
+42
to
+46
| console.print(f"[bold]scope:[/bold] {data.get('scope', 'project')}") | ||
| targets = data.get("target_project_ids") | ||
| if targets: | ||
| console.print(f"[bold]target_project_ids:[/bold] {targets}") | ||
| pending = data.get("scope_elevation_requested_at") |
…semantic layer Adds kbagent CLI support for PSGO-140's metastore scope model: `--scope project|organization|targeted` and `--target-project` on `model create` / `add <kind>`, plus a new `semantic-layer scope` command group (status/grant/request-elevation/withdraw-elevation/elevate/pending) for managing target-project grants and organization-wide elevation on existing items. Bumps the metastore envelope schema version from 1.0.0 to 1.1.0 -- every semantic-* schema only supports scope="project" at 1.0.0; 1.1.0 is what adds organization/targeted support (purely additive ACL block, verified against go-monorepo). Without this the whole feature would 400 server-side. The DELETE+POST `edit` path now reads and re-applies an item's original scope/target-project grants, so editing an organization/targeted-scope item no longer silently resets it to project scope. Widening visibility always requires an explicit --target-project (or an interactive picker on a real terminal; hard fail in --json/non-interactive) -- never a silent default. keboola-expert.md gets an explicit rule to always ask the user before passing --scope organization|targeted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…oint response
`scope request-elevation` / `withdraw-elevation` / `elevate` rendered the
mutating endpoint's own response body. That body omits
`meta.targetProjectIds`, so the commands printed `target_project_ids: null`
for an item whose grants were fully intact.
Verified live against metastore.us-east4 (project keboola-ai, model with
scope=targeted, targets=[5024]):
scope status -> targets=[5024]
request-elevation -> targets=None <-- wrong, grants untouched
scope status -> targets=[5024]
An operator reading that output would reasonably conclude that requesting
elevation had just wiped every grant on the object.
`grant_target_projects` already re-read the item after its PUT for exactly
this reason; the three elevation helpers now do the same. Re-verified live:
command output matches `scope status` at every step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…en for every call (PSGO-282) go-monorepo#596 (PSGO-282) fixed the metastore to accept any valid, non-disabled, non-expired Storage token for reads -- writes still need a project-admin token. Every doc this repo carries about the old behavior (added in #711/#717) still claimed the *whole* semantic-layer family needed a master token, which is now false and would make agents refuse a plain read or hunt for a master token they don't need. Updates CLAUDE.md, keboola-expert.md, commands-reference.md, gotchas.md, semantic-layer-workflow.md and docs/error-codes.md to state the real split (reads: any valid token: writes: project-admin token), and softens metastore_client.py's 401-reclassification message so it no longer overclaims a blanket master-token requirement -- kept as a safety net for a deployment that predates the fix. New gate entries use the vNEXT placeholder per this repo's release convention (check_version_gates.py). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Matovidlo
force-pushed
the
martinvasko-ai-3790-verify-that-the-new-semantic-models-and-datasets
branch
from
August 31, 2026 12:57
e5c7c64 to
8c8fd94
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds kbagent CLI support for the new metastore ACL scope model (PSGO-140): items can now be shared with specific projects or made organization-wide, instead of only ever being project-private.
--scope project|organization|targetedand--target-project ALIAS(repeatable) onsemantic-layer model createand everysemantic-layer add <kind>. Default staysproject(owner-only) — existing behavior is unchanged unless the flags are passed.semantic-layer scopecommand group:status,grant(merge or--replace/--clear),request-elevation,withdraw-elevation,elevate(org-admin only, irreversible),pending(org-admin's discovery queue).--scope targetedand no--target-project: an interactive picker runs on a real terminal;--json/non-interactive fails fast (exit 2) instead of guessing which project(s) should see the object.keboola-expert.mdgets an explicit rule: never widen an item's visibility (--scope organization|targeted,scope elevate,scope request-elevation) without the user having named the target project(s) first — elevation to organization scope has no downgrade endpoint.Bug fixed along the way: kbagent's metastore envelope hardcoded
schemaVersion: "1.0.0", but everysemantic-*schema only supportsscope="project"at that version —organization/targetedrequire1.1.0(verified by diffing all six schema pairs in go-monorepo; purely additive ACL block, no data-shape change). Without this fix, every scope-widening call would 400 server-side. Bumped to1.1.0.Regression fixed along the way: the existing
editcommand (DELETE+POST, no PATCH on the metastore) did not carry an item's scope/target-project grants across the cycle, so editing an organization/targeted-scope item would have silently reset it back to project-only visibility. Fixed by readingmeta.scope/meta.targetProjectIdsbefore the delete and re-applying them on the post.Impact analysis
metastore_client.py: new scope/grant/elevation primitives;post_itemgainsscope/target_project_ids(default"project", fully backwards-compatible); envelopeschemaVersionbumped1.0.0→1.1.0(additive only).services/_semantic_layer_scope.py(new),services/semantic_layer_service.py,services/_semantic_layer_crud.py: scope business logic + edit-path scope preservation.commands/_semantic_layer_scope.py(new),commands/_semantic_layer_crud.py,commands/_semantic_layer_helpers.py,commands/semantic_layer.py: newscopesub-app,--scope/--target-projectflags, interactive-picker/hard-fail helper.permissions.py: newsemantic-layer.scope.*entries (elevateclassified destructive).CLAUDE.md,commands/context.py,commands-reference.md,gotchas.md,SKILL.md(regenerated), newmetastore-scope-workflow.md,semantic-layer-workflow.mdcross-link — full sync verified viascripts/check_command_sync.py.Test plan
tests/test_metastore_client.py(scope validation, elevate/grant/elevation-request/organization-list primitives),tests/test_semantic_layer_scope.py(alias resolution, grant merge/replace, service orchestration, edit-preserves-scope regression),tests/test_semantic_layer_scope_cli.py(CLI flags, interactive-picker fallback, non-TTY hard-fail, permission gating).uv run pytest tests/→ 6410 passed, 183 skipped (e2e, needE2E_API_TOKEN/E2E_URL), 0 failed.make check-equivalent:ruff check,ruff format --check,ty checkall clean;scripts/check_command_sync.py,scripts/check_sentinel_guards.py,scripts/generate_skill.py(SKILL.md regen),tests/test_check_version_gates.pyall pass.Related issues
Linear: https://linear.app/keboola/issue/AI-3790/verify-that-the-new-semantic-models-and-datasets