feat(governance): Policy Agent batch settings + per-task manual-grant reason#28746
feat(governance): Policy Agent batch settings + per-task manual-grant reason#28746yan-3005 wants to merge 6 commits into
Conversation
… reason Supporting OpenMetadata changes for the Collate Data Access Request (DAR) concurrency work — clubbing concurrent policy-agent grants into a single ingestion run. - workflowSettings: add policyAgentConfiguration (pollingIntervalSeconds, batchWindowSeconds, batchMaxSize), read by the Collate batch coordinator. - policyAgentTaskDefinition: add pollingIntervalSeconds node config. - CreateTask: optionally carry a workflow-supplied reason into the created task's payload as `manualGrantReason` (its own structured field). Strict no-op (try/catch + instanceof) for workflows that do not set it, so generic approval workflows (Glossary, etc.) are byte-for-byte unaffected. - Tests: CreateTaskManualGrantReasonTest; the 44 generic CreateTaskTest pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
✅ PR checks passedThe linked issue has a description and all required Shipping project fields set. Thanks! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
yan-3005
added
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR extends OpenMetadata governance/workflow capabilities to support Policy Agent batching for Data Access Request (DAR) concurrency, and adds support for carrying a workflow-supplied manual-grant reason into created Task payloads as a structured field.
Changes:
- Added
policyAgentConfigurationtoworkflowSettings(polling interval + batching window/size controls) for the Policy Agent batch coordinator. - Extended
PolicyAgentTaskDefinitionconfig withpollingIntervalSeconds(per-node override). - Updated
CreateTaskto optionally merge a workflow-suppliedmanualGrantReasoninto task payload, with new unit tests covering merge behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| openmetadata-spec/src/main/resources/json/schema/governance/workflows/elements/nodes/automatedTask/policyAgentTaskDefinition.json | Adds per-node polling interval setting for Policy Agent task nodes. |
| openmetadata-spec/src/main/resources/json/schema/configuration/workflowSettings.json | Introduces server-wide Policy Agent batching configuration under workflow settings. |
| openmetadata-service/src/test/java/org/openmetadata/service/governance/workflows/elements/nodes/userTask/CreateTaskManualGrantReasonTest.java | Adds unit tests validating manualGrantReason payload merge behavior. |
| openmetadata-service/src/main/java/org/openmetadata/service/governance/workflows/elements/nodes/userTask/CreateTask.java | Merges optional workflow manualGrantReason into task payload during create/update. |
✅ TypeScript Types Auto-UpdatedThe generated TypeScript types have been automatically updated based on JSON schema changes in this PR. |
🔴 Playwright Results — 2 failure(s), 13 flaky✅ 4265 passed · ❌ 2 failed · 🟡 13 flaky · ⏭️ 88 skipped
Genuine Failures (failed on all attempts)❌
|
- workflowSettings.policyAgentConfiguration: add `minimum: 1` to pollingIntervalSeconds / batchWindowSeconds / batchMaxSize; add `batchWorkerThreads` (default 4) so the coordinator's worker-pool size is configurable instead of a hard-coded cap. - policyAgentTaskDefinition: add `minimum: 1` to pollingIntervalSeconds and correct the fallback doc path to workflowSettings.policyAgentConfiguration.pollingIntervalSeconds. - Regenerate UI TypeScript for WorkflowSettings and PolicyAgentTaskDefinition so the generated types stay in sync with the schema. - Align the new test's license-header year (2026) with neighbouring tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
✅ TypeScript Types Auto-UpdatedThe generated TypeScript types have been automatically updated based on JSON schema changes in this PR. |
…er-wide default Removing the per-node default (30) makes the generated field nullable, so an unset node value is now distinguishable from a configured one. The Collate PolicyAgentTask resolves it to the node override → server-wide policyAgentConfiguration.pollingIntervalSeconds → built-in default, which is what the field description already promised. (minimum:1 stays.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mergeManualGrantReason called JsonUtils.getMap (a Jackson convertValue) even when the payload was already a Map, which can re-type nested values and can throw. Copy the existing map entries directly (mirroring withGrantExpirationDate) and only fall back to a guarded conversion for the unexpected non-Map case, so task creation is never broken by a conversion issue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code Review ✅ Approved 2 resolved / 2 findingsImplements Policy Agent batching and manual-grant reason support for DAR concurrency, resolving issues with schema documentation and missing bounds on interval settings. ✅ 2 resolved✅ Quality: Schema doc references wrong settings path
✅ Edge Case: No minimum bound on new integer interval settings
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
Jest test CoverageUI tests summary
|
|
|
Fixes #28759
What
Supporting OpenMetadata changes for the Collate Data Access Request (DAR) concurrency work — clubbing concurrent policy-agent grants into a single ingestion run per pipeline (Collate PR links to this one).
workflowSettings— addpolicyAgentConfiguration:pollingIntervalSeconds(default 30),batchWindowSeconds(default 60, the scheduler/window),batchMaxSize(default 200),batchWorkerThreads(default 4, the coordinator's worker-pool size — bounds how many distinct pipelines provision concurrently). All boundedminimum: 1.policyAgentTaskDefinition— add an optional per-nodepollingIntervalSeconds. When unset it inherits the server-widepolicyAgentConfiguration.pollingIntervalSeconds(then the built-in default); it has no own default so "unset" stays meaningful.CreateTask— optionally carry a workflow-supplied reason into the created task's payload asmanualGrantReason(its own structured field), instead of overloading the description.Generic approval workflows are unaffected
The
CreateTaskchange is a strict no-op for any workflow that doesn't set themanualGrantReasonworkflow variable (Glossary approval, tag/description suggestions, test-case resolution, …): the read is wrapped intry/catch+instanceof, andmergeManualGrantReason(payload, null)returns the payload unchanged. So those task types are byte-for-byte identical to before.Tests
CreateTaskManualGrantReasonTest— no-op for null/blank reason, setspayload.manualGrantReason, preserves existing payload keys, overwrites on stage re-entry.CreateTaskTest(44 existing) still pass — the generic task-creation path is unchanged.🤖 Generated with Claude Code