Repository navigation
Live verification default for AD scoring - #636
Merged
Merged
Conversation
Make full Active Directory scoring deterministic for console agents instead of relying on the model to discover the live verification flag. Co-Authored-By: Codex <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change making live verification the default for built-in Active Directory /score commands.
No open issues 🎉
What changed in this PR
Makes live verification the console-agent default for built-in Active Directory /score commands, while preserving explicit static scoring with --live-verify=false and leaving other range scorers unchanged.
Changes:
- Applies range-aware defaults before agent commands reach the runner.
- Documents the default and static-scoring opt-out in agent guidance and updates console help.
- Adds tests for default injection, explicit opt-out, and range-specific behavior.
| File | Description |
|---|---|
console/frontend/src/help.ts |
Updates /score help to mention the built-in AD live default. |
console/backend/tests/test_commands.py |
Tests range-aware defaults and agent instructions. |
console/backend/tests/test_chat.py |
Tests live defaults and static opt-out through the agent tool. |
console/backend/prompts/score.md |
Documents the live default and explicit static-scoring option. |
console/backend/commands.py |
Adds shared guidance and range-aware argument defaults. |
console/backend/agent.py |
Passes capabilities into the tool and applies and exposes command defaults. |
Review effort: Balanced
Keep executable defaults independent from their human-facing guidance while co-locating both with the command they govern. Co-Authored-By: Codex <noreply@anthropic.com>
Exercise the production helper with empty guidance so prose changes cannot silently disable the live-verification default. Co-Authored-By: Codex <noreply@anthropic.com>
Keep command policy guidance in the system prompt only and derive runnable commands directly from the capability snapshot. Co-Authored-By: Codex <noreply@anthropic.com>
Verify the agent factory gives its prompt and command tool the same capability-derived command set. Co-Authored-By: Codex <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changed
/scoreguidance now documents the range-aware live default in agent instructions and console help.Breaking
/scorenow live-verifies built-in Active Directory results by default; pass--live-verify=falseto retain static scoring.