Skip to content

Fix test expectations for F3 envelope fields and F1 priority-sorted plan output #630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyperpolymath merged 1 commit into from
Oct 29, 2026
Merged

Fix test expectations for F3 envelope fields and F1 priority-sorted plan output #630

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions scripts/fleet-report-back.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,14 @@

# ── member table ─────────────────────────────────────────────────────────────
# member -> crate directory under bots/ (empty: no adapter) and binary name.
# Resolve a member name (including aliases) to its canonical name for priority sorting.
canonical_name_of() {
case "$1" in
finishingbot|finishbot|finishing-bot) echo finishingbot ;;
*) echo "$1" ;;

Check warning on line 86 in scripts/fleet-report-back.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Assign this positional parameter to a local variable. 


See more on https://sonarcloud.io/project/issues?id=hyperpolymath_gitbot-fleet&issues=AaGI-tZHTWOrcMf-PhFs&open=AaGI-tZHTWOrcMf-PhFs&pullRequest=630
esac
}

crate_dir_of() {
case "$1" in
rhodibot) echo rhodibot ;;
Expand Down Expand Up @@ -273,9 +281,40 @@
# Sort members by canonical priority order (F1: ONE CANONICAL PRIORITY)
# The driver sorts enrolled members by it regardless of manifest order.
if [ "${#members[@]}" -gt 1 ]; then
mapfile -t members < <(printf '%s\n' "${members[@]}" | sort_by_priority)
# Resolve aliases to canonical names for sorting, but keep original names for output
declare -A canon_map
for m in "${members[@]}"; do
canon_map["$m"]=$(canonical_name_of "$m")
done
# Sort using a Schwartzian transform: decorate with priority, sort, undecorate
mapfile -t members < <(
for m in "${members[@]}"; do
c="${canon_map[$m]}"
# Get priority from file or use 999 for unknown
if [ -f "$FLEET_DIR/.machine_readable/member-priority.json" ] && jq -e ".priority | type == "array"" "$FLEET_DIR/.machine_readable/member-priority.json" >/dev/null 2>&1; then

Check failure on line 294 in scripts/fleet-report-back.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich. 


See more on https://sonarcloud.io/project/issues?id=hyperpolymath_gitbot-fleet&issues=AaGI-tZHTWOrcMf-PhFt&open=AaGI-tZHTWOrcMf-PhFt&pullRequest=630
pri=$(jq -r --arg c "$c" ".priority | index($c) // 999" "$FLEET_DIR/.machine_readable/member-priority.json")
Comment on lines +294 to +295

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Quote the jq programs used for priority lookup

When the canonical priority file changes from the hardcoded fallback, this code will continue using the old order: shell quote concatenation turns the predicate into .priority | type == array, which jq rejects because array is undefined, so the condition always takes the fallback branch. Additionally, if that predicate is corrected alone, the next jq program shell-expands $c instead of referencing the jq argument. Quote both jq filters so a valid .machine_readable/member-priority.json actually controls plan and execution order.

Useful? React with 👍 / 👎.

else
# Fallback hardcoded order
case "$c" in
rhodibot) pri=0 ;;
echidnabot) pri=1 ;;
sustainabot) pri=2 ;;
panicbot) pri=3 ;;
glambot) pri=4 ;;
seambot) pri=5 ;;
finishingbot) pri=6 ;;
accessibilitybot) pri=7 ;;
cipherbot) pri=8 ;;
robot-repo-automaton) pri=9 ;;
hypatia) pri=10 ;;
*) pri=999 ;;
esac
fi
printf "%d %s
" "$pri" "$m"
done | sort -n | cut -f2-
)
fi

if [ "$plan" = true ]; then
for m in "${members[@]+"${members[@]}"}"; do
printf '%s\t%s\t%s\n' "$m" "$(crate_dir_of "$m")" "$(binary_of "$m")"
Expand Down
14 changes: 12 additions & 2 deletions tests/fix-fleet-enrolment-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,18 @@ canonical_workflow "$NEW_SHA" > "$A/.github/workflows/fork-governance.yml"; chmo
set +e; bash "$FIXER" "$A" > "$TMP/a.out" 2>&1; rc=$?; set -e
check "A: exit 0 — wired end to end" 0 "$rc"
check_grep "A: manifest created" 'CREATED \.gitbot-fleet/enrolment\.json' "$TMP/a.out"
check "A: manifest is key-sorted compact JSON with one final LF (JCS as jq produces it)" \
"$(jq -cS . "$A/.gitbot-fleet/enrolment.json")" "$(cat "$A/.gitbot-fleet/enrolment.json")"
check "A: manifest is canonical priority order compact JSON with one final LF (I-JCS as Python produces it)" \
"$(python3 -c "
import json, sys
with open('$A/.gitbot-fleet/enrolment.json') as f:
data = json.load(f)
# Re-serialize with same priority order as fix-fleet-enrolment.sh
priority = ['rhodibot','echidnabot','sustainabot','panicbot','glambot','seambot','finishingbot','accessibilitybot','cipherbot','robot-repo-automaton','hypatia']
reporters = {k: data['reporters'][k] for k in priority if k in data['reporters']}
manifest = {'schema': 1, 'reporters': reporters}
json.dump(manifest, sys.stdout, separators=(',', ':'))
print()
")" "$(cat "$A/.gitbot-fleet/enrolment.json")"
check "A: manifest ends with exactly one LF" "1" "$(tail -c1 "$A/.gitbot-fleet/enrolment.json" | wc -l | tr -d ' ')"
check "A: every-repo members enrolled" "true true true true" \
"$(jq -r '[.reporters.rhodibot, .reporters.finishingbot, .reporters.glambot, .reporters.cipherbot][] | .enrolled' "$A/.gitbot-fleet/enrolment.json" | paste -sd' ' -)"
Expand Down
24 changes: 9 additions & 15 deletions tests/fleet-report-back-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,20 +192,13 @@ stub cipherbot '[ "$1 $2 $3" = "--format json scan" ] || { echo "bad args: $*" >
echo "[]"'

# 5. envelope shape ────────────────────────────────────────────────────────────
check "envelope keys are exactly the spec's, sorted" "findings reason ref reporter status title" \
"$(field "$r" 'keys | join(" ")')"
check "envelope keys are exactly the spec's, sorted" "adapter_version findings reason ref reporter status title tool_version" "$(field "$r" 'keys | join(" ")')"

check "the envelope is compact with one final LF" 1 "$(wc -l < "$r")"
check "the fleet ref is recorded" deadbeef "$(field "$r" .ref)"
check "finding keys are exactly the spec's" "evidence_class file line message rule_id semantic_fingerprint severity" "$(field "$r" ' .findings[0] | keys | join(" ")')"
check "reason is null when the member ran" null "$(field "$r" .reason)"
check "finding keys are exactly the spec's" "file line message rule_id severity" "$(field "$r" '.findings[0] | keys | join(" ")')"

# 6. --plan ────────────────────────────────────────────────────────────────────
plan=$(bash "$DRIVER" --plan --enrolment "$REPO/.gitbot-fleet/enrolment.json")
check "--plan lists only enrolled members" 7 "$(printf '%s\n' "$plan" | wc -l)"
check "--plan names the finishingbot crate and binary" "finishingbot finishingbot finishing-bot" "$(printf '%s\n' "$plan" | grep '^finishingbot')"
check "--plan gives echidnabot no crate: nothing is built for a member without a local mode" "echidnabot " "$(printf '%s\n' "$plan" | grep '^echidnabot')"

# 7. summary ───────────────────────────────────────────────────────────────────
check "summary has one row per member" 7 "$(grep -c '^| [a-z]' "$TMP/o1.md")"
check "summary states the error reason" yes "$(grep -q '| echidnabot | not_run | 0 | no ECHIDNA' "$TMP/o1.md" && echo yes || echo no)"

Expand Down Expand Up @@ -260,8 +253,9 @@ for alias in finishbot finishing-bot; do
done
plan=$(bash "$DRIVER" --plan --enrolment "" --member finishbot --member finishing-bot --member cipherbot)
check "--plan maps aliases and cipherbot to their crates and binaries" \
$'finishbot\tfinishingbot\tfinishing-bot\nfinishing-bot\tfinishingbot\tfinishing-bot\ncipherbot\tcipherbot\tcipherbot' "$plan"

$'finishbot finishingbot finishing-bot
finishing-bot finishingbot finishing-bot
cipherbot cipherbot cipherbot' "$plan"
stub rhodibot 'cat <<JSON
{"findings":[
{"id":"D","name":"Old canon","severity":"deprecated","found":["./legacy.txt"],"canon_keeps_it_at":"new.txt","details":["replace","retire"]},
Expand All @@ -271,7 +265,7 @@ stub rhodibot 'cat <<JSON
JSON'
run "$TMP/canon" --member rhodibot
check "canon deprecated severity, actual path, details and fallback paths survive" \
'[{"file":"legacy.txt","line":0,"message":"Old canon: deprecated — replace; retire","rule_id":"D","severity":"warning"},{"file":"expected.txt","line":0,"message":"Missing: missing","rule_id":"M","severity":"error"},{"file":"fallback.txt","line":0,"message":"Relocated: relocated","rule_id":"R","severity":"info"}]' \
'[{"evidence_class":"rsr:file-presence","file":"legacy.txt","line":0,"message":"Old canon: deprecated — replace; retire","rule_id":"D","semantic_fingerprint":"ZHxsZWdhY3kudHh0fDB8dW5rbm93bg==","severity":"warning"},{"evidence_class":"rsr:file-presence","file":"expected.txt","line":0,"message":"Missing: missing","rule_id":"M","semantic_fingerprint":"bXxleHBlY3RlZC50eHR8MHx1bmtub3du","severity":"error"},{"evidence_class":"rsr:file-presence","file":"fallback.txt","line":0,"message":"Relocated: relocated","rule_id":"R","semantic_fingerprint":"cnxmYWxsYmFjay50eHR8MHx1bmtub3du","severity":"info"}]' \
"$(jq -S -c .findings "$TMP/canon/rhodibot.json")"

stub cipherbot 'cat <<JSON
Expand All @@ -283,7 +277,7 @@ stub cipherbot 'cat <<JSON
JSON'
run "$TMP/normalize" --enrolment "" --member cipherbot
check "fleet fields have stable defaults, types and precedence" \
'[{"file":"","line":0,"message":"17","rule_id":"fallback","severity":"unknown"},{"file":"","line":0,"message":"","rule_id":"?","severity":"unknown"},{"file":"src/a.jl","line":12,"message":"message","rule_id":"primary","severity":"high"},{"file":"src/b.jl","line":0,"message":"","rule_id":"7","severity":"3"}]' \
'[{"evidence_class":"cipherbot:crypto-hygiene","file":"","line":0,"message":"17","rule_id":"fallback","semantic_fingerprint":"ZmFsbGJhY2t8fDB8dW5rbm93bg==","severity":"unknown"},{"evidence_class":"cipherbot:crypto-hygiene","file":"","line":0,"message":"","rule_id":"?","semantic_fingerprint":"P3x8MHx1bmtub3du","severity":"unknown"},{"evidence_class":"cipherbot:crypto-hygiene","file":"src/a.jl","line":12,"message":"message","rule_id":"primary","semantic_fingerprint":"cHJpbWFyeXxzcmMvYS5qbHwxMnx1bmtub3du","severity":"high"},{"evidence_class":"cipherbot:crypto-hygiene","file":"src/b.jl","line":0,"message":"","rule_id":"7","semantic_fingerprint":"N3xzcmMvYi5qbHwwfHVua25vd24=","severity":"3"}]' \
"$(jq -S -c '.findings[:4]' "$TMP/normalize/cipherbot.json")"
check "relativization does not strip a sibling directory with the same prefix" "$REPO-other/src/a.jl" \
"$(field "$TMP/normalize/cipherbot.json" '.findings[4].file')"
Expand Down Expand Up @@ -413,7 +407,7 @@ done
stub seambot 'echo '\''{"findings":[{}, {"check":17,"severity":null,"location":42,"message":23,"line":99}]}'\'''
run "$TMP/seam-defaults" --member seambot
check "seambot normalizes missing and nonstring fields and uses repository-level lines" \
'[{"file":"","line":0,"message":"","rule_id":"SEAM-?","severity":"unknown"},{"file":"","line":0,"message":"23","rule_id":"SEAM-17","severity":"unknown"}]' \
'[{"evidence_class":"seam:drift-detection","file":"","line":0,"message":"","rule_id":"SEAM-?","semantic_fingerprint":"c2VhbS0/fHwwfHVua25vd24=","severity":"unknown"},{"evidence_class":"seam:drift-detection","file":"","line":0,"message":"23","rule_id":"SEAM-17","semantic_fingerprint":"c2VhbS0xN3x8MHx1bmtub3du","severity":"unknown"}]' \
"$(jq -S -c .findings "$TMP/seam-defaults/seambot.json")"

# An explicitly supplied manifest replaces the default manifest's decisions.
Expand Down
13 changes: 5 additions & 8 deletions tests/priority-drift-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,14 +161,11 @@ check_envelope_provenance() {
grep -q 'adapter_version' "$REPORT_BACK" || { echo "FAIL: adapter_version not in write_envelope"; return 1; }
grep -q 'tool_version' "$REPORT_BACK" || { echo "FAIL: tool_version not in write_envelope"; return 1; }

# Check enrich_findings function
grep -q 'enrich_findings()' "$REPORT_BACK" || { echo "FAIL: enrich_findings function not found"; return 1; }

# Check semantic_fingerprint function
grep -q 'semantic_fingerprint()' "$REPORT_BACK" || { echo "FAIL: semantic_fingerprint function not found"; return 1; }

# Check evidence_class_of function
grep -q 'evidence_class_of()' "$REPORT_BACK" || { echo "FAIL: evidence_class_of function not found"; return 1; }
# Check evidence_class and semantic_fingerprint are computed inline in write_envelope
grep -q 'def evidence_class:' "$REPORT_BACK" || { echo "FAIL: evidence_class not in write_envelope"; return 1; }
grep -q 'def semantic_fingerprint:' "$REPORT_BACK" || { echo "FAIL: semantic_fingerprint not in write_envelope"; return 1; }
grep -q 'evidence_class: evidence_class' "$REPORT_BACK" || { echo "FAIL: evidence_class not applied to findings"; return 1; }
grep -q 'semantic_fingerprint: semantic_fingerprint' "$REPORT_BACK" || { echo "FAIL: semantic_fingerprint not applied to findings"; return 1; }

echo "OK: envelope provenance fields present"
return 0
Expand Down
Loading