From 0ae086c99782fd629bf9831bb88e2d73aae04cc1 Mon Sep 17 00:00:00 2001 From: JonJagger Date: Tue, 25 Aug 2026 14:23:59 +0100 Subject: [PATCH] Update empty-flag-audit docs --- .gitignore | 4 + hack/empty-flag-audit/README.md | 58 ++ .../docs/2026-08-13-empty-value-decision.md | 45 +- ...13-upsert-overwrites-unmentioned-fields.md | 40 + ...-08-15-auditing-empty-values-at-the-api.md | 68 +- hack/empty-flag-audit/replay.py | 299 ++++++- hack/empty-flag-audit/results-api.tsv | 836 +++++++++--------- hack/empty-flag-audit/spec.json | 29 +- 8 files changed, 927 insertions(+), 452 deletions(-) diff --git a/.gitignore b/.gitignore index 37892df08..212588752 100644 --- a/.gitignore +++ b/.gitignore @@ -25,3 +25,7 @@ junit-test-results/. .claude/settings.local.json client_reference/ __pycache__/ +# The empty-flag audit's evidence sidecar: whole request and response bodies for +# every replay, so it is large and changes on every run. results-api.tsv is the +# artefact worth committing and diffing. +hack/empty-flag-audit/results-api-evidence.jsonl diff --git a/hack/empty-flag-audit/README.md b/hack/empty-flag-audit/README.md index e62e40e70..9a5b91bc2 100644 --- a/hack/empty-flag-audit/README.md +++ b/hack/empty-flag-audit/README.md @@ -25,6 +25,58 @@ sending it again with one field emptied. It is a separate script rather than a third mode because it measures something else: `--ci` produces rows that line up with the plain run's, and these do not. +**`results-api.tsv` is an answer about the server it ran against, not a permanent +fact.** The checked-in file was taken on 2026-08-19 against a server built from +kosli-dev/server master at b9e213d9, which is the first to refuse an empty +`filename` and an empty `template` attestation name. `make test_setup` otherwise +pulls a published image, so a row can change without anything in this directory +changing. Re-run before quoting it, and say which server the run was against. + +Running it against a locally built server takes two extra steps, because +`docker-compose.yml` pins `platform: linux/amd64`: build the server image for +amd64 (`docker image build --platform linux/amd64 --target test ...`, since an +arm64 image makes compose try to pull an amd64 one that does not exist), then pass +it in with `KOSLI_SERVER_IMAGE= ./bin/reset-or-start-server.sh` rather than +`make test_setup`. It runs emulated, so expect the run to take minutes rather than +seconds. + +`replay.py` asks two questions of each field, and the second one is only worth +asking when the first says the server accepts the value: + +| Column | Question | +|---|---| +| `verdict` | does the server refuse an emptied field? | +| `stored` | when it accepts one, does the empty value reach the record? | + +The `stored` answer is `reaches the record` or `does not reach the record`, or a +sentence saying why the row is neither. It is measured by replaying the emptied +payload **without** freshening the name, so the request names the resource the +`set` run created, and by reading that resource back through the spec's own +`verify` steps before and after. A fresh name every time leaves no record holding +a value for the empty one to reach, which is why the freshening that keeps the +`verdict` question honest would destroy this one. + +The wording is deliberately about the record rather than about a stored value +being replaced, because the two are not the same event. `create flow` writes its +description in place; the `attest` and `report` commands append a document and a +read answers with the most recent one, destroying nothing. Both show up here as +the empty value reaching the record. What the column is for is deciding whether +the endpoint should accept an empty value for that field at all, and that +question does not depend on which of the two mechanisms is underneath. + +Each run also writes `results-api-evidence.jsonl` beside the results, which is +gitignored. Its first line names what the run was against: UTC time, the server +container's image id and start time, the CLI binary's sha256, the audit's git +commit and which of its files were uncommitted. Every line after that is one +replayed request - the label naming its row and role, the method, the url, the +payload as sent, the status, and the whole response body. + +The results file keeps only the first 120 characters of an answer, so that a +column holding a whole listing cannot stop the file being diffed. The sidecar is +where an answer that surprises you is read in full, and where "was it even the +same server?" is answered without a rebuild. Both questions cost a day the first +time they were asked without it. + Narrower runs, for working on one entry: ```bash @@ -127,6 +179,12 @@ commands, hidden commands, and flags hidden by deprecation. wherever they appear, so they are audited once, on `archive flow`, rather than on every command. +**It repeats.** Two `replay.py` runs against one server image, a day apart, gave +identical verdicts for 416 of 417 rows. That is what makes a row that does change +worth investigating rather than shrugging at - and one row did, `list flows +--name`, whose cause is still unknown. Comparing runs is only meaningful because +the rest of the file holds still. + ## Known gaps **The audit invents the values it gives flags.** A flag it knows nothing about diff --git a/hack/empty-flag-audit/docs/2026-08-13-empty-value-decision.md b/hack/empty-flag-audit/docs/2026-08-13-empty-value-decision.md index 444da30ff..3f39e5e15 100644 --- a/hack/empty-flag-audit/docs/2026-08-13-empty-value-decision.md +++ b/hack/empty-flag-audit/docs/2026-08-13-empty-value-decision.md @@ -16,11 +16,14 @@ An audit easily found nineteen cases so far: 1. the server accepts an empty value it should refuse. **Proposal** - _not_ covered in this document: - - add schema `minLength: 1` on `filename`, `template`, `remove_tags` + - refuse an empty `filename` and an empty `template` attestation name + - `remove_tags` turned out not to need it, see below - decide empty query params - leave the 6 description fields alone - measured in `hack/empty-flag-audit/docs/2026-08-15-auditing-empty-values-at-the-api.md` + - what was done, and the trap in the original wording, in "Refusing an empty + value on the server" below 2. the CLI passes an empty value to the server where it should send nothing. **Proposal** - _not_ covered in this document: @@ -200,6 +203,46 @@ It matters most on the flags carrying a verdict - `--compliant`, bare form on those flags, requiring `--compliant=true`, which is a separate decision. +## Refusing an empty value on the server (done 2026-08-19) + +Problem 1's proposal above originally read "add schema `minLength: 1` on +`filename`, `template`, `remove_tags`". Two of the three landed; the wording of +the third was a trap worth recording, because it would have caused an outage. + +**The trap: a model that validates a write may also validate a read.** A Pydantic +constraint is not a rule about incoming requests. It is a rule about every +construction of that model, and in this codebase several models are built both +from a request and from a stored document. Adding a constraint there does not +reject bad input; it rejects **data already in the database**, turning a bad +record into one that cannot be loaded at all. That is strictly worse: a wrong +requirement becomes a failed read, and one record can break a listing for +everyone in the org, which is the shape of +[server issue 6503](https://github.com/kosli-dev/server/issues/6503). + +So the question to ask of any empty-value constraint is not "is this field +user-supplied?" but "is this class ever constructed from Mongo?". + +| Field | Where the constraint went | Why | +|---|---|---| +| `filename` | `min_length=1` on `CreateArtifact` | Bound only as a FastAPI request body, constructed nowhere else. `ArtifactResponseBase` carries the same field and **is** built from stored artifacts, so it was left permissive on purpose. | +| template attestation name | a check on the write path in `common/flow.py`, not on the model | `Attestation` is built when a stored template is read back, and no shape difference distinguishes a write from reading a legacy record, so there was no model-level place to put it. | +| `remove_tags` | nothing | Measured: an empty entry removes no tag, so it does not reach the record. Refusing it is tidiness, not a fix. | + +Both fixes carry a test that loads the bad value from the stored shape and +asserts it still works, and both were checked by restoring the constraint and +watching that test fail. A constraint of this kind that has not been checked that +way has not been checked. + +Everything above is about newly submitted values. Records already holding an +empty value stay as they are, and reading them keeps working. Cleaning them up +would be a migration, which nobody has asked for. + +Confirmed by measurement rather than by the tests alone: `replay.py` re-run +against a server carrying both fixes answers "the server refuses it" for all five +rows that reach `filename` and `template`, taking the acceptances from 25 to 22. +See "What the re-run changed" in +`2026-08-15-auditing-empty-values-at-the-api.md`. + ## The one capability this removes Two flags, and only two, use an empty value to mean something you cannot say any diff --git a/hack/empty-flag-audit/docs/2026-08-13-upsert-overwrites-unmentioned-fields.md b/hack/empty-flag-audit/docs/2026-08-13-upsert-overwrites-unmentioned-fields.md index 1c747daf9..fd6a531c5 100644 --- a/hack/empty-flag-audit/docs/2026-08-13-upsert-overwrites-unmentioned-fields.md +++ b/hack/empty-flag-audit/docs/2026-08-13-upsert-overwrites-unmentioned-fields.md @@ -53,6 +53,46 @@ too, adopt patch and every one of them is wrong today. description survives, so what the CLI sends does not settle it on its own - the server decides. Only running each one shows which is which. +## The server half, with line numbers (2026-08-19) + +Read in the server repo, which is what lines 25-27 above could not do. The three +`create` commands are three different shapes, and only one of them behaves the +way this document assumed. + +**`create flow`.** `description: str` is required (`models/flows.py:17`), and the +handler passes it straight to `flow.update` with no presence guard +(`common/flow.py:155`). So a re-run empties the stored description. (`visibility` +on the same endpoint is `None`-tolerant, handled at `common/flow.py:147-149` and +`:164`. Recorded as a fact about the code, not as a decision about description.) + +**`create environment`.** Required twice: `description: str` +(`models/environments.py:22`), and a second gate below it raising `BadRequest` +when the key is absent (`model/environments.py:179`). But the update path drops a +falsy description before applying it (`_upsert_data`, +`model/environments.py:266-275`) and writes only `if "description" in data` +(`_apply_update`, `:282`). That is why the description survives, as line 52 above +records. Clearing is done through the PATCH endpoint, whose docstring names the +contrast (`v2/environment.py:87-89`). + +**`create policy`.** `description: str = Field("")` (`models/policies.py:25`), so +an absent description has already become `""` by the time the handler sees it, +and the update writes the field whenever it differs from the stored value +(`model/policies/commands.py:180-181`). `comment` is declared the same way +(`models/policies.py:27`). Adding `omitempty` to the CLI changes nothing here on +its own: the model default has to become `None` as well. + +## Two questions the line numbers separate + +They were one question while the server was unread, and answering only the first +looks like a fix. + +1. **Will the server accept an absent field?** Today, no: `create flow` answers + 422 and `create environment` answers 400. So a clean CLI payload is blocked on + the server whatever is decided about meaning. +2. **What does an absent field mean once accepted?** Under the apply rule in this + document, "use the default", so a re-run still empties a flow's description and + only the wire payload gets tidier. Under patch, "preserve". Undecided. + ## How much is at stake Less than "data loss" would suggest. A trail keeps its earlier value in its diff --git a/hack/empty-flag-audit/docs/2026-08-15-auditing-empty-values-at-the-api.md b/hack/empty-flag-audit/docs/2026-08-15-auditing-empty-values-at-the-api.md index d41e25c50..ae8911407 100644 --- a/hack/empty-flag-audit/docs/2026-08-15-auditing-empty-values-at-the-api.md +++ b/hack/empty-flag-audit/docs/2026-08-15-auditing-empty-values-at-the-api.md @@ -24,6 +24,15 @@ reach. - Both are measured instances of the gap the decision document argues about: refusing empty flag values in the CLI does nothing for a customer calling the API directly, so neither of these closes. +- **Both were closed on the server on 2026-08-19.** The figures in this document + are the measurement that prompted that work, and are left as they were taken; + `results-api.tsv` has since been re-run against the fixed server and now reads + 22 acceptances and 56 refusals. What changed, and why the obvious way to change + it would have broken reads of existing records, is in + `2026-08-13-empty-value-decision.md` under "Refusing an empty value on the + server"; which rows moved is in "What the re-run changed" below. +- `replay.py` now also answers, for an accepted value, whether it reaches the + record - the `stored` column, described in the README. - Every emptied filter is accepted: `tag=`, `search=`, `name=` are answered 200 rather than refused. What that means then differs by endpoint - an empty tag matches nothing, an empty repo name matches everything - and the status does @@ -117,8 +126,8 @@ customer can empty by calling the API however the CLI behaves: `create flow --template` is the one to look at. `--template ""` is refused by the CLI, by the wrapper every flag's value carries (`cmd/kosli/nonEmptyValue.go`, applied by the walk in `cmd/kosli/root.go`). -Sending `template: [""]` straight to the API is accepted, and reading the flow -back shows it stored: +Sending `template: [""]` straight to the API was accepted, and reading the flow +back showed it stored: ``` version: 1 @@ -138,8 +147,8 @@ document's proposal, which says a CLI rule can only reach CLI traffic, with a measured instance behind it instead of a worked example. The four rows reaching `filename` say the same thing about a different field, -and it is the one a customer would feel. The server accepts an artifact whose -`filename` is empty, stores it, and serves it back that way: +and it is the one a customer would feel. The server accepted an artifact whose +`filename` is empty, stored it, and served it back that way: ``` $ kosli get artifact FLOW@aaaa... @@ -319,3 +328,54 @@ It also gives the server-side work a test list. What the server has to enforce is not "refuse an empty value" but "an absent field means what the verb says it means", and a probe that replays every captured request with one field emptied, and again with it removed, is how that gets checked rather than asserted. + +That test list was used. The two rows with a named consequence - `template` and +`filename` - were closed on the server on 2026-08-19, each with an endpoint test +asserting the refusal and a second test asserting that a record already holding +the empty value still loads. The second test is the one this probe could not have +suggested: measuring what a request does says nothing about what a schema change +would do to data already stored, and that is where the obvious fix turned out to +be an outage. See "Refusing an empty value on the server" in +`2026-08-13-empty-value-decision.md`. + +The 25 acceptances also collapse further than the row count suggests. Four rows +reach one `filename` field on one endpoint, so they are four CLI routes to a +single server-side defect rather than four defects. A row count is a count of +command-and-flag pairs, not of things to fix. + +## What the re-run changed (2026-08-19) + +Re-run against a server built from kosli-dev/server master at b9e213d9. Eight of +the 417 rows moved. Five are the fixes: + +| Row | Field | Now | +|---|---|---| +| `attest artifact --artifact-type` | `filename` | refuses | +| `attest artifact --display-name` | `filename` | refuses | +| `report artifact --artifact-type` | `filename` | refuses | +| `report artifact --name` | `filename` | refuses | +| `create flow --template` | `template` | refuses | + +Three moved for reasons other than this work, and the third of them turned out not +to be a move at all: + +- `attest override --commit` went from an unusable control to a real refusal, + because [server#6504](https://github.com/kosli-dev/server/issues/6504) is closed + and the control request now succeeds. +- `create environment --included-environments` went from refusing to accepting, + because [server#6503](https://github.com/kosli-dev/server/issues/6503) is closed + and a logical environment with nothing included is now a record the server can + handle. Its `stored` answer is "does not reach the record". +- `list flows --name` went from refusing to accepting, and this is **not + attributable to the fixes**. Running the whole audit against the pre-fix image + again gives "accepts" for that row, with all 416 other rows identical to the + earlier pre-fix run, so the row differs between runs of the same server rather + than between servers. The 400 it recorded on 2026-08-18, `{"message":"Empty + string is not allowed for flow names."}`, could not be reproduced by curl, by an + isolated probe of that row, or by a full run, and its message can only come from + a create or a rename, neither of which a GET of the list endpoint reaches. Cause + unknown; treat the row as unstable rather than as a finding. + +Every write acceptance that remains is either a description field, which is out of +scope by decision, or a value that does not reach the record. Nothing with a named +consequence is still open. diff --git a/hack/empty-flag-audit/replay.py b/hack/empty-flag-audit/replay.py index 132d3f981..5dbc4ef67 100755 --- a/hack/empty-flag-audit/replay.py +++ b/hack/empty-flag-audit/replay.py @@ -18,23 +18,48 @@ two answers is the server's, and it is an answer about every client rather than about the CLI. +A third replay asks whether an accepted empty value reaches the record. The +resource the `set` run created still holds a real value, so replaying the emptied +payload without freshening it names that same resource, and the spec's own verify +steps say whether reading it back afterwards shows the empty value. + +What the server did underneath is not measured, and it differs by endpoint: some +commands write a field in place, while the attest and report commands append a +document that a later read answers with, destroying nothing. Both appear here as +the empty value reaching the record, which is the question the API is being asked +- whether accepting an empty value for this field is right - rather than a claim +about anything being lost. + +This is the one question freshening destroys: a fresh name every time leaves no +record holding a value for the empty one to reach. + Everything here talks to the local test server. It never points at app.kosli.com. """ import argparse +import datetime +import hashlib import json +import pathlib import re +import subprocess import tempfile import urllib.error import urllib.parse import urllib.request import uuid -from audit import (HOST, SPEC, TOKEN, invocation_for, normalise, prepare, - reset_server, run) +from audit import (GLOBALS, HOST, SERVER_CONTAINER, SPEC, TOKEN, expand, + invocation_for, normalise, prepare, reset_server, run) RESULTS = "results-api.tsv" +# Everything volatile or bulky goes here rather than into RESULTS, whose worth is +# that two runs of it diff line by line. A timestamp or a whole response body in a +# column would end that. +EVIDENCE_FILE = "results-api-evidence.jsonl" +EVIDENCE = [] + # The line --debug prints before a request body, naming what it is sending to # where. requests.go logs the method beside the URL for this. SENT = re.compile(r"payload sent to: (\w+) (\S+)") @@ -54,6 +79,8 @@ # Payload fields that must differ between replays. A replay of a create is a # create: sending the captured payload twice would update the first resource # rather than make a second, and the control would then be measuring an update. +# Asking whether an empty value reaches the record needs the captured name kept, +# so that replay is the one that goes unfreshened. FRESHEN = ["name"] @@ -95,7 +122,10 @@ def captured_read(text, command): if not match: return None url = match.group(1) - return "GET", url, dict(urllib.parse.parse_qsl(urllib.parse.urlsplit(url).query)) + # keep_blank_values, or a parameter sent as blank reads as one that was never + # sent, and the diff against the other run names the wrong parameter. + return "GET", url, dict(urllib.parse.parse_qsl( + urllib.parse.urlsplit(url).query, keep_blank_values=True)) def controlled_parameters(omitted, given): @@ -107,14 +137,82 @@ def controlled_parameters(omitted, given): def with_parameter(url, parameter, value): - """Return the url with one query parameter set to value.""" + """Return the url with one query parameter set to value. + + keep_blank_values, or a parameter the url already carries as blank is dropped + rather than kept, so the replayed request differs from the captured one by more + than the parameter under test and its verdict is about something else. + """ parts = urllib.parse.urlsplit(url) - query = dict(urllib.parse.parse_qsl(parts.query)) + query = dict(urllib.parse.parse_qsl(parts.query, keep_blank_values=True)) query[parameter] = value return urllib.parse.urlunsplit(parts._replace( query=urllib.parse.urlencode(query))) +def server_log(since): + """Return the server's own log from when a request was sent, or why not. + + Fetched only for a 5xx, where the answer body says nothing and the traceback + says everything: a 4xx is the server explaining itself and needs no log. A + second is taken off the start because the request and the log entry are + timestamped by different clocks. + """ + window = (since - datetime.timedelta(seconds=1)).isoformat() + try: + done = subprocess.run( + ["docker", "logs", SERVER_CONTAINER, "--since", window], + capture_output=True, text=True, timeout=30) + except (OSError, subprocess.SubprocessError) as exc: + return f"unavailable: {exc}" + return (done.stdout + done.stderr).strip().splitlines()[-200:] + + +def only_this_parameter_changed(control_url, emptied_url, parameter): + """Return why the emptied url differs from the control beyond one parameter. + + A row's verdict is only about the parameter it names if that is the one thing + the two requests do not share. Without this, a verdict can be recorded against + a request that could not have produced it, and the row reads as a finding. + """ + control, emptied_parts = (urllib.parse.urlsplit(control_url), + urllib.parse.urlsplit(emptied_url)) + if (control.scheme, control.netloc, control.path) != ( + emptied_parts.scheme, emptied_parts.netloc, emptied_parts.path): + return (f"the emptied request went to {emptied_parts.path} rather than" + f" {control.path}") + # keep_blank_values, or a parameter that is already blank looks like one that + # is absent, and dropping it counts as a second change. + before = dict(urllib.parse.parse_qsl(control.query, keep_blank_values=True)) + after = dict(urllib.parse.parse_qsl(emptied_parts.query, keep_blank_values=True)) + changed = {key for key in set(before) | set(after) + if before.get(key) != after.get(key)} + if changed != {parameter}: + return (f"the emptied request changed {sorted(changed)} rather than only" + f" {parameter}") + if after.get(parameter) != "": + return f"the emptied request did not empty {parameter}" + return None + + +def only_this_field_changed(control, emptied_payload, field): + """Return why the emptied payload differs from the control beyond one field. + + FRESHEN's fields are expected to differ, because each replay of a create needs + a name of its own. + """ + if control is None or emptied_payload is None: + return None + changed = {key for key in set(control) | set(emptied_payload) + if control.get(key) != emptied_payload.get(key)} + unexpected = changed - {field} - set(FRESHEN) + if unexpected: + return f"the emptied request also changed {sorted(unexpected)}" + if emptied_payload.get(field) not in ("", [""]): + return f"the emptied request did not empty {field}" + return None + + def controlled_fields(omitted, given): """Return the payload fields that differ between two captured payloads. @@ -171,26 +269,105 @@ def freshened(payload): return copy -def replay(method, url, payload): - """Send one payload to the local server and return (status, first line). +def replay(method, url, payload, label="replay"): + """Send one payload to the local server and return (status, answer). A 4xx arrives as an exception rather than a response, and it is the answer being looked for, so it is read from the exception instead of raised. + + The answer is returned whole, and the request and answer are both appended to + EVIDENCE. The results file keeps only the first 120 characters, because a + field holding a whole listing stops the file being read or diffed, and a + truncated answer is the one thing that cannot be recovered afterwards: a row + whose verdict surprises you is answered by its body, and by the request that + earned it. """ if not url.startswith(HOST): raise SystemExit(f"refusing to send anywhere but {HOST}: {url}") # A read carries nothing. Sending it an empty body rather than no body is a # different request from the one the CLI made. body = None if payload is None else json.dumps(payload).encode() - request = urllib.request.Request( - url, method=method, data=body, - headers={"Content-Type": "application/json; charset=utf-8", - "Authorization": f"Bearer {TOKEN}"}) + headers = {"Content-Type": "application/json; charset=utf-8", + "Authorization": f"Bearer {TOKEN}"} + request = urllib.request.Request(url, method=method, data=body, headers=headers) + started = datetime.datetime.now(datetime.timezone.utc) try: with urllib.request.urlopen(request, timeout=30) as response: - return response.status, response.read().decode()[:120] + status, answer = response.status, response.read().decode() except urllib.error.HTTPError as error: - return error.code, error.read().decode()[:120] + status, answer = error.code, error.read().decode() + item = { + "label": label, + "at": started.isoformat(), + "method": method, + "url": url, + # The headers are here because a request that cannot be seen has to be + # guessed at: this GET carries a Content-Type, which is not obvious from + # the row and took a reading of this function to discover. + "headers": dict(headers, Authorization="Bearer "), + "sent": payload, + "status": status, + "answer": answer, + } + if status >= 500: + item["server_log"] = server_log(started) + EVIDENCE.append(item) + return status, answer + + +def read_back(binary, entry, command, key, captured, home): + """Return what the spec's verify steps read back, or None if it has none. + + The steps the audit uses to see what a command did, run here to see whether + an emptied field reached the store. A command with no verify step cannot + answer the question, which is reported rather than guessed at. + """ + steps = entry.get("verify", []) + if not steps: + return None + return "\n".join( + run(binary, [expand(a, command, key, captured) for a in step] + GLOBALS, + False, home)[2] + for step in steps) + + +def stored_answer(binary, entry, command, method, url, carried, field, owned, + runs, home): + """Say whether an accepted empty value reaches the record. + + Asked of the resource the `set` run already created, by replaying the + emptied payload without freshening it, so the request names that resource + rather than a new one. The comparison is of everything the verify steps + print, before against after, because what a read calls a field is not always + what the payload calls it. + + Says nothing about how the server got there, deliberately. A command that + writes a field in place and one that appends a document a later read answers + with both come out as the empty value reaching the record; which of the two + happened is a property of the endpoint, and reading it off this measurement + would be inventing it. + """ + key, captured = owned + before = read_back(binary, entry, command, key, captured, home) + if before is None: + return "not asked, the command has no verify step" + status, _ = replay(method, url, emptied(carried, field), + label=f"{command} {field} emptied, unfreshened") + if not 200 <= status < 300: + return f"not an answer, the replay was itself refused with {status}" + after = read_back(binary, entry, command, key, captured, home) + # A changed read-back is evidence and outranks the status, which does not + # say which record was reached: reporting an artifact that already exists + # answers 201 and the empty value is what a read returns afterwards. The + # status is consulted only when nothing changed, where a 201 leaves it open + # whether the record was reached and kept its value or a separate record was + # named instead. + if normalise(before, command, runs) != normalise(after, command, runs): + return "reaches the record" + if status == 201: + return ("not an answer, the reply was 201 and nothing changed, so the" + " replay may have named a separate record") + return "does not reach the record" def capture(binary, entry, command, flag, how, home): @@ -216,8 +393,9 @@ def probe(binary, entry, command, flag, home): omitted = capture(binary, entry, command, flag, "omitted", home) given = capture(binary, entry, command, flag, "set", home) if not given: - return [f"{command}\t--{flag}\t\t\t\t\t\tnothing to read\tthe run with" - f" the flag set sent no request"] + return [f"{command}\t--{flag}\t\t\t\t\t\tnothing to read" + f"\tnot asked, nothing was replayed" + f"\tthe run with the flag set sent no request"] method, url, carried, owned = given runs = [owned] + ([omitted[3]] if omitted else []) @@ -229,15 +407,32 @@ def probe(binary, entry, command, flag, home): without_fixtures(omitted[2], command, runs) if omitted else None, without_fixtures(carried, command, runs)) if not fields: - return [f"{command}\t--{flag}\t{method}\t{url}\t\t\t\tno field\tthe flag" - f" changes no field of the payload"] + return [f"{command}\t--{flag}\t{method}\t{url}\t\t\t\tno field" + f"\tnot asked, nothing was replayed" + f"\tthe flag changes no field of the payload"] rows = [] - for field in fields: - control = replay(method, url, freshened(carried)) - answer = replay(method, url, emptied(freshened(carried), field)) + for index, field in enumerate(fields): + control_payload = freshened(carried) + emptied_payload = emptied(freshened(carried), field) + control = replay(method, url, control_payload, + label=f"{command} --{flag} {field} control") + answer = replay(method, url, emptied_payload, + label=f"{command} --{flag} {field} emptied") + suspect = only_this_field_changed(control_payload, emptied_payload, field) + outcome = f"suspect, {suspect}" if suspect else verdict(control[0], answer[0]) + if suspect: + stored = "not asked, the emptied request was not the one intended" + elif outcome != "the server accepts it": + stored = f"not asked, {outcome}" + elif index: + stored = ("not asked, an earlier field of this flag already changed" + " the resource") + else: + stored = stored_answer(binary, entry, command, method, url, carried, + field, owned, runs, home) rows.append(f"{command}\t--{flag}\t{method}\t{url}\t{field}" - f"\t{control[0]}\t{answer[0]}\t{verdict(control[0], answer[0])}" + f"\t{control[0]}\t{answer[0]}\t{outcome}\t{stored}" f"\t{answer[1].strip()[:120]}") return rows @@ -252,14 +447,23 @@ def read_rows(command, flag, method, url, query, omitted_query): """ parameters = controlled_parameters(omitted_query, query) if not parameters: - return [f"{command}\t--{flag}\t{method}\t{url}\t\t\t\tno field\tthe flag" - f" changes no query parameter"] + return [f"{command}\t--{flag}\t{method}\t{url}\t\t\t\tno field" + f"\tnot asked, nothing was replayed" + f"\tthe flag changes no query parameter"] rows = [] for parameter in parameters: - control = replay(method, url, None) - answer = replay(method, with_parameter(url, parameter, ""), None) + emptied_url = with_parameter(url, parameter, "") + control = replay(method, url, None, + label=f"{command} --{flag} {parameter} control") + answer = replay(method, emptied_url, None, + label=f"{command} --{flag} {parameter} emptied") + suspect = only_this_parameter_changed(url, emptied_url, parameter) + outcome = f"suspect, {suspect}" if suspect else verdict(control[0], answer[0]) + stored = ("not asked, the emptied request was not the one intended" + if suspect else "not asked, a read stores nothing") rows.append(f"{command}\t--{flag}\t{method}\t{url}\t{parameter}" - f"\t{control[0]}\t{answer[0]}\t{verdict(control[0], answer[0])}" + f"\t{control[0]}\t{answer[0]}\t{outcome}" + f"\t{stored}" f"\t{answer[1].strip()[:120]}") return rows @@ -280,6 +484,41 @@ def verdict(control, emptied_status): return "the server refuses it" +def provenance(binary): + """Name what this run was against, so two runs can be told apart. + + A row that answers differently between runs raises one question first: was it + the same server? Answering it from memory costs a rebuild and a re-run, so it + is recorded here instead. The CLI is identified by the hash of the binary + rather than by `kosli version`, which reports what was built, not which build + is on disk. + """ + def said(argv): + """Return a command's whole output, or why there is none.""" + try: + done = subprocess.run(argv, capture_output=True, text=True, timeout=30) + except (OSError, subprocess.SubprocessError) as exc: + return f"unavailable: {exc}" + return (done.stdout or done.stderr).strip() + + binary_path = pathlib.Path(binary) + changed = said(["git", "status", "--porcelain", "."]) + return { + "label": "provenance", + "at": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "host": HOST, + "server_image": said( + ["docker", "inspect", SERVER_CONTAINER, "--format", "{{.Image}}"]), + "server_started": said( + ["docker", "inspect", SERVER_CONTAINER, "--format", "{{.State.StartedAt}}"]), + "cli_binary": str(binary_path), + "cli_sha256": hashlib.sha256(binary_path.read_bytes()).hexdigest() + if binary_path.is_file() else "unavailable: not a file", + "audit_commit": said(["git", "rev-parse", "HEAD"]), + "audit_uncommitted": changed.splitlines(), + } + + def main(): """Replay every named combination and write what the server answered.""" parser = argparse.ArgumentParser(description=__doc__) @@ -290,9 +529,11 @@ def main(): spec = json.loads(SPEC.read_text()) reset_server() + EVIDENCE.append(provenance(args.binary)) home = tempfile.mkdtemp(prefix="kosli-replay-home-") - rows = ["command\tflag\tmethod\turl\tfield\tcontrol\temptied\tverdict\tanswer"] + rows = ["command\tflag\tmethod\turl\tfield\tcontrol\temptied\tverdict" + "\tstored\tanswer"] for command, entry in sorted(spec.items()): if args.only and args.only not in command: continue @@ -311,7 +552,11 @@ def main(): out = SPEC.parent / RESULTS out.write_text("\n".join(rows) + "\n") + evidence = SPEC.parent / EVIDENCE_FILE + evidence.write_text( + "".join(json.dumps(item) + "\n" for item in EVIDENCE)) print(f"\nwrote {out}") + print(f"wrote {evidence}") if __name__ == "__main__": diff --git a/hack/empty-flag-audit/results-api.tsv b/hack/empty-flag-audit/results-api.tsv index c15c0f294..c968206e7 100644 --- a/hack/empty-flag-audit/results-api.tsv +++ b/hack/empty-flag-audit/results-api.tsv @@ -1,418 +1,418 @@ -command flag method url field control emptied verdict answer -allow artifact --artifact-type PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-artifact-type-set-en no field the flag changes no field of the payload -allow artifact --dry-run nothing to read the run with the flag set sent no request -allow artifact --environment PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-environment-set-en no field the flag changes no field of the payload -allow artifact --exclude PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-exclude-set-en no field the flag changes no field of the payload -allow artifact --fingerprint PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-fingerprint-set-en no field the flag changes no field of the payload -allow artifact --reason PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-reason-set-en no field the flag changes no field of the payload -allow artifact --registry-password nothing to read the run with the flag set sent no request -allow artifact --registry-provider PUT http://localhost:8001/api/v2/allowlists/docs-cmd-test-user-shared/probe-allow-artifact-registry-provider-set-en no field the flag changes no field of the payload -allow artifact --registry-username nothing to read the run with the flag set sent no request -archive attestation-type --dry-run nothing to read the run with the flag set sent no request -archive control --dry-run nothing to read the run with the flag set sent no request -archive environment --dry-run nothing to read the run with the flag set sent no request -archive flow --api-token nothing to read the run with the flag set sent no request -archive flow --config-file nothing to read the run with the flag set sent no request -archive flow --debug nothing to read the run with the flag set sent no request -archive flow --dry-run nothing to read the run with the flag set sent no request -archive flow --host nothing to read the run with the flag set sent no request -archive flow --http-proxy nothing to read the run with the flag set sent no request -archive flow --max-api-retries nothing to read the run with the flag set sent no request -archive flow --org nothing to read the run with the flag set sent no request -archive flow --quiet nothing to read the run with the flag set sent no request -assert artifact --artifact-type nothing to read the run with the flag set sent no request -assert artifact --dry-run nothing to read the run with the flag set sent no request -assert artifact --environment nothing to read the run with the flag set sent no request -assert artifact --exclude nothing to read the run with the flag set sent no request -assert artifact --fingerprint nothing to read the run with the flag set sent no request -assert artifact --flow nothing to read the run with the flag set sent no request -assert artifact --output nothing to read the run with the flag set sent no request -assert artifact --policy nothing to read the run with the flag set sent no request -assert artifact --registry-password nothing to read the run with the flag set sent no request -assert artifact --registry-provider nothing to read the run with the flag set sent no request -assert artifact --registry-username nothing to read the run with the flag set sent no request -assert snapshot --dry-run nothing to read the run with the flag set sent no request -attach-policy --dry-run nothing to read the run with the flag set sent no request -attach-policy --environment POST http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-attach-policy-environment-set-en/policies no field the flag changes no field of the payload -attest artifact --annotate POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-annotate-set-fl annotations 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"annotations":"Input should be a valid dictionary [input: ]"}} -attest artifact --artifact-type POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-artifact-type-set-fl filename 201 201 the server accepts it {"message":"OK"} -attest artifact --build-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-build-url-set-fl no field the flag changes no field of the payload -attest artifact --commit POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-commit-set-fl no field the flag changes no field of the payload -attest artifact --commit-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-commit-url-set-fl no field the flag changes no field of the payload -attest artifact --display-name POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-display-name-set-fl filename 201 201 the server accepts it {"message":"OK"} -attest artifact --dry-run nothing to read the run with the flag set sent no request -attest artifact --exclude POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-exclude-set-fl no field the flag changes no field of the payload -attest artifact --external-fingerprint POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-external-fingerprint-set-fl external_urls 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"external_urls":"Input should be a valid dictionary [input: ]"}} -attest artifact --external-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-external-url-set-fl external_urls 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"external_urls":"Input should be a valid dictionary [input: ]"}} -attest artifact --fingerprint POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-fingerprint-set-fl no field the flag changes no field of the payload -attest artifact --flow POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-flow-set-fl no field the flag changes no field of the payload -attest artifact --name POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-name-set-fl no field the flag changes no field of the payload -attest artifact --redact-commit-info POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-redact-commit-info-set-fl git_commit_info 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"git_commit_info":"Input should be a valid dictionary or object t -attest artifact --registry-password nothing to read the run with the flag set sent no request -attest artifact --registry-provider POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-registry-provider-set-fl no field the flag changes no field of the payload -attest artifact --registry-username nothing to read the run with the flag set sent no request -attest artifact --repo-id POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-repo-id-set-fl no field the flag changes no field of the payload -attest artifact --repo-provider POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-repo-provider-set-fl no field the flag changes no field of the payload -attest artifact --repo-root POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-repo-root-set-fl no field the flag changes no field of the payload -attest artifact --repo-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-repo-url-set-fl no field the flag changes no field of the payload -attest artifact --repository POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-repository-set-fl no field the flag changes no field of the payload -attest artifact --trail POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-attest-artifact-trail-set-fl no field the flag changes no field of the payload -attest custom --annotate POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-annotate-set-fl/trail/probe-attest-custom-annotate-set-tr/custom annotations 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --artifact-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-artifact-type-set-fl/trail/probe-attest-custom-artifact-type-set-tr/custom no field the flag changes no field of the payload -attest custom --attachments POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-attachments-set-fl/trail/probe-attest-custom-attachments-set-tr/custom no field the flag changes no field of the payload -attest custom --attestation-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-attestation-data-set-fl/trail/probe-attest-custom-attestation-data-set-tr/custom no field the flag changes no field of the payload -attest custom --commit POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-commit-set-fl/trail/probe-attest-custom-commit-set-tr/custom git_commit_info 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --description POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-description-set-fl/trail/probe-attest-custom-description-set-tr/custom description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --dry-run nothing to read the run with the flag set sent no request -attest custom --exclude POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-exclude-set-fl/trail/probe-attest-custom-exclude-set-tr/custom no field the flag changes no field of the payload -attest custom --external-fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-external-fingerprint-set-fl/trail/probe-attest-custom-external-fingerprint-set-tr/custom external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --external-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-external-url-set-fl/trail/probe-attest-custom-external-url-set-tr/custom external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-fingerprint-set-fl/trail/probe-attest-custom-fingerprint-set-tr/custom artifact_fingerprint 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --flow POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-flow-set-fl/trail/probe-attest-custom-flow-set-tr/custom no field the flag changes no field of the payload -attest custom --name POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-name-set-fl/trail/probe-attest-custom-name-set-tr/custom no field the flag changes no field of the payload -attest custom --origin-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-origin-url-set-fl/trail/probe-attest-custom-origin-url-set-tr/custom origin_url 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest custom --redact-commit-info POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-redact-commit-info-set-fl/trail/probe-attest-custom-redact-commit-info-set-tr/custom no field the flag changes no field of the payload -attest custom --registry-password nothing to read the run with the flag set sent no request -attest custom --registry-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-registry-provider-set-fl/trail/probe-attest-custom-registry-provider-set-tr/custom no field the flag changes no field of the payload -attest custom --registry-username nothing to read the run with the flag set sent no request -attest custom --repo-id POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-repo-id-set-fl/trail/probe-attest-custom-repo-id-set-tr/custom no field the flag changes no field of the payload -attest custom --repo-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-repo-provider-set-fl/trail/probe-attest-custom-repo-provider-set-tr/custom no field the flag changes no field of the payload -attest custom --repo-root POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-repo-root-set-fl/trail/probe-attest-custom-repo-root-set-tr/custom no field the flag changes no field of the payload -attest custom --repo-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-repo-url-set-fl/trail/probe-attest-custom-repo-url-set-tr/custom no field the flag changes no field of the payload -attest custom --repository POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-repository-set-fl/trail/probe-attest-custom-repository-set-tr/custom no field the flag changes no field of the payload -attest custom --trail POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-trail-set-fl/trail/probe-attest-custom-trail-set-tr/custom no field the flag changes no field of the payload -attest custom --type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-type-set-fl/trail/probe-attest-custom-type-set-tr/custom no field the flag changes no field of the payload -attest custom --user-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-custom-user-data-set-fl/trail/probe-attest-custom-user-data-set-tr/custom user_data 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --annotate POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-annotate-set-fl/trail/probe-attest-decision-annotate-set-tr/system annotations 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --artifact-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-artifact-type-set-fl/trail/probe-attest-decision-artifact-type-set-tr/system no field the flag changes no field of the payload -attest decision --attachments POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-attachments-set-fl/trail/probe-attest-decision-attachments-set-tr/system no field the flag changes no field of the payload -attest decision --commit POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-commit-set-fl/trail/probe-attest-decision-commit-set-tr/system git_commit_info 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --compliant POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-compliant-set-fl/trail/probe-attest-decision-compliant-set-tr/system no field the flag changes no field of the payload -attest decision --control POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-control-set-fl/trail/probe-attest-decision-control-set-tr/system no field the flag changes no field of the payload -attest decision --description POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-description-set-fl/trail/probe-attest-decision-description-set-tr/system description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --dry-run nothing to read the run with the flag set sent no request -attest decision --exclude POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-exclude-set-fl/trail/probe-attest-decision-exclude-set-tr/system no field the flag changes no field of the payload -attest decision --external-fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-external-fingerprint-set-fl/trail/probe-attest-decision-external-fingerprint-set-tr/system external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --external-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-external-url-set-fl/trail/probe-attest-decision-external-url-set-tr/system external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-fingerprint-set-fl/trail/probe-attest-decision-fingerprint-set-tr/system artifact_fingerprint 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --flow POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-flow-set-fl/trail/probe-attest-decision-flow-set-tr/system no field the flag changes no field of the payload -attest decision --name POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-name-set-fl/trail/probe-attest-decision-name-set-tr/system no field the flag changes no field of the payload -attest decision --origin-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-origin-url-set-fl/trail/probe-attest-decision-origin-url-set-tr/system origin_url 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest decision --redact-commit-info POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-redact-commit-info-set-fl/trail/probe-attest-decision-redact-commit-info-set-tr/system no field the flag changes no field of the payload -attest decision --registry-password nothing to read the run with the flag set sent no request -attest decision --registry-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-registry-provider-set-fl/trail/probe-attest-decision-registry-provider-set-tr/system no field the flag changes no field of the payload -attest decision --registry-username nothing to read the run with the flag set sent no request -attest decision --repo-id POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-repo-id-set-fl/trail/probe-attest-decision-repo-id-set-tr/system no field the flag changes no field of the payload -attest decision --repo-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-repo-provider-set-fl/trail/probe-attest-decision-repo-provider-set-tr/system no field the flag changes no field of the payload -attest decision --repo-root POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-repo-root-set-fl/trail/probe-attest-decision-repo-root-set-tr/system no field the flag changes no field of the payload -attest decision --repo-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-repo-url-set-fl/trail/probe-attest-decision-repo-url-set-tr/system no field the flag changes no field of the payload -attest decision --repository POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-repository-set-fl/trail/probe-attest-decision-repository-set-tr/system no field the flag changes no field of the payload -attest decision --trail POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-trail-set-fl/trail/probe-attest-decision-trail-set-tr/system no field the flag changes no field of the payload -attest decision --user-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-decision-user-data-set-fl/trail/probe-attest-decision-user-data-set-tr/system user_data 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --annotate POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-annotate-set-fl/trail/probe-attest-generic-annotate-set-tr/generic annotations 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --artifact-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-artifact-type-set-fl/trail/probe-attest-generic-artifact-type-set-tr/generic no field the flag changes no field of the payload -attest generic --attachments POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-attachments-set-fl/trail/probe-attest-generic-attachments-set-tr/generic no field the flag changes no field of the payload -attest generic --commit POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-commit-set-fl/trail/probe-attest-generic-commit-set-tr/generic git_commit_info 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --compliant POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-compliant-set-fl/trail/probe-attest-generic-compliant-set-tr/generic no field the flag changes no field of the payload -attest generic --description POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-description-set-fl/trail/probe-attest-generic-description-set-tr/generic description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --dry-run nothing to read the run with the flag set sent no request -attest generic --exclude POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-exclude-set-fl/trail/probe-attest-generic-exclude-set-tr/generic no field the flag changes no field of the payload -attest generic --external-fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-external-fingerprint-set-fl/trail/probe-attest-generic-external-fingerprint-set-tr/generic external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --external-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-external-url-set-fl/trail/probe-attest-generic-external-url-set-tr/generic external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-fingerprint-set-fl/trail/probe-attest-generic-fingerprint-set-tr/generic artifact_fingerprint 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --flow POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-flow-set-fl/trail/probe-attest-generic-flow-set-tr/generic no field the flag changes no field of the payload -attest generic --name POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-name-set-fl/trail/probe-attest-generic-name-set-tr/generic no field the flag changes no field of the payload -attest generic --origin-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-origin-url-set-fl/trail/probe-attest-generic-origin-url-set-tr/generic origin_url 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest generic --redact-commit-info POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-redact-commit-info-set-fl/trail/probe-attest-generic-redact-commit-info-set-tr/generic no field the flag changes no field of the payload -attest generic --registry-password nothing to read the run with the flag set sent no request -attest generic --registry-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-registry-provider-set-fl/trail/probe-attest-generic-registry-provider-set-tr/generic no field the flag changes no field of the payload -attest generic --registry-username nothing to read the run with the flag set sent no request -attest generic --repo-id POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-repo-id-set-fl/trail/probe-attest-generic-repo-id-set-tr/generic no field the flag changes no field of the payload -attest generic --repo-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-repo-provider-set-fl/trail/probe-attest-generic-repo-provider-set-tr/generic no field the flag changes no field of the payload -attest generic --repo-root POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-repo-root-set-fl/trail/probe-attest-generic-repo-root-set-tr/generic no field the flag changes no field of the payload -attest generic --repo-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-repo-url-set-fl/trail/probe-attest-generic-repo-url-set-tr/generic no field the flag changes no field of the payload -attest generic --repository POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-repository-set-fl/trail/probe-attest-generic-repository-set-tr/generic no field the flag changes no field of the payload -attest generic --trail POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-trail-set-fl/trail/probe-attest-generic-trail-set-tr/generic no field the flag changes no field of the payload -attest generic --user-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-generic-user-data-set-fl/trail/probe-attest-generic-user-data-set-tr/generic user_data 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --annotate POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-annotate-set-fl/trail/probe-attest-junit-annotate-set-tr/junit annotations 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --artifact-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-artifact-type-set-fl/trail/probe-attest-junit-artifact-type-set-tr/junit no field the flag changes no field of the payload -attest junit --attachments POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-attachments-set-fl/trail/probe-attest-junit-attachments-set-tr/junit no field the flag changes no field of the payload -attest junit --commit POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-commit-set-fl/trail/probe-attest-junit-commit-set-tr/junit git_commit_info 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --description POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-description-set-fl/trail/probe-attest-junit-description-set-tr/junit description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --dry-run nothing to read the run with the flag set sent no request -attest junit --exclude POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-exclude-set-fl/trail/probe-attest-junit-exclude-set-tr/junit no field the flag changes no field of the payload -attest junit --external-fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-external-fingerprint-set-fl/trail/probe-attest-junit-external-fingerprint-set-tr/junit external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --external-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-external-url-set-fl/trail/probe-attest-junit-external-url-set-tr/junit external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-fingerprint-set-fl/trail/probe-attest-junit-fingerprint-set-tr/junit artifact_fingerprint 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --flow POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-flow-set-fl/trail/probe-attest-junit-flow-set-tr/junit no field the flag changes no field of the payload -attest junit --name POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-name-set-fl/trail/probe-attest-junit-name-set-tr/junit no field the flag changes no field of the payload -attest junit --origin-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-origin-url-set-fl/trail/probe-attest-junit-origin-url-set-tr/junit origin_url 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest junit --redact-commit-info POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-redact-commit-info-set-fl/trail/probe-attest-junit-redact-commit-info-set-tr/junit no field the flag changes no field of the payload -attest junit --registry-password nothing to read the run with the flag set sent no request -attest junit --registry-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-registry-provider-set-fl/trail/probe-attest-junit-registry-provider-set-tr/junit no field the flag changes no field of the payload -attest junit --registry-username nothing to read the run with the flag set sent no request -attest junit --repo-id POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-repo-id-set-fl/trail/probe-attest-junit-repo-id-set-tr/junit no field the flag changes no field of the payload -attest junit --repo-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-repo-provider-set-fl/trail/probe-attest-junit-repo-provider-set-tr/junit no field the flag changes no field of the payload -attest junit --repo-root POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-repo-root-set-fl/trail/probe-attest-junit-repo-root-set-tr/junit no field the flag changes no field of the payload -attest junit --repo-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-repo-url-set-fl/trail/probe-attest-junit-repo-url-set-tr/junit no field the flag changes no field of the payload -attest junit --repository POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-repository-set-fl/trail/probe-attest-junit-repository-set-tr/junit no field the flag changes no field of the payload -attest junit --results-dir POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-results-dir-set-fl/trail/probe-attest-junit-results-dir-set-tr/junit no field the flag changes no field of the payload -attest junit --trail POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-trail-set-fl/trail/probe-attest-junit-trail-set-tr/junit no field the flag changes no field of the payload -attest junit --upload-results POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-upload-results-set-fl/trail/probe-attest-junit-upload-results-set-tr/junit no field the flag changes no field of the payload -attest junit --user-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-junit-user-data-set-fl/trail/probe-attest-junit-user-data-set-tr/junit user_data 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -attest override --annotate POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-annotate-set-fl/trail/probe-attest-override-annotate-set-tr/override annotations 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"annotations":"Input should be a valid dictionary [input: ]"}} -attest override --artifact-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-artifact-type-set-fl/trail/probe-attest-override-artifact-type-set-tr/override no field the flag changes no field of the payload -attest override --attachments POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-attachments-set-fl/trail/probe-attest-override-attachments-set-tr/override no field the flag changes no field of the payload -attest override --commit POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-commit-set-fl/trail/probe-attest-override-commit-set-tr/override git_commit_info 500 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"git_commit_info":"Input should be a valid dictionary or object t -attest override --description POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-description-set-fl/trail/probe-attest-override-description-set-tr/override description 200 200 the server accepts it "OK" -attest override --dry-run nothing to read the run with the flag set sent no request -attest override --exclude POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-exclude-set-fl/trail/probe-attest-override-exclude-set-tr/override no field the flag changes no field of the payload -attest override --external-fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-external-fingerprint-set-fl/trail/probe-attest-override-external-fingerprint-set-tr/override external_urls 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"external_urls":"Input should be a valid dictionary [input: ]"}} -attest override --external-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-external-url-set-fl/trail/probe-attest-override-external-url-set-tr/override external_urls 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"external_urls":"Input should be a valid dictionary [input: ]"}} -attest override --fingerprint POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-fingerprint-set-fl/trail/probe-attest-override-fingerprint-set-tr/override artifact_fingerprint 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"artifact_fingerprint":"String should match pattern '^[a-f0-9]{64 -attest override --flow POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-flow-set-fl/trail/probe-attest-override-flow-set-tr/override no field the flag changes no field of the payload -attest override --name POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-name-set-fl/trail/probe-attest-override-name-set-tr/override no field the flag changes no field of the payload -attest override --new-compliance-status POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-new-compliance-status-set-fl/trail/probe-attest-override-new-compliance-status-set-tr/override no field the flag changes no field of the payload -attest override --origin-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-origin-url-set-fl/trail/probe-attest-override-origin-url-set-tr/override origin_url 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"origin_url":"Input should be a valid URL, input is empty [input: -attest override --original-attestation-type POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-original-attestation-type-set-fl/trail/probe-attest-override-original-attestation-type-set-tr/override no field the flag changes no field of the payload -attest override --reason POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-reason-set-fl/trail/probe-attest-override-reason-set-tr/override no field the flag changes no field of the payload -attest override --redact-commit-info POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-redact-commit-info-set-fl/trail/probe-attest-override-redact-commit-info-set-tr/override no field the flag changes no field of the payload -attest override --registry-password nothing to read the run with the flag set sent no request -attest override --registry-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-registry-provider-set-fl/trail/probe-attest-override-registry-provider-set-tr/override no field the flag changes no field of the payload -attest override --registry-username nothing to read the run with the flag set sent no request -attest override --repo-id POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-repo-id-set-fl/trail/probe-attest-override-repo-id-set-tr/override no field the flag changes no field of the payload -attest override --repo-provider POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-repo-provider-set-fl/trail/probe-attest-override-repo-provider-set-tr/override no field the flag changes no field of the payload -attest override --repo-root POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-repo-root-set-fl/trail/probe-attest-override-repo-root-set-tr/override no field the flag changes no field of the payload -attest override --repo-url POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-repo-url-set-fl/trail/probe-attest-override-repo-url-set-tr/override no field the flag changes no field of the payload -attest override --repository POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-repository-set-fl/trail/probe-attest-override-repository-set-tr/override no field the flag changes no field of the payload -attest override --trail POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-trail-set-fl/trail/probe-attest-override-trail-set-tr/override no field the flag changes no field of the payload -attest override --user-data POST http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-attest-override-user-data-set-fl/trail/probe-attest-override-user-data-set-tr/override user_data 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"user_data.dict[any,any]":"Input should be a valid dictionary [in -begin trail --commit PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-commit-set-fl git_commit_info 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -begin trail --description PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-description-set-fl description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -begin trail --dry-run nothing to read the run with the flag set sent no request -begin trail --external-fingerprint PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-external-fingerprint-set-fl external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -begin trail --external-url PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-external-url-set-fl external_urls 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -begin trail --flow PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-flow-set-fl no field the flag changes no field of the payload -begin trail --origin-url PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-origin-url-set-fl origin_url 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -begin trail --redact-commit-info PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-redact-commit-info-set-fl no field the flag changes no field of the payload -begin trail --repo-id PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-repo-id-set-fl no field the flag changes no field of the payload -begin trail --repo-provider PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-repo-provider-set-fl no field the flag changes no field of the payload -begin trail --repo-root PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-repo-root-set-fl no field the flag changes no field of the payload -begin trail --repo-url PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-repo-url-set-fl no field the flag changes no field of the payload -begin trail --repository PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-repository-set-fl no field the flag changes no field of the payload -begin trail --template-file PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-template-file-set-fl no field the flag changes no field of the payload -begin trail --user-data PUT http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-begin-trail-user-data-set-fl user_data 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -config --set nothing to read the run with the flag set sent no request -config --unset nothing to read the run with the flag set sent no request -create api-key --description POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-create-api-key-description-set-sa/api-keys no field the flag changes no field of the payload -create api-key --dry-run nothing to read the run with the flag set sent no request -create api-key --expires-at POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-create-api-key-expires-at-set-sa/api-keys expires_at 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"expires_at":"Input should be a valid integer, unable to parse st -create api-key --output POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-create-api-key-output-set-sa/api-keys no field the flag changes no field of the payload -create api-key --service-account POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-create-api-key-service-account-set-sa/api-keys no field the flag changes no field of the payload -create attestation-type --description POST http://localhost:8001/api/v2/custom-attestation-types/docs-cmd-test-user-shared description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -create attestation-type --dry-run nothing to read the run with the flag set sent no request -create attestation-type --jq POST http://localhost:8001/api/v2/custom-attestation-types/docs-cmd-test-user-shared evaluator 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -create attestation-type --schema POST http://localhost:8001/api/v2/custom-attestation-types/docs-cmd-test-user-shared no field the flag changes no field of the payload -create control --description POST http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared description 409 409 unusable, the control failed {"message":"A control with identifier 'probe-create-control-description-set-co' already exists in organization 'docs-cmd -create control --dry-run nothing to read the run with the flag set sent no request -create control --name POST http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared no field the flag changes no field of the payload -create environment --description PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared description 201 201 the server accepts it "OK" -create environment --dry-run nothing to read the run with the flag set sent no request -create environment --exclude-scaling PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared include_scaling 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"include_scaling":"Input should be a valid boolean, unable to int -create environment --include-scaling PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared include_scaling 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"include_scaling":"Input should be a valid boolean, unable to int -create environment --included-environments PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared included_environments 201 404 the server refuses it {"message":"Environment named '' does not exist for organization 'docs-cmd-test-user-shared'"} -create environment --require-provenance PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared require_provenance 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"require_provenance":"Input should be a valid boolean, unable to -create environment --type PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared no field the flag changes no field of the payload -create flow --description PUT http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared description 201 201 the server accepts it "OK" -create flow --dry-run nothing to read the run with the flag set sent no request -create flow --template PUT http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared template 201 201 the server accepts it "OK" -create flow --template-file PUT http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared/template_file template 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -create flow --use-empty-template PUT http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared/template_file template 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"data_json":"Field required [input: None]"}} -create flow --visibility PUT http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared visibility 201 400 the server refuses it {"message":"Input payload validation failed","errors":{"visibility":"Input should be 'public' or 'private' [input: ]"}} -create policy --comment PUT http://localhost:8001/api/v2/policies/docs-cmd-test-user-shared comment 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"payload":"Field required [input: None]"}} -create policy --description PUT http://localhost:8001/api/v2/policies/docs-cmd-test-user-shared description 400 400 unusable, the control failed {"message":"Input payload validation failed","errors":{"payload":"Field required [input: None]"}} -create policy --dry-run nothing to read the run with the flag set sent no request -create policy --type PUT http://localhost:8001/api/v2/policies/docs-cmd-test-user-shared no field the flag changes no field of the payload -create service-account --description POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared description 201 201 the server accepts it {"name":"replay-a80f20d7","display_name":"replay-a80f20d7","privilege":"reader","creating_user_id":"34b8435e-3f56-4892-a -create service-account --dry-run nothing to read the run with the flag set sent no request -create service-account --privilege POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared no field the flag changes no field of the payload -delete api-key --assume-yes nothing to read the run with the flag set sent no request -delete api-key --dry-run nothing to read the run with the flag set sent no request -delete api-key --service-account nothing to read the run with the flag set sent no request -delete api-key --yes nothing to read the run with the flag set sent no request -delete service-account --assume-yes nothing to read the run with the flag set sent no request -delete service-account --dry-run nothing to read the run with the flag set sent no request -delete service-account --yes nothing to read the run with the flag set sent no request -detach-policy --dry-run nothing to read the run with the flag set sent no request -detach-policy --environment DELETE http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-detach-policy-environment-set-en/policies no field the flag changes no field of the payload -diff snapshots --output GET http://localhost:8001/api/v2/env-diff/docs-cmd-test-user-shared?snappish1=probe-diff-snapshots-output-set-en&snappish2=probe-diff-snapshots-output-set-en snappish1 200 400 the server refuses it {"message":"Bad snappish "} -diff snapshots --output GET http://localhost:8001/api/v2/env-diff/docs-cmd-test-user-shared?snappish1=probe-diff-snapshots-output-set-en&snappish2=probe-diff-snapshots-output-set-en snappish2 200 400 the server refuses it {"message":"Bad snappish "} -diff snapshots --show-unchanged GET http://localhost:8001/api/v2/env-diff/docs-cmd-test-user-shared?snappish1=probe-diff-snapshots-show-unchanged-set-en&snappish2=probe-diff-snapshots-show-unchanged-set-en snappish1 200 400 the server refuses it {"message":"Bad snappish "} -diff snapshots --show-unchanged GET http://localhost:8001/api/v2/env-diff/docs-cmd-test-user-shared?snappish1=probe-diff-snapshots-show-unchanged-set-en&snappish2=probe-diff-snapshots-show-unchanged-set-en snappish2 200 400 the server refuses it {"message":"Bad snappish "} -evaluate input --assert nothing to read the run with the flag set sent no request -evaluate input --attestations nothing to read the run with the flag set sent no request -evaluate input --flow nothing to read the run with the flag set sent no request -evaluate input --input-file nothing to read the run with the flag set sent no request -evaluate input --no-assert nothing to read the run with the flag set sent no request -evaluate input --output nothing to read the run with the flag set sent no request -evaluate input --params nothing to read the run with the flag set sent no request -evaluate input --policy nothing to read the run with the flag set sent no request -evaluate input --show-input nothing to read the run with the flag set sent no request -evaluate trail --assert nothing to read the run with the flag set sent no request -evaluate trail --attestations nothing to read the run with the flag set sent no request -evaluate trail --flow nothing to read the run with the flag set sent no request -evaluate trail --no-assert nothing to read the run with the flag set sent no request -evaluate trail --output nothing to read the run with the flag set sent no request -evaluate trail --params nothing to read the run with the flag set sent no request -evaluate trail --policy nothing to read the run with the flag set sent no request -evaluate trail --show-input nothing to read the run with the flag set sent no request -evaluate trails --assert nothing to read the run with the flag set sent no request -evaluate trails --attestations nothing to read the run with the flag set sent no request -evaluate trails --flow nothing to read the run with the flag set sent no request -evaluate trails --no-assert nothing to read the run with the flag set sent no request -evaluate trails --output nothing to read the run with the flag set sent no request -evaluate trails --params nothing to read the run with the flag set sent no request -evaluate trails --policy nothing to read the run with the flag set sent no request -evaluate trails --show-input nothing to read the run with the flag set sent no request -fingerprint --artifact-type nothing to read the run with the flag set sent no request -fingerprint --e nothing to read the run with the flag set sent no request -fingerprint --exclude nothing to read the run with the flag set sent no request -fingerprint --registry-password nothing to read the run with the flag set sent no request -fingerprint --registry-provider nothing to read the run with the flag set sent no request -fingerprint --registry-username nothing to read the run with the flag set sent no request -get api-key --output GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-get-api-key-output-set-sa/api-keys/7954691b-ffdd-4ad6-baf7-5630b372 no field the flag changes no query parameter -get api-key --service-account GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-get-api-key-service-account-set-sa/api-keys/2b1ac497-5b5c-47ab-8e07-b58c5456 no field the flag changes no query parameter -get artifact --output GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-get-artifact-output-set-fl/fingerprint/1bef738d0bb1e690500f99a5b57d958caf3a5eb3e00d9012e1f4369fc6812e01 no field the flag changes no query parameter -get artifact --trail GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-get-artifact-trail-set-fl/fingerprint/1bef738d0bb1e690500f99a5b57d958caf3a5eb3e00d9012e1f4369fc6812e01?trail=probe-get-artifact-trail-set-tr trail 200 200 the server accepts it {"id":"f609b8e2-ca15-4de2-9928-43e8979c","trail_name":"probe-get-artifact-trail-set-tr","template_reference_name":"probe -get attestation --attestation-id nothing to read the run with the flag set sent no request -get attestation --fingerprint GET http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-get-attestation-fingerprint-set-fl/artifact/1bef738d0bb1e690500f99a5b57d958caf3a5eb3e00d9012e1f4369fc6812e01/probe-on-artifact no field the flag changes no query parameter -get attestation --flow GET http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-get-attestation-flow-set-fl/trail/probe-get-attestation-flow-set-tr/probe-attestation no field the flag changes no query parameter -get attestation --output GET http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-get-attestation-output-set-fl/trail/probe-get-attestation-output-set-tr/probe-attestation no field the flag changes no query parameter -get attestation --trail GET http://localhost:8001/api/v2/attestations/docs-cmd-test-user-shared/probe-get-attestation-trail-set-fl/trail/probe-get-attestation-trail-set-tr/probe-attestation no field the flag changes no query parameter -get attestation-type --output GET http://localhost:8001/api/v2/custom-attestation-types/docs-cmd-test-user-shared/probe-get-attestation-type-output-set-nm no field the flag changes no query parameter -get control --output GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared/probe-get-control-output-set-co no field the flag changes no query parameter -get default-org --output GET http://localhost:8001/api/v2/user/default-org no field the flag changes no query parameter -get environment --output GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-get-environment-output-set-en no field the flag changes no query parameter -get flow --output GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared/probe-get-flow-output-set-fl no field the flag changes no query parameter -get policy --output GET http://localhost:8001/api/v2/policies/docs-cmd-test-user-shared/probe-get-policy-output-set-po no field the flag changes no query parameter -get service-account --output GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-get-service-account-output-set-sa no field the flag changes no query parameter -get snapshot --output GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-get-snapshot-output-set-en/-1 no field the flag changes no query parameter -get trail --flow GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-get-trail-flow-set-fl/probe-get-trail-flow-set-tr no field the flag changes no query parameter -get trail --output GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared/probe-get-trail-output-set-fl/probe-get-trail-output-set-tr no field the flag changes no query parameter -join environment --dry-run nothing to read the run with the flag set sent no request -join environment --logical PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-join-environment-logical-set-lg/join no field the flag changes no field of the payload -join environment --physical PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-join-environment-physical-set-lg/join no field the flag changes no field of the payload -list api-keys --output GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-list-api-keys-output-set-sa/api-keys no field the flag changes no query parameter -list api-keys --service-account GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-list-api-keys-service-account-set-sa/api-keys no field the flag changes no query parameter -list artifacts --flow GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared?flow_name=probe-list-artifacts-flow-set-fl&page=1&per_page=15 flow_name 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.flow_name":"String should match pattern '^[a-zA-Z0-9][a-zA -list artifacts --output GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list artifacts --page GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list artifacts --page-limit GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list artifacts --repo GET http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared?page=1&per_page=15&repo_name=probe-repo repo_name 404 200 unusable, the control failed [{"id":"c24d352b-e1e4-4077-abb0-e7020146","trail_name":"probe-get-attestation-trail-set-tr","template_reference_name":"p -list attestation-types --output GET http://localhost:8001/api/v2/custom-attestation-types/docs-cmd-test-user-shared no field the flag changes no query parameter -list controls --archived GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?archived=true&page=1&per_page=15 archived 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.archived":"Input should be a valid boolean, unable to inte -list controls --output GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list controls --page GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list controls --page-limit GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list controls --search GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15&search=probe-search search 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15&search="}, -list controls --sort-direction GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15&sort_direction=asc sort_direction 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.sort_direction":"Input should be 'asc' or 'desc' [input: ] -list controls --tag GET http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15&tag=probe-tag tag 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared?page=1&per_page=15&tag="},"fi -list environments --name GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?name=probe-list-environments-name-set-nm&page=1&per_page=5 name 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?name=&page=1&per_page=5"} -list environments --output GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5 no field the flag changes no query parameter -list environments --page GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5 no field the flag changes no query parameter -list environments --page-limit GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5 page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.page":"Input should be a valid integer, unable to parse st -list environments --page-limit GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list environments --sort GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&sort=name sort 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.sort":"Input should be 'name', 'last_modified_at' or 'last -list environments --sort-direction GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&sort_direction=asc sort_direction 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.sort_direction":"Input should be 'asc' or 'desc' [input: ] -list environments --space-id GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&space_id=probe-space-id space_id 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&space_i -list environments --tag GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&tag=probe-tag tag 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&tag="}, -list environments --type GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared?page=1&per_page=5&type=K8S type 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.type[0]":"Input should be 'K8S', 'ECS', 'S3', 'lambda', 's -list flows --ignore-case GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared no field the flag changes no query parameter -list flows --name GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared?search_by_name=probe-list-flows-name-set-nm search_by_name 200 400 the server refuses it {"message":"Empty string is not allowed for flow names."} -list flows --output GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared no field the flag changes no query parameter -list flows --page GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared?page=1&per_page=20 page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.page":"Input should be a valid integer, unable to parse st -list flows --page GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared?page=1&per_page=20 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list flows --page-limit GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared?page=1&per_page=5 page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.page":"Input should be a valid integer, unable to parse st -list flows --page-limit GET http://localhost:8001/api/v2/flows/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list policies --output GET http://localhost:8001/api/v2/policies/docs-cmd-test-user-shared no field the flag changes no query parameter -list repos --name GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?name=probe-list-repos-name-set-nm&page=1&per_page=15 name 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?name=&page=1&per_page=15"},"firs -list repos --output GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list repos --page GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15 no field the flag changes no query parameter -list repos --page-limit GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list repos --provider GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&provider=github provider 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.provider":"Input should be 'github', 'gitlab', 'bitbucket' -list repos --repo-id GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&repo_id=probe-repo-id repo_id 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&repo_id="},"f -list repos --search GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&search=probe-search search 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&search="},"fi -list repos --sort-direction GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&sort_direction=asc sort_direction 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.sort_direction":"Input should be 'asc' or 'desc' [input: ] -list repos --tag GET http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&tag=probe-tag tag 200 200 the server accepts it {"_links":{"self":{"href":"http://localhost:8001/api/v2/repos/docs-cmd-test-user-shared?page=1&per_page=15&tag="},"first -list service-accounts --output GET http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared no field the flag changes no query parameter -list snapshots --interval GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-list-snapshots-interval-set-en?interval=1&page=1&per_page=15&reverse=false interval 200 200 the server accepts it [{"index":1,"from":1786803891.6454034,"to":0.0,"compliant":false,"duration":0.4360325336456299}] -list snapshots --output GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-list-snapshots-output-set-en?interval=&page=1&per_page=15&reverse=false no field the flag changes no query parameter -list snapshots --page GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-list-snapshots-page-set-en?interval=&page=1&per_page=15&reverse=false no field the flag changes no query parameter -list snapshots --page-limit GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-list-snapshots-page-limit-set-en?interval=&page=1&per_page=5&reverse=false per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -list snapshots --reverse GET http://localhost:8001/api/v2/snapshots/docs-cmd-test-user-shared/probe-list-snapshots-reverse-set-en?interval=&page=1&per_page=15&reverse=true reverse 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.reverse":"Input should be a valid boolean, unable to inter -list trails --fingerprint GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?fingerprint=1bef738d0bb1e690500f99a5b57d958caf3a5eb3e00d9012e1f4369fc6812e01&page=1&per_page=20 fingerprint 200 200 the server accepts it {"data":[],"pagination":{"total":0,"page":1,"per_page":20,"page_count":0}} -list trails --flow GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?flow=probe-list-trails-flow-set-fl&page=1&per_page=20 flow 200 404 the server refuses it {"message":"Flow named '' does not exist for organization 'docs-cmd-test-user-shared'"} -list trails --flow-tag GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?flow_tag=probe%3Dflow-tag&page=1&per_page=20 flow_tag 200 200 the server accepts it {"data":[],"pagination":{"total":0,"page":1,"per_page":20,"page_count":0}} -list trails --output GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?page=1&per_page=20 no field the flag changes no query parameter -list trails --page GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?page=1&per_page=20 no field the flag changes no query parameter -list trails --page-limit GET http://localhost:8001/api/v2/trails/docs-cmd-test-user-shared?page=1&per_page=5 per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -log environment --end GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-end-set-en/events?end=1&interval=&page=1&per_page=15&reverse=false end 200 400 the server refuses it {"message":"Invalid interval value: "} -log environment --end-ts GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-end-ts-set-en/events?end_ts=4102444800&interval=&page=1&per_page=15&reverse=false end_ts 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.end_ts":"Input should be a valid number, unable to parse s -log environment --interval GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-interval-set-en/events?interval=1&page=1&per_page=15&reverse=false interval 200 200 the server accepts it [{"environment_name":"probe-log-environment-interval-set-en","snapshot_index":1,"artifact_name":"probe-artifact","sha256 -log environment --output GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-output-set-en/events?interval=&page=1&per_page=15&reverse=false no field the flag changes no query parameter -log environment --page GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-page-set-en/events?interval=&page=1&per_page=15&reverse=false no field the flag changes no query parameter -log environment --page-limit GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-page-limit-set-en/events?interval=&page=1&per_page=5&reverse=false per_page 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.per_page":"Input should be a valid integer, unable to pars -log environment --repo GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-repo-set-en/events?interval=&page=1&per_page=15&repo_name=probe-repo&reverse=false repo_name 404 404 unusable, the control failed {"message":"Repo '' not found"} -log environment --reverse GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-reverse-set-en/events?interval=&page=1&per_page=15&reverse=true reverse 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.reverse":"Input should be a valid boolean, unable to inter -log environment --start GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-start-set-en/events?interval=&page=1&per_page=15&reverse=false&start=1 start 200 400 the server refuses it {"message":"Invalid interval value: "} -log environment --start-ts GET http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-log-environment-start-ts-set-en/events?interval=&page=1&per_page=15&reverse=false&start_ts=1 start_ts 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"query.start_ts":"Input should be a valid number, unable to parse -rename environment --dry-run nothing to read the run with the flag set sent no request -rename flow --dry-run nothing to read the run with the flag set sent no request -report artifact --artifact-type POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-artifact-type-set-fl filename 201 201 the server accepts it {"message":"OK"} -report artifact --build-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-build-url-set-fl no field the flag changes no field of the payload -report artifact --commit-url POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-commit-url-set-fl no field the flag changes no field of the payload -report artifact --dry-run nothing to read the run with the flag set sent no request -report artifact --exclude POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-exclude-set-fl no field the flag changes no field of the payload -report artifact --fingerprint POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-fingerprint-set-fl no field the flag changes no field of the payload -report artifact --flow POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-flow-set-fl no field the flag changes no field of the payload -report artifact --git-commit POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-git-commit-set-fl no field the flag changes no field of the payload -report artifact --name POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-name-set-fl filename 201 201 the server accepts it {"message":"OK"} -report artifact --registry-password nothing to read the run with the flag set sent no request -report artifact --registry-provider POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-registry-provider-set-fl no field the flag changes no field of the payload -report artifact --registry-username nothing to read the run with the flag set sent no request -report artifact --repo-root POST http://localhost:8001/api/v2/artifacts/docs-cmd-test-user-shared/probe-report-artifact-repo-root-set-fl no field the flag changes no field of the payload -rotate api-key --dry-run nothing to read the run with the flag set sent no request -rotate api-key --expires-at POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-rotate-api-key-expires-at-set-sa/api-keys/7980eb22-1fe1-4172-ae30-97d5ac0b/rotate no field the flag changes no field of the payload -rotate api-key --grace-period-hours POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-rotate-api-key-grace-period-hours-set-sa/api-keys/eeb31830-f024-4618-b5cc-2b1c5de5/rotate no field the flag changes no field of the payload -rotate api-key --output POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-rotate-api-key-output-set-sa/api-keys/2d08a369-ae18-4ba4-b1cd-ffbbab9a/rotate no field the flag changes no field of the payload -rotate api-key --service-account POST http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-rotate-api-key-service-account-set-sa/api-keys/2e40fcd4-5e25-4a8b-b311-0cf5d2ef/rotate no field the flag changes no field of the payload -search --output GET http://localhost:8001/api/v2/search/docs-cmd-test-user-shared/sha/c4bec949ae58e8c07ac28ccb4205be3f42e9c3415c2a02594622bd386d5ce4bf no field the flag changes no query parameter -snapshot docker --dry-run nothing to read the run with the flag set sent no request -snapshot path --dry-run nothing to read the run with the flag set sent no request -snapshot path --exclude PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-path-exclude-set-en/report/server artifacts 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"artifacts[0]":"Input should be a valid dictionary or object to e -snapshot path --name PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-path-name-set-en/report/server no field the flag changes no field of the payload -snapshot path --path PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-path-path-set-en/report/server no field the flag changes no field of the payload -snapshot path --watch PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-path-watch-set-en/report/server artifacts 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"artifacts[0]":"Input should be a valid dictionary or object to e -snapshot paths --dry-run nothing to read the run with the flag set sent no request -snapshot paths --paths-file PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-paths-paths-file-set-en/report/server no field the flag changes no field of the payload -snapshot paths --watch PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-paths-watch-set-en/report/server no field the flag changes no field of the payload -snapshot server --dry-run nothing to read the run with the flag set sent no request -snapshot server --e PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-server-e-set-en/report/server no field the flag changes no field of the payload -snapshot server --exclude PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-server-exclude-set-en/report/server no field the flag changes no field of the payload -snapshot server --paths PUT http://localhost:8001/api/v2/environments/docs-cmd-test-user-shared/probe-snapshot-server-paths-set-en/report/server no field the flag changes no field of the payload -status --assert nothing to read the run with the flag set sent no request -tag --dry-run nothing to read the run with the flag set sent no request -tag --provider nothing to read the run with the flag set sent no request -tag --repo-id nothing to read the run with the flag set sent no request -tag --set PATCH http://localhost:8001/api/v2/tags/docs-cmd-test-user-shared/flow/probe-tag-set-set-fl set_tags 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"set_tags":"Input should be a valid dictionary [input: ]"}} -tag --unset PATCH http://localhost:8001/api/v2/tags/docs-cmd-test-user-shared/flow/probe-tag-unset-set-fl remove_tags 200 200 the server accepts it "OK" -unarchive control --dry-run nothing to read the run with the flag set sent no request -update control --description PUT http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared/probe-update-control-description-set-co description 200 200 the server accepts it {"identifier":"probe-update-control-description-set-co","name":"replay-a5f8b56a","version":4,"created_at":1786803930.998 -update control --dry-run nothing to read the run with the flag set sent no request -update control --link PUT http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared/probe-update-control-link-set-co links 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"links":"Input should be a valid dictionary [input: ]"}} -update control --name PUT http://localhost:8001/api/v2/controls/docs-cmd-test-user-shared/probe-update-control-name-set-co no field the flag changes no field of the payload -update default-org --dry-run nothing to read the run with the flag set sent no request -update service-account --description PATCH http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-update-service-account-description-set-sa no field the flag changes no field of the payload -update service-account --dry-run nothing to read the run with the flag set sent no request -update service-account --privilege PATCH http://localhost:8001/api/v2/service-accounts/docs-cmd-test-user-shared/probe-update-service-account-privilege-set-sa privilege 200 400 the server refuses it {"message":"Input payload validation failed","errors":{"privilege":"Input should be , ,