Repository navigation
Testes em shell do harness entram na cobertura do aceite (LUI-183) - #17
Merged
Merged
Conversation
…183) Acceptance file docs/tickets/LUI-182/LUI-183/aceite.md (CA1-CA4). No route, contract or schema changes: the ticket only changes harness scripts. Decision: the shell tests write their own report, apps/api/reports/harness/shell-tests.json, in the Vitest format the checker and check-report.cjs already read, and the checker receives it as one more vitest:<file> spec. It is not merged into node-tests.json because the Node reporter writes that one inside the api container, while the shell suites run on the machine (git and jq); two files keep each writer alone and need no change in the checker. Decision: the helpers record the cases, and a new runner assembles the report. check, expect_hook and the new record_case append "suite, status, name" to the file in SH_TEST_LOG when it is set, and the runner turns that log into the report with jq at the end of the run. Suites that print their own ok/FALHOU lines (stop-verify and record-touched) call record_case, so every shell case of the harness is in the report, not only the ones that use check. Without SH_TEST_LOG the helpers behave as today. Decision: the tag comes from the case name with the same pattern as the Node reporter (@<ticket>-CA<n>), and the title stays as written, so the checker's rule against an ID only in the title is satisfied the same way. The pattern lives in two places (JS and jq), which is accepted: it is one line, and the test of CA1 would catch a drift. Decision: the runner removes the report before anything runs and always writes a new one at the end, even when a suite fails, so a report from a previous run never covers a criterion whose case did not run now (CA3). It keeps going after a failing suite, which the old && chain did not, so the report lists every suite and the Aceite step does not add a missing-report error to a failure already shown. A suite that exits non-zero without a failed case, or that does not exist, becomes one failed case with the exit code, so a crash cannot pass as a report without failures. Decision: harness_tests runs the shell suites even when the Node tests fail, and then runs check_report on the shell report, so zero cases, a failed case or a skipped one fail the "Testes do harness" step as they do for the Node report. Decision: the tests of this ticket are shell tests too and carry the @LUI-183-CA<n> token, so they are in the new report and the ticket proves itself with the mechanism it builds. They run the checker for real, through scripts/node-in-api.sh, on a fake acceptance file and a report written under apps/api/reports/ (git-ignored), the only writable folder the api container shares with the machine, so the checker reads them under /repo. Files and interfaces the implementation will touch: - scripts/test/sh-helpers.sh: record_case <status> <name> (new) and its use in check and expect_hook; the header documents how to name an acceptance test and where the report is (CA4). - scripts/test/shell-tests.sh (new): shell-tests.sh <report> <suite>... removes the report, runs each suite with SH_TEST_LOG set, writes the report with jq, exits 1 if any suite failed. - scripts/test/stop-verify.test.sh and scripts/test/record-touched.test.sh: their own assertions call record_case. - scripts/test/shell-tests.test.sh (new): the tests of CA1 to CA3, with the new suite added to the list the runner receives. - scripts/verify.sh: HARNESS_SHELL_REPORT; harness_tests runs the shell suites through the runner and checks the shell report; acceptance adds it to the checker specs; the header comment of harness_tests says so. - AGENTS.md: the "Cobertura" bullet points to the helpers header for naming a shell acceptance test. Existing tests whose behavior changes: none (the shell suites print the same lines; the two with their own assertions also record them). Data changes: none. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ChFCP7iJd9oEJ81Mc5L2p
Shell suites now write a report in the Vitest format (apps/api/reports/harness/shell-tests.json) through a new runner, with the @<ticket>-CA<n> token of a case name as its tag, and the acceptance checker reads it as one more vitest report. The report is rebuilt on every run, so a case that did not run now never covers a criterion. The helpers record each case through pass_case and fail_case, with record_case behind them; the plan named only record_case. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ANcDtWf5vRMzv1r42Ax9ce
Contributor
Author
Revisão de conformidade: LUI-183
Relatório do revisor, na íntegraAchados
Conferências que não geraram achado:
O que não consegui conferir
|
There was a problem hiding this comment.
🔵 Needs a closer look
The reviewed report-generation issue can yield zero recorded tests, and harness changes require human verification.
No open issues 🎉
What changed in this PR
This PR integrates shell harness tests into acceptance coverage through Vitest-compatible reports.
Changes:
- Adds shell test execution and reporting.
- Integrates reports into
verify.shand acceptance checks. - Updates helpers, tests, acceptance criteria, and documentation.
Review note (nit): the report generation currently combines jq -n with inputs, which can produce zero recorded tests.
| File | Description |
|---|---|
scripts/verify.sh |
Runs and validates shell reports. |
scripts/test/stop-verify.test.sh |
Uses updated case helpers. |
scripts/test/shell-tests.test.sh |
Tests shell execution and reporting. |
scripts/test/shell-tests.sh |
Executes suites and generates reports. |
scripts/test/sh-helpers.sh |
Records passed and failed cases. |
scripts/test/record-touched.test.sh |
Uses updated case helpers. |
docs/tickets/LUI-182/LUI-183/aceite.md |
Defines acceptance criteria. |
AGENTS.md |
Documents shell-test coverage. |
Review effort: Lite
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aceite
docs/tickets/LUI-182/LUI-183/aceite.md
O que muda
Os testes em shell do harness passam a gravar um relatório no formato do Vitest (
apps/api/reports/harness/shell-tests.json), com o token@<ticket>-CA<n>do nome do caso como tag, e o conferidor do aceite o lê como mais umvitest:. O relatório é apagado e regravado a cada execução, então um caso que não rodou agora nunca cobre um critério.scripts/test/shell-tests.sh(novo): o executor das suítes em shell e do relatório.scripts/test/sh-helpers.sh: orecord_casee, por cima dele, opass_casee ofail_case. O cabeçalho diz como nomear um caso de aceite e onde fica o relatório (CA4).scripts/test/shell-tests.test.sh(novo): os testes do CA1 a CA3, em shell, com o conferidor de verdade.scripts/verify.sh: oharness_testsroda as suítes pelo executor e confere o relatório, e o "Aceite" lê o relatório novo.Desvio do plano: o commit de plano cita só o
record_casenos auxiliares. O parpass_caseefail_caseveio da/simplify, para o trio imprimir, registrar e contar não ficar copiado em seis pontos. O comportamento é o planejado. Está no achado 1 da revisão de conformidade.Harness
O PR altera
scripts/e oAGENTS.md, então a Guarda do harness vai reprová-lo e ele só entra por bypass do humano.🤖 Generated with Claude Code
https://claude.ai/code/session_01ANcDtWf5vRMzv1r42Ax9ce