Skip to content

fix(test): render the app test-runner HTML report (#3251)#3258

Merged
bpamiri merged 1 commit into
developfrom
claude/nice-mclaren-613ac6
Jun 24, 2026
Merged

fix(test): render the app test-runner HTML report (#3251)#3258
bpamiri merged 1 commit into
developfrom
claude/nice-mclaren-613ac6

Conversation

@bpamiri

@bpamiri bpamiri commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

What & why

Resolves item 1 of #3251: /wheels/app/tests?format=html — and the no-format default — returned raw JSON instead of the TestBox-style HTML report the core runner (/wheels/core/tests) renders. A user opening the endpoint in a browser got an unreadable JSON blob.

Items 2 (subpath include) and 3 (testing-guide reconciliation) already shipped in #3255, so this PR completes #3251.

Design choice: render (not document)

The issue offered "render a real HTML report or document the JSON-only limitation." I chose render because the infrastructure was already built for it — html.cfm has had a dedicated type="App" branch (package=tests.specs, route=testbox) all along, and the testbox route resolves to /wheels/app/tests. The app runner simply never wired the fall-through. This is the completion of pre-existing work, and matches what the core runner does (type="Core").

Changes

  • TestFormatResolver.cfc (new) — extracts the format→output decision (reporter / content-type / rendersHtml / recognized) into a pure, unit-testable helper, mirroring the existing TestDirectoryResolver / TestDbResolver in the same directory.
  • app-runner.cfm — consumes the resolver. The html / no-format branch now falls through to html.cfm with type="App". json / txt / junit are unchanged. An unrecognized ?format= value (e.g. xml, or an empty value) still produces no body — exactly as before.
  • html.cfm — fixes a latent Adobe ColdFusion 500. Its recursive processNestedSuites helper was a named function declaration; on Adobe a named function in an included .cfm leaks into the cached Public.cfc singleton scope and throws Routines cannot be declared more than once on the second include. It's now a variables-scoped function expression — the same pattern the core runner already uses for its helpers. This also fixes /wheels/core/tests?format=html on Adobe, which 500'd on every request.
  • AppRunnerTestFormatSpec.cfc (new) — TDD unit tests for the resolver (10 specs).
  • HtmlReportFunctionDeclarationGuardSpec.cfc (new) — source guard that fails if a named function is reintroduced into html.cfm. CI exercises the runners via format=json and never the html render path, so without this guard a regression of the Adobe bug would be invisible (which is exactly why it stayed latent).

TDD

Red → green on the resolver (watched the missing-CFC error, then the recognized-field refinement fail, then pass). The Adobe 500 was found via systematic debugging — captured the real exception (Routines cannot be declared more than once) and confirmed the original code also 500'd the core HTML report on Adobe, proving it pre-existing rather than introduced.

Verification (Lucee 7 + Adobe 2023, Docker)

Check Lucee 7 Adobe 2023
Resolver spec (10)
Full dispatch area (135)
app html / no-format → HTML report
app json / txt / junit ✅ unchanged ✅ unchanged
app unknown / empty format → empty 200 ✅ (was 500)
core html repeated requests ✅ (was 500)
guard catches a reintroduced named fn

Note: CI runs the test endpoints with format=json, so the html.cfm render path is not covered by the normal suite on any engine — the format=html behavior above was verified locally on Lucee 7 and Adobe 2023.

Fixes #3251

🤖 Generated with Claude Code

The built-in fallback app test runner (vendor/wheels/tests/app-runner.cfm,
used when an app has no tests/runner.cfm of its own) emitted raw JSON for the
no-format default and ?format=html, so opening /wheels/app/tests in a browser
returned an unreadable JSON blob instead of the TestBox-style HTML report the
core runner renders. This is item 1 of #3251 (items 2 and 3 shipped in #3255).

- Extract the format-to-output decision into TestFormatResolver so the rule is
  unit-testable without HTTP (mirrors TestDirectoryResolver / TestDbResolver).
  The html / no-format branch now falls through to html.cfm with type="App" — a
  branch html.cfm already supported. json / txt / junit are unchanged, and an
  unrecognized ?format= value still produces no body, matching prior behavior.

- Fix a latent Adobe ColdFusion 500 ("Routines cannot be declared more than
  once") in the shared report template html.cfm: its recursive
  processNestedSuites helper was a named function declaration, which leaks into
  the cached Public.cfc scope on Adobe and collides on the second include. It is
  now a variables-scoped function expression, the same pattern the core runner
  uses. This also fixes /wheels/core/tests?format=html on Adobe, which 500'd.

- Add AppRunnerTestFormatSpec (resolver unit tests) and
  HtmlReportFunctionDeclarationGuardSpec (source guard against reintroducing a
  named function in html.cfm, since CI exercises the runners via format=json and
  never the html render path).

Verified on Lucee 7 and Adobe 2023 (Docker): resolver + full dispatch area green
(135 specs); /wheels/app/tests renders the HTML report and json/txt/junit are
unchanged; format=html works on repeated requests on both engines.

Fixes #3251

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <petera@pai.com>

@wheels-bot wheels-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wheels Bot — Reviewer

TL;DR — This PR completes item 1 of #3251: /wheels/app/tests (and the no-format default) now renders the TestBox-style HTML report instead of raw JSON, and it fixes a latent Adobe ColdFusion Routines cannot be declared more than once 500 in the shared html.cfm template. The implementation is tightly scoped, mirrors the existing core-runner conventions exactly, ships a pure unit-testable resolver with TDD coverage plus a structural regression guard, and preserves every prior edge-case behavior. I traced each claim against the source and found no correctness, cross-engine, or security issues. Verdict: approve.

Correctness

Behavior preservation checks out across every branch:

  • format=json|txt|junit are unchanged; the json branch keeps its 417-on-failure logic, Access-Control-Allow-Origin: * header, and injectScopeMetadata call (app-runner.cfm:139-161).
  • Unrecognized/empty ?format= resolves to recognized=false, so testBox is never run and no body is emitted — identical to the old if/elseif chain that had no final else (app-runner.cfm:162-165, TestFormatResolver.cfc:79-86).
  • The no-format→HTML switch does not break the CLI/MCP: every caller passes format=json explicitly (cli/lucli/Module.cfc:7854, cli/lucli/services/TestRunner.cfc:101, mcp-server.js.txt:400), so only browser users see the new HTML report — exactly the intent.
  • The HTML branch correctly distinguishes a missing format key (→ HTML) from an empty format value (→ no output), and both are covered by specs.

Cross-engine

This change is a net positive here. The html.cfm helper conversion from a named declaration to variables.processNestedSuites = function(){...} (html.cfm:44-117) matches the prior-art pattern at vendor/wheels/tests/runner.cfm:20-23, whose comment explicitly cites avoiding "Adobe CF's DuplicateFunctionDefinitionException." Recursion via the variables-scoped reference (html.cfm:48) is safe on all engines. TestFormatResolver.cfc uses only switch/struct literals/LCase/Trim — no .map(), closures, or reserved-scope params. The guard spec's Left(trimmed, 2) / Left(trimmed, 1) use literal lengths (never 0), so Cross-Engine Invariant #8 does not fire.

Tests

  • AppRunnerTestFormatSpec.cfc gives 10 BDD specs (extends wheels.WheelsTest) covering each format, case-insensitivity, trimming, and both unrecognized paths.
  • HtmlReportFunctionDeclarationGuardSpec.cfc is a sound structural guard: I confirmed \bfunction\s+[a-zA-Z_] matches only the two //-prefixed comment lines in html.cfm (37-38), which the comment-skip logic drops, so the guard passes cleanly and won't false-positive on the inline-<script> IIFE or function expressions.
  • Informational only (not blocking): the actual app-runner.cfm include wiring + statuscode is not exercised by an integration test, because CI hits the runners with format=json and never the html render path. The PR acknowledges this and mitigates it with the structural guard plus manual Lucee 7 / Adobe 2023 verification — a reasonable trade-off given the constraint.

Docs

Changelog fragment changelog.d/3251-app-test-html-report.fixed.md follows the <slug>.<type>.md convention (fixed) — no direct [Unreleased] edit. Inline comments are thorough and accurate.

Commits

fix(test): render the app test-runner HTML report (#3251) conforms to commitlint.config.js: valid type fix, valid scope test, subject ≤ 100 chars, not ALL-CAPS.

Security

No concerns. The Access-Control-Allow-Origin: * header is confined to the json branch exactly as before; the HTML branch is human-facing and adds no new header. No user input reaches SQL, and the directory scoping continues to flow through the existing TestDirectoryResolver allowlist.

Nice work — the latent-Adobe-500 catch and the accompanying guard are exactly the right instinct given that the format=html path is invisible to CI.

@bpamiri bpamiri merged commit 2a2ca9c into develop Jun 24, 2026
10 checks passed
@bpamiri bpamiri deleted the claude/nice-mclaren-613ac6 branch June 24, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test runner: app test endpoint and runner.cfm break under subfolder installs

1 participant