fix(test): render the app test-runner HTML report (#3251)#3258
Conversation
The built-in fallback app test runner (vendor/wheels/tests/app-runner.cfm, used when an app has no tests/runner.cfm of its own) emitted raw JSON for the no-format default and ?format=html, so opening /wheels/app/tests in a browser returned an unreadable JSON blob instead of the TestBox-style HTML report the core runner renders. This is item 1 of #3251 (items 2 and 3 shipped in #3255). - Extract the format-to-output decision into TestFormatResolver so the rule is unit-testable without HTTP (mirrors TestDirectoryResolver / TestDbResolver). The html / no-format branch now falls through to html.cfm with type="App" — a branch html.cfm already supported. json / txt / junit are unchanged, and an unrecognized ?format= value still produces no body, matching prior behavior. - Fix a latent Adobe ColdFusion 500 ("Routines cannot be declared more than once") in the shared report template html.cfm: its recursive processNestedSuites helper was a named function declaration, which leaks into the cached Public.cfc scope on Adobe and collides on the second include. It is now a variables-scoped function expression, the same pattern the core runner uses. This also fixes /wheels/core/tests?format=html on Adobe, which 500'd. - Add AppRunnerTestFormatSpec (resolver unit tests) and HtmlReportFunctionDeclarationGuardSpec (source guard against reintroducing a named function in html.cfm, since CI exercises the runners via format=json and never the html render path). Verified on Lucee 7 and Adobe 2023 (Docker): resolver + full dispatch area green (135 specs); /wheels/app/tests renders the HTML report and json/txt/junit are unchanged; format=html works on repeated requests on both engines. Fixes #3251 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Peter Amiri <petera@pai.com>
There was a problem hiding this comment.
Wheels Bot — Reviewer
TL;DR — This PR completes item 1 of #3251: /wheels/app/tests (and the no-format default) now renders the TestBox-style HTML report instead of raw JSON, and it fixes a latent Adobe ColdFusion Routines cannot be declared more than once 500 in the shared html.cfm template. The implementation is tightly scoped, mirrors the existing core-runner conventions exactly, ships a pure unit-testable resolver with TDD coverage plus a structural regression guard, and preserves every prior edge-case behavior. I traced each claim against the source and found no correctness, cross-engine, or security issues. Verdict: approve.
Correctness
Behavior preservation checks out across every branch:
format=json|txt|junitare unchanged; the json branch keeps its417-on-failure logic,Access-Control-Allow-Origin: *header, andinjectScopeMetadatacall (app-runner.cfm:139-161).- Unrecognized/empty
?format=resolves torecognized=false, sotestBoxis never run and no body is emitted — identical to the oldif/elseifchain that had no finalelse(app-runner.cfm:162-165,TestFormatResolver.cfc:79-86). - The no-format→HTML switch does not break the CLI/MCP: every caller passes
format=jsonexplicitly (cli/lucli/Module.cfc:7854,cli/lucli/services/TestRunner.cfc:101,mcp-server.js.txt:400), so only browser users see the new HTML report — exactly the intent. - The HTML branch correctly distinguishes a missing
formatkey (→ HTML) from an emptyformatvalue (→ no output), and both are covered by specs.
Cross-engine
This change is a net positive here. The html.cfm helper conversion from a named declaration to variables.processNestedSuites = function(){...} (html.cfm:44-117) matches the prior-art pattern at vendor/wheels/tests/runner.cfm:20-23, whose comment explicitly cites avoiding "Adobe CF's DuplicateFunctionDefinitionException." Recursion via the variables-scoped reference (html.cfm:48) is safe on all engines. TestFormatResolver.cfc uses only switch/struct literals/LCase/Trim — no .map(), closures, or reserved-scope params. The guard spec's Left(trimmed, 2) / Left(trimmed, 1) use literal lengths (never 0), so Cross-Engine Invariant #8 does not fire.
Tests
AppRunnerTestFormatSpec.cfcgives 10 BDD specs (extendswheels.WheelsTest) covering each format, case-insensitivity, trimming, and both unrecognized paths.HtmlReportFunctionDeclarationGuardSpec.cfcis a sound structural guard: I confirmed\bfunction\s+[a-zA-Z_]matches only the two//-prefixed comment lines inhtml.cfm(37-38), which the comment-skip logic drops, so the guard passes cleanly and won't false-positive on the inline-<script>IIFE or function expressions.- Informational only (not blocking): the actual
app-runner.cfminclude wiring + statuscode is not exercised by an integration test, because CI hits the runners withformat=jsonand never thehtmlrender path. The PR acknowledges this and mitigates it with the structural guard plus manual Lucee 7 / Adobe 2023 verification — a reasonable trade-off given the constraint.
Docs
Changelog fragment changelog.d/3251-app-test-html-report.fixed.md follows the <slug>.<type>.md convention (fixed) — no direct [Unreleased] edit. Inline comments are thorough and accurate.
Commits
fix(test): render the app test-runner HTML report (#3251) conforms to commitlint.config.js: valid type fix, valid scope test, subject ≤ 100 chars, not ALL-CAPS.
Security
No concerns. The Access-Control-Allow-Origin: * header is confined to the json branch exactly as before; the HTML branch is human-facing and adds no new header. No user input reaches SQL, and the directory scoping continues to flow through the existing TestDirectoryResolver allowlist.
Nice work — the latent-Adobe-500 catch and the accompanying guard are exactly the right instinct given that the format=html path is invisible to CI.
What & why
Resolves item 1 of #3251:
/wheels/app/tests?format=html— and the no-format default — returned raw JSON instead of the TestBox-style HTML report the core runner (/wheels/core/tests) renders. A user opening the endpoint in a browser got an unreadable JSON blob.Items 2 (subpath include) and 3 (testing-guide reconciliation) already shipped in #3255, so this PR completes #3251.
Design choice: render (not document)
The issue offered "render a real HTML report or document the JSON-only limitation." I chose render because the infrastructure was already built for it —
html.cfmhas had a dedicatedtype="App"branch (package=tests.specs,route=testbox) all along, and thetestboxroute resolves to/wheels/app/tests. The app runner simply never wired the fall-through. This is the completion of pre-existing work, and matches what the core runner does (type="Core").Changes
TestFormatResolver.cfc(new) — extracts the format→output decision (reporter / content-type /rendersHtml/recognized) into a pure, unit-testable helper, mirroring the existingTestDirectoryResolver/TestDbResolverin the same directory.app-runner.cfm— consumes the resolver. Thehtml/ no-format branch now falls through tohtml.cfmwithtype="App".json/txt/junitare unchanged. An unrecognized?format=value (e.g.xml, or an empty value) still produces no body — exactly as before.html.cfm— fixes a latent Adobe ColdFusion 500. Its recursiveprocessNestedSuiteshelper was a namedfunctiondeclaration; on Adobe a named function in an included.cfmleaks into the cachedPublic.cfcsingleton scope and throwsRoutines cannot be declared more than onceon the second include. It's now a variables-scoped function expression — the same pattern the core runner already uses for its helpers. This also fixes/wheels/core/tests?format=htmlon Adobe, which 500'd on every request.AppRunnerTestFormatSpec.cfc(new) — TDD unit tests for the resolver (10 specs).HtmlReportFunctionDeclarationGuardSpec.cfc(new) — source guard that fails if a named function is reintroduced intohtml.cfm. CI exercises the runners viaformat=jsonand never thehtmlrender path, so without this guard a regression of the Adobe bug would be invisible (which is exactly why it stayed latent).TDD
Red → green on the resolver (watched the missing-CFC error, then the
recognized-field refinement fail, then pass). The Adobe 500 was found via systematic debugging — captured the real exception (Routines cannot be declared more than once) and confirmed the original code also 500'd the core HTML report on Adobe, proving it pre-existing rather than introduced.Verification (Lucee 7 + Adobe 2023, Docker)
dispatcharea (135)html/ no-format → HTML reportjson/txt/junitformat→ empty 200htmlrepeated requestsFixes #3251
🤖 Generated with Claude Code