testaro

Ensemble testing for web accessibility

Breaking change notices

Version 75.0.0 introduced a breaking change in the methods for making screenshots of web pages.

Version 68.0.0 introduced a breaking change in the format of reports.

Changes in each version since 73.0.0 are listed in Version change notes.

Purposes

Testaro is an application that performs ensemble testing of web pages for accessibility, usability, and conformity to HTML and CSS specifications.

The purposes of Testaro are to:

• provide programmatic access to tests defined by multiple rule engines
• standardize and integrate the reports of the rule engines

The need for ensemble testing of web accessibility, and the obstacles to it, are discussed in Accessibility Metatesting: Comparing Nine Testing Tools.

Testaro is described in two papers:

• How to run a thousand accessibility tests
• Testaro: Efficient Ensemble Testing for Web Accessibility

Functionality

Testaro performs tasks defined by a job. Typically, a job identifies the URL of a web page and asks Testaro to call an ensemble of rule engines to test the page. Testaro adds the results of the testing to the job, thereby converting the job to a report.

Testaro can be given a job to perform, in which case it performs the job, delivers the report, and quits.

Alternatively, testaro can run as a daemon, polling a server or a directory for new jobs and performing them when they are provided or when they appear in the directory.

A practical application that leverages Testaro will use other software to prepare jobs, schedule them, post-process the reports as needed, and manage the report files. Some utilities for such purposes can be found in the Testilo project. One application that leverages Testaro for a web service is Kilotest.

Dependencies

Testaro uses:

• Playwright to launch browsers, perform user actions in them, and perform tests
• playwright-extra and puppeteer-extra-plugin-stealth to make a Playwright-controlled browser more indistinguishable from a human-operated browser and thus make its requests more likely to succeed
• playwright-dompath to retrieve XPaths of elements
• pixelmatch to measure motion
• dotenv to load environment variables

Testaro can perform tests of these rule engines:

• Accessibility Checker (IBM)
• Alfa (Siteimprove)
• ASLint (eSSENTIAL Accessibility)
• Axe (Deque)
• Editoria11y (Princeton University)
• HTML CodeSniffer (Squiz Labs)
• Nu Html Checker (World Wide Web Consortium)
• Pour Engine (David Yarham and Geoffrey Crofte)
• QualWeb (University of Lisbon)
• SureA11y (Jorge Rumoroso)
• Testaro (CVS Health)
• WAVE (WebAIM)

For the rule engines that are open-source, the identified organizations or persons are their principal or original sponsors or authors.

As shown, Testaro is not only an integrator but also one of the integrated rule engines. That is because it provides about 50 tests of its own, mostly to complement tests provided by the other rule engines. Some of those Testaro tests are designed to act as approximate alternatives to tests of vulnerable, restricted, or no longer available rule engines. In all such cases the Testaro tests are independently designed and implemented, without reference to the code of the tests that inspired them.

Concepts and terms

The main concepts of Testaro are:

• job: a document that tells Testaro what to do.
• act: one step in a job.
• report: a job that Testaro has added results to.
• rule engine: one of the testing applications in the ensemble assembled by Testaro.
• rule: a success or failure criterion defined by a rule engine (currently about 1300 across all rule engines).
• test: the software that a rule engine uses to apply a rule.
• target: a web page that a job tells Testaro to test.
• result: the information that Testaro adds to a job to describe the outcomes of the tests of a rule engine.
• native result: the outcomes of the tests of a rule engine in exactly or approximately the original form.
• standard result: the outcomes of the tests of a rule engine in a uniform Testaro-defined form.
• catalog: a collection of data on the HTML elements of a target relevant to one or more tests.

System requirements

Operating system and Node.js version

Testaro can be installed under a MacOS, Windows, Debian, or Ubuntu operating system with the latest long-term-support version of Node.js. The minimum version is Node 22.12: Node 20 reached end of life in April 2026, and some dependencies (the Alfa packages and pixelmatch) are ES modules that Testaro loads with require(), which the Node 22 line supports from 22.12 onward. When Node 22 reaches end of life in April 2027, the minimum is expected to rise to Node 24.

Browser security

Testaro is configured so that, when Playwright launches a chromium browser, the browser is sandboxed for improved security. That is the default for Playwright, and Testaro does not override that default. The host must therefore permit sandboxed browsers. If you try to run Testaro on a host that prohibits sandboxed browsers, each attempted launch of a chromium browser will throw an error with a message complaining about the unavailability of a sandbox.

In some operating systems a sandboxed browser requires an unprivileged user namespace. In one case, a …userns.conf file in the /etc/sysctl.d directory with the content kernel.apparmor_restrict_unprivileged_userns = 1 prohibits unprivileged user namespaces and thereby makes sandboxed browsers unlaunchable.

Option A

One way to cope with this prohibition is to configure Playwright to launch chromium non-sandboxed. Launch arguments '--no-sandbox' and '--disable-setuid-sandbox' are available to specify this. They are added to the arguments of browserOptionArgs.push in the Testaro run.js file.

This option is not available for the Playwright chromium browser launched by QualWeb.

Non-sandboxed browsers are less secure than sandboxed ones, particularly when there is no restriction on who can use Testaro and what targets (web pages) they can test with it.

Option B

Another solution is to leave the chromium configuration unchanged, but configure the operating system to permit a sandboxed browser to be launched. In one case, this is implemented with:

sudo sysctl -w kernel.unprivileged_userns_clone=1
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
sudo tee /etc/sysctl.d/99-kilotest-userns.conf >/dev/null <<'EOF'
kernel.unprivileged_userns_clone = 1
kernel.apparmor_restrict_unprivileged_userns = 0
EOF
sudo sysctl --system

This application implements option B.

Installation as an independent application

To install Testaro as an independent application, rather than a dependency, clone the Testaro repository. To ensure that the binary browsers of its Playwright dependency get installed, execute (p)npx playwright install after executing (p)npm install.

To update Testaro when it is an independent application, execute:

git checkout package-lock.json
git pull
(p)npm run deps

Environment configuration

The .env file stores your decisions about the environment in which Testaro runs. The variables that can be defined there are documented in the env.example file.

Deployment options let an operator set a fleet-wide policy that a job can override: the load state and time limit a navigation waits for and whether a 4xx response ends launch retries at once (NAV_WAIT_UNTIL, NAV_TIMEOUT, NAV_FAIL_FAST_4XX, or a job's navigation property); a scanner identity sent as the X-YRA-Scanner request header (SCANNER_ID or a job's scannerId); a full-page scroll after navigation so lazily loaded content is present (PRESCAN_SCROLL or a job's scroll); a branded Chromium channel (a job's browserChannel); launch retries per rule of the testaro tool (TESTARO_RULE_RETRIES or a testaro test act's retries); and whether the qualWeb tool's browser runs stealth evasions and blocks ads and trackers (QUALWEB_STEALTH, QUALWEB_ADBLOCK, or a qualWeb test act's stealth and adBlock). The defaults keep the behavior of earlier versions.

Jobs

Jobs tell Testaro what to do.

Here is a sample job, showing properties that you can set:

{
  id: 'healthcheck2611', // Job identifier
  what: 'monthly health check', // Job description
  strict: true, // Whether to reject redirections from the target URL
  standard: 'only', // Report native (no), standard (only), or both (also) results
  imageColor: 0, // Color type (0 (grayscale), 2 (RGB), 4 (grayscale alpha), 6 (RGBA)) if initial page image required
  imageScale: 2, // Optional: also capture the page image at this device pixel density (see the images section)
  device: { // Device to emulate
    id: 'iPhone 8',
    windowOptions: {
      reducedMotion: 'no-preference',
      userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/17.4 Mobile/15A372 Safari/604.1',
      viewport: {
        width: 375,
        height: 667
      },
      deviceScaleFactor: 2,
      isMobile: true,
      hasTouch: true,
      defaultBrowserType: 'webkit'
    }
  },
  browserID: 'chromium', // or 'webkit' or 'firefox'
  stealth: true, // Optional. Whether to enable puppeteer-extra-plugin-stealth
                 // evasions. Only applies to Chromium (the plugin is
                 // Chromium-specific). Defaults to true. Set false to opt
                 // out — useful for sites whose anti-bot heuristics react
                 // badly to stealth's patches.
  browserChannel: 'chrome', // Optional. Run an installed branded Chromium (chrome or msedge)
                            // instead of the bundled build; bundled or absent keeps the default.
  scannerId: 'MyScanner/1.0; +https://example.com/scanner', // Optional. Sent as the
                            // X-YRA-Scanner request header (default SCANNER_ID, else no header).
  scroll: false, // Optional. Scroll the full page after navigation so lazily loaded content
                 // is present before tools run (default PRESCAN_SCROLL, else false).
  navigation: { // Optional. Load state to wait for (networkidle, load, or domcontentloaded),
                // time limit in ms, and whether a 4xx response ends launch retries at once
                // (defaults NAV_WAIT_UNTIL, NAV_TIMEOUT, NAV_FAIL_FAST_4XX, else networkidle,
                // 10000, false).
    waitUntil: 'load',
    timeout: 30000,
    failFast4xx: true
  },
  creationTimeStamp: '241229T0537', // When job was created
  executionTimeStamp: '250110T1200', // When job will be ready to be performed
  target: {
    what: 'Real Estate Management',
    url: 'https://abccorp.com/mgmt/realproperty'
  },
  sources: { // Any data the requester chooses to add
    script: 'ts99',
    batch: 'departments',
    mergeID: '7f',
    requester: 'malavu@abccorp.com'
  },
  acts: [ // Steps in this job
    {
      type: 'test', // Act type (the 'test' type performs tests of a rule engine)
      launch: {}, // Act-specific overrides for the browserID and/or target
      which: 'axe', // ID of the rule engine
      detailLevel: 2, // An argument required by this rule engine
      rules: ['landmark-complementary-is-top-level'], // Which rules of the rule engine to test for
    },
    {
      type: 'test',
      launch: {
        browserID: 'webkit', // For this act, override browserID to use Webkit
        target: { // For this act, override target to test the contact page
          what: 'Real Estate Management contact',
          url: 'https://abccorp.com/mgmt/realproperty/contactus'
        }
      },
      which: 'qualWeb',
      rules: ['QW-BP25', 'QW-BP26'] // Which rules of the rule engine to test for
    }
  ]
}

The device property lets you choose among about 125 devices recognized by Playwright.

The browserID property dictates the first browser type that Testaro uses for its requests. When requests with that browser type are repeatedly rejected, Testaro switches to a different browser type and also replaces the browserID property of the job with that new browser type. Targets and their edge-management platforms often block requests from one browser type but not another, so switching browser types can make a job succeed instead of failing or make it finish quickly instead of slowly.

The act types and their options are documented in the etc property of the actSpecs.js object and in the actSpecs-doc.md file.

Running jobs

Job as an object

Simple execution

An application can execute a job with:

const {doJob} = require('testaro/run');
// Perform the job, which adds content to the job, making it a report.
doJob(job)
.then(report => {
  // Optionally, modify the report.
  …
  return report;
});

Execution with observability

If the application is capable of listening for events while a job is being executed, it is possible to do that by passing a second argument to doJob:

const {doJob} = require('testaro/run');
// Perform the job, which adds content to the job, making it a report.
doJob(job, {
  onProgress: event => {
    // Handle the event. For example:
    console.log(event);
  }
})
.then(report => {
  // Optionally, modify the report.
  …
  return report;
});

Testaro emits events during job execution that allow an application to monitor progress. The events are:

• jobStart - Emitted when a job starts.
• catalogStart - Emitted when catalog compilation starts.
• catalogEnd - Emitted when catalog compilation ends.
• actStart - Emitted when an act starts.
• actEnd - Emitted when an act ends.
• jobEnd - Emitted when a job ends.

Job as a file

Jobs can be stored as JSON files in the todo subdirectory of a directory identified by the JOBDIR environment variable. After Testaro performs such a job, Testaro moves the job file to the done subdirectory of the same directory and saves the report in the raw subdirectory of the directory identified by the REPORTDIR environment variable.

Immediate performance

A user can make Testaro perform a job from a file with a command like either of:

node call run
node call run 250725T

Testaro will find the first file in the todo subdirectory, or, if the second form of the command is used, the first file there whose name begins with the specified string.

Directory polling

An application can poll the todo subdirectory for jobs with:

const {dirWatch} = require('testaro/dirWatch');
dirWatch(true, 300);

A user can make Testaro start to poll that subdirectory with:

node call dirWatch true 300

In both cases, the first argument of dirWatch tells Testaro whether to continue polling after performing one job, and the second argument tells Testaro how many seconds to wait after not finding a job to perform, before polling again.

Server polling

Testaro can poll a server for jobs to be performed. The server can act as the “controller” described in How to run a thousand accessibility tests. The server is responsible for preparing Testaro jobs, assigning them to Testaro workers, receiving reports back from those workers, and performing any further processing of the reports, including enhancement, storage, and disclosure to audiences. It can be any server reachable with a URL. That includes a server running on the same host as Testaro, with a URL such as localhost:3000.

To allow Testaro to poll a server for jobs, define the environment variables documented under netWatch variables in the env.example file. The URL paths are determined by agreement between Testaro and the server. A single Testaro instance can watch one server.

NETWATCH_AUTH_TYPE selects how Testaro authenticates to the server:

• none: no credentials are sent. NETWATCH_WORKER_ID and NETWATCH_WORKER_SECRET are not required.
• pathBody: the password (NETWATCH_WORKER_SECRET) is transmitted in the request body as the value of an agentPW property. If the server requires the ID of the Testaro instance in the URL path, include it in NETWATCH_URL_JOB and NETWATCH_URL_REPORT yourself; Testaro does not insert it.
• header: the request carries an authorization header whose value is Basic, followed by a space and the base64 encoding of NETWATCH_WORKER_ID:NETWATCH_WORKER_SECRET. The worker ID must not contain a colon.

Testaro sends job requests and completed reports as POST requests. When Testaro sends a report to the server, the report is the value of a report property in the request body. If NETWATCH_WORKER_ID is defined, Testaro also records it as the sources.agent property of the report, so the server can attribute the report to this instance under any auth type.

The AGENT and NETWATCH_URL_AUTH variables of earlier versions are deprecated. Testaro still honors them (as NETWATCH_WORKER_ID and as a pathBody password, respectively) but warns; rename them.

An application can make Testaro poll a server for jobs with:

const {netWatch} = require('testaro/netWatch');
netWatch(true, 300);

A user can make Testaro poll a server for jobs with:

node call netWatch true 300

The first argument of netWatch tells Testaro whether to continue polling after performing the first job. The second argument tells Testaro how many seconds to wait after receiving a no-jobs response before polling again. The optional third argument tells Testaro whether to be certificate-tolerant, i.e. to accept SSL certificates that fail verification against a list of certificate authorities (the default is false). Certificate tolerance disables the protection of an https connection, exposing credentials and reports to interception, so use it only against servers you control, such as local test servers with self-signed certificates.

Reports

A report is a job with information about the results of the performance of the job inserted by Testaro into the job.

Whole-job data

As Testaro performs a job, information about the job as a whole is inserted into the job. That information is organized into one, two, or three properties:

• jobData: Facts about the performance of the job
• catalog: A collection of data about the HTML elements of the target that are relevant to any test failures
• images: A collection of page images captured during the job

jobData

Testaro inserts the jobData property into every job.

catalog

Testaro inserts the catalog property only into jobs that instruct Testaro to produce standard results. The catalog is an inventory of HTML elements in the DOM of the target.

The catalog property has an object value. Here is an example:

'123': {
  tagName: 'SUMMARY',
  id: 'functionsummary',
  startTag: '<summary>',
  text: 'Functional\nAlways active',
  textLinkable: true,
  boxID: '46:230:860:263',
  pathID: '/html/body/div[1]/div[1]/div[3]/div[2]/details[1]/summary[1]'
},

If the inner text of the element consists of only one line, that is the value of text. If the inner text consists of two or more lines, text is the first and last of these, delimited with a newline.

The textLinkable property has a true value whenever text is non-empty and can generate a text-fragment URL that uniquely identifies an element.

The segments of boxID are x, y, width, and height.

The catalog is a mechanism for the integration of the rule engines. Most rule violations that rule engines report are blamed on particular HTML elements. A rule engine typically reports that an element violated a rule by having some defect in its configuration or behavior. But rule engines describe elements differently. Testaro makes the rule engines identify the XPaths of the elements they report as violators. Testaro then finds, for each XPath, the correct catalog entry.

Testaro uses the following techniques to make the rule engines calculate XPaths:

• alfa and aslint: They report XPaths, so Testaro needs only to normalize them.
• ed11y: Testaro adds it and a window.getXPath method to the page. When the rule engine reports an element, Testaro computes its XPath.
• pour and surea11y: Testaro adds the vendored engine bundle and a window.getXPath method to the page. Each engine reports a CSS selector for each element; Testaro finds the element in the page via its selector and executes window.getXPath on it.
• wave: It reports a selector for each element; Testaro finds each element in the page via its selector and executes window.getXPath on the element.
• htmlcs, ibm, nuVal, nuVnu, qualWeb: Testaro adds data-xpath attributes to all elements. The rule engines include code excerpts, with the data-xpath attributes, in the reported violations.
• axe: It reports a selector for each element, and Testaro adds data-xpath attributes to all elements. Testaro finds each element in the page via its selector and uses the data-xpath attribute. When this fails, Testaro uses the data-xpath attribute if its complete value is included in the reported node.html value.
• testaro: Testaro designs each of its own tests to report element XPaths.

By attaching a catalog entry to each reported element, Testaro allows an application that uses Testaro to tell users, for any particular HTML element, which rule engines ascribed violations of which rules to that element. An application could, for example, use a screenshot or a text-fragment link or could ask the user to paste the XPath into a browser developer tool.

In some cases no catalog entry can be found. The reasons may include:

• The element was dynamically created after the catalog was created.
• The element is inside a noscript element and therefore not considered an element in the DOM.
• The violation is not ascribed to a single element.

images

Testaro inserts an images array property if necessary to store page images in the report. If the job has an imageColor property with 0, 2, 4, or 6 as its value and Testaro will insert a catalog property, then Testaro also creates a page image with that color type and makes its base64-encoded PNG the first item in the images array. The first item is always captured at CSS-pixel scale (one image pixel per CSS pixel), so the motion test of the testaro tool can compare it with its own CSS-pixel screenshot.

If the job also has an imageScale property with a number greater than 1 as its value, then the catalog page is rendered at that device scale factor, and Testaro captures a second page image at device-pixel scale and makes it the second item in the images array. That image has imageScale times the pixels of the CSS layout in each dimension, for crisp display on high-resolution screens. The boxID properties of the catalog remain in CSS pixels; consumers can map them onto the second image by multiplying the coordinates by imageScale. Fractional values (such as a device's native 2.625) are valid. A natural choice is the emulated device's own deviceScaleFactor, which also makes the catalog page select the same srcset/image-set resources as the test pages. If imageScale is omitted, 1, or invalid, the behavior is identical to that before this property existed.

There is a shoot act type that can be used to make additional page images during a job.

Act data

As Testaro performs the acts of a job, information about the result of each act is inserted into that act. For acts of type test, the added properties are:

• startTime: When Testaro began to perform the act
• actualURL: The tested URL (different from the target URL if the request was redirected)
• data: Data generated by the rule engine
• result: Result of the testing by the rule engine

The result property is an object with one or two (depending on the value of standard, as described above) subproperties:

• nativeResult: The result (or a compact version of the result) natively produced by the rule engine
• standardResult: A Testaro-standardized version of the result

If an act of type test contains an expect property (specifying expectations about the result), then Testaro also inserts these properties into the act:

• expectations: Data on what was expected versus the actual result
• expectationFailures: The count of failed expectations

Details about these expectation properties are documened in the VALIDATION.md file.

Standard results

If the job instructs Testaro to include standard results, then the result.standardResult property of each act of type test will have four properties:

• prevented: Whether the rule engine was prevented from performing the act
• totals: An array of 4 integers, counting the rule violations at 4 severity levels
• outcomeTotals: An object counting the rule violations by outcome: {failed, cantTell}
• instances: An array of data about the violations reported by the rule engine

More specifically:

• The totals value is an array like this: [3, 0, 87, 4]. This example would mean that the rule engine reported 3 failures at severity 0 (the least severe level), none at severity 1, 87 at severity 2, and 4 at severity 3. These four severities are conceptually ordinal, not metric.
• The outcomeTotals value is an object like this: {failed: 87, cantTell: 7}. It counts the violations (weighted by count) that the rule engine asserted versus those it flagged as uncertain.
• The instances value is an array of objects, each having these properties:
  • ruleID: The ID of the rule that was violated
  • what: A description of the rule or of the violation
  • ordinalSeverity: The severity of the violation
  • outcome: failed if the rule engine asserted the violation, or cantTell if the rule engine reported that it could not determine whether the rule was violated (the vocabulary of the ACT Rules Format). Every instance has an outcome.
  • uncertainty (only with cantTell, and only if the rule engine gave a reason): One of not-computable, judgement-required, runtime-dependent, spec-only, equivalence-unknown, out-of-scope
  • needed (only with cantTell, and only if the rule engine said): What a reviewer must determine to resolve the uncertainty
  • count: How many violations of the rule this instance reports
  • catalogIndex: Key of the HTML element in the catalog

If no catalog entry was found for the instance, then instead of a catalogIndex property Testaro tries to insert a pathID property, whose value is a normalized XPath of the offending HTML element.

The outcome property is the authoritative certainty signal. In version 78, each tool's ordinalSeverity conventions are unchanged from earlier versions: most tools encode uncertainty as a low severity (for example, axe incomplete results have severities 0 and 1, and violations have severities 2 and 3), so ordinalSeverity still mixes certainty with impact, and it mixes them differently per tool. Consumers should read outcome for certainty and should not infer it from ordinalSeverity. A later major version will redefine ordinalSeverity as impact only (0 minor, 1 moderate, 2 serious, 3 critical). Design record: docs/standard-result-outcome.md.

Testaro's own rules report failed unless the rule's entry in allRules (in tests/testaro.ts) specifies outcome: 'cantTell' (for example, allCaps, whose violations are AI estimates) or a violation description carries a prefix: 2: sets severity 2, 2?: sets severity 2 and outcome cantTell, and ?: sets outcome cantTell at the rule's default severity.

Checkpoints

A job's acts run in order: a launch act opens a page, interaction acts (button, link, text, press, url, and so on) act on it, and test acts run rule engines. A checkpoint is a named page state reached by that flow, snapshotted and tested. Checkpoint 0 is the job target as launched; a checkpoint act ({type: 'checkpoint', which: 'name'}) creates the next one from the live page. A test act tests the most recent checkpoint, and each of its standard instances carries the checkpoint index. report.checkpoints[k] describes each checkpoint: its name, URL, title, the page images (imageIndexes into report.images), the catalog entries it added (catalogRange; every catalog entry also carries checkpoint), an ARIA snapshot, and, for a state reached by interaction, the acts (replay) that a test act's browser re-enacts after navigating to launchURL before the rule engine runs. Each such test act records data.replay with the count of replayed acts and a fidelity of exact or divergent, comparing the replayed page's DOM with the snapshot.

A job's isolation property (or the ISOLATION environment default) sets how test acts are isolated: process (the default, and the previous behavior) runs each test act in a child process with its own browser; browser runs test acts in the job's process, each in a fresh context of one browser shared by the job; page runs them in the job's process on the live page of the current checkpoint, with no replay, so tools may contaminate one another's page (contaminating testaro rules still get a fresh page). jobData.isolation records the level used. Only process isolation kills a tool that overruns its time limit; the others report the act as timed out and continue.

When a job has two or more checkpoints, the report gains flow, the running list of issues across them. flow.checkpoints[k] summarizes each checkpoint (the tools that observed it and its issue count), and flow.deltas[k - 1] compares checkpoints k - 1 and k: the issues added by the acts between them, those that persisted, and those removed, each identified by tool, rule ID, element XPath (pathID), and start tag, so the same element in two page states is one issue. Only tools that observed both checkpoints are compared (tools); the rest are listed as notObserved. An earlier issue that a tool did not re-test at the later checkpoint, because its acts there were all scoped to changed subtrees that exclude the issue's element, is listed as notRetested, not removed. Each delta also carries structure, the difference between the two checkpoints' catalogs as XPaths (added, removed, changed start tags, textChanged, and the outermost changed elements as roots), and aria, a line diff of the two ARIA snapshots.

Interaction acts can identify their elements by a selector property (any Playwright selector, such as #name, text="Continue", or role=button[name="Continue"]) instead of an element type and text substring; a text act with clear: true replaces the input's value. With these, a recorded user path (a start URL and actions of the kinds a Playwright recording yields: click, fill, select, navigate, wait, checkpoint) converts directly into a job: require('testaro/procs/userPath').getUserPathActs({startUrl, actions, testActs}) returns the acts, one per action, with each checkpoint action becoming a checkpoint act followed by a copy of each test act in testActs.

When a checkpoint is created, the catalog entries of the previous checkpoint that none of its test acts cited are pruned at once, since no later test act can cite them, so a long flow does not accumulate every page state's catalog in the report that each test act reads.

A test act may have a scope property. The default, page, tests the whole page of the act's checkpoint. With changed, the act tests only the subtrees that changed since the previous checkpoint (the roots of the structure diff, as CSS selectors), for the rules and tools that can be so restricted: the testaro rules declared element-local (local: true in allRules), axe (by its include context), and surea11y (by its context selector, the nearest common ancestor of the roots). Page-level testaro rules (heading order, landmarks, duplicate IDs, focus order, hover, motion, and others) and all other tools test the whole page whatever the scope, because any change can alter their verdicts. The act records data.scope (requested, applied, reason, roots, pathIDs, and for testaro the localRules and pageRules), so a reader can see what was and was not re-tested. The act is not scoped, and jobData.warnings says why, when there is no previous checkpoint, nothing changed, more than 50 subtrees changed, or the tool cannot restrict itself.

Checkpoint acts are optional. A job without one behaves as before: every test act tests checkpoint 0, and interaction acts before a test act only produce a warning in jobData.warnings. In a job with checkpoint acts, interaction acts followed by a test act without a checkpoint act produce an implicit checkpoint and a warning. A rule engine that tests a URL rather than a page (WAVE, and the Nu checkers with withSource) cannot test a state reached by interaction; such a test act is prevented with the reason. Details: docs/checkpoint-scanning.md.

Rule-engine details

The rule engines whose tests Testaro performs have particularities described below.

ASLint

The aslint rule engine makes use of the aslint-testaro fork of the aslint repository, which, unlike the published aslint package, contains the aslint.bundle.js file.

HTML CodeSniffer

The htmlcs rule engine makes use of the htmlcs/HTMLCS.js file. That file was created, and can be recreated if necessary, as follows:

1. Clone the HTML CodeSniffer package.
2. Make that package’s directory the active directory.
3. Install the HTML CodeSniffer dependencies by executing npm install.
4. Build the HTML CodeSniffer auditor by executing grunt build.
5. Copy the build/HTMLCS.js and build/licence.txt files into the htmlcs directory of Testaro.
6. Edit the Testaro copy of htmlcs/HTMLCS.js to produce the changes shown below.

The changes in htmlcs/HTMLCS.js are:

479a480
>     '4_1_2_attribute': 'attribute',
6482a6484
>     var messageStrings = new Set();
6496d6497
<         console.log('done');
6499d6499
<         console.log('done');
6500a6501
>       return Array.from(messageStrings);
6531c6532,6534
<       console.log('[HTMLCS] ' + typeName + '|' + msg.code + '|' + nodeName + '|' + elementId + '|' + msg.msg + '|' + html);
---
>       messageStrings.add(
>         typeName + '|' + msg.code + '|' + nodeName + '|' + elementId + '|' + msg.msg + '|' + html
>       );

Accessibility Checker

The ibm tests require the aceconfig.js file.

As of 2 March 2023 (version 3.1.45 of accessibility-checker), the ibm rule engine threw errors when hosted under the Windows operating system. To prevent these errors, it was possible to edit two files in the accessibility-checker package as follows:

In node_modules/accessibility-checker/lib/ACEngineManager.js, remove or comment out these lines starting on line 169:

if (nodePath.charAt(0) !== '/') {
    nodePath = "../../" + nodePath;
}

In node_modules/accessibility-checker/lib/reporters/ACReporterJSON.js, add these lines starting on line 106, immediately before the line var resultsFileName = pathLib.join(resultDir, results.label + '.json');:

// Replace the colons in the label with hyphen-minuses.
results.label = results.label.replace(/:/g, '-');

These changes were proposed as pull requests 1333 and 1334.

The ibm rule engine is one of two rule engines (testaro is the other) with a withItems property. If you set withItems to false, the result includes the counts of “violations” and “recommendations”, but no information about the rules that gave rise to them.

Nu Html Checker

The nuVal and nuVnu rule engines perform the tests of the Nu Html Checker. The nuVal rule engine is a remote service with an API. The nuVnu rule engine is installed as a dependency. A job can choose either one, or can try nuVal and if it fails then invoke nuVnu.

Its rules argument is not an array of rule IDs, but instead is an array of rule specifications. A rule specification for nuVal or nuVnu is a string with the format =ruleID or ~ruleID. The = prefix indicates that the rule ID is invariable. The ~ prefix indicates that the rule ID is variable, in which case the ruleID part of the specification is a matching regular expression, rather than the exact text of a message. This rules format arises from the fact that nuVal and nuVnu generate customized messages and do not accompany them with rule identifiers.

Pour Engine

The pour rule engine makes use of the pour/pour.min.js file, a bundle that Testaro builds from the upstream pour-engine repository (MIT), because upstream publishes no distributable bundle. The pour/README.md file documents the build command and the pinned upstream version. Testaro injects the bundle into the page and runs every rule. Findings in the engine's violations bucket become standard instances with outcome failed; findings in its incomplete bucket become instances with outcome cantTell. The engine's passes, inapplicable, and manualReview buckets are tallied in the act's data property and never become instances. If the bundle is missing or fails to define its global, the act is reported as prevented.

QualWeb

The qualWeb rule engine performs the ACT rules, WCAG Techniques, and best-practices tests of QualWeb. Only failures and warnings are included in the report. The EARL report of QualWeb is not generated, because it is equivalent to the report of the ACT rules tests.

QualWeb allows specification of rules for 3 modules: act-rules, wcag-techniques, and best-practices. If you include a rules argument in a QualWeb test act, its value must be an array of 1, 2, or 3 strings. Any string in that array is a specification for one of these modules. The string has this format:

'mod:m,n,o,p,…'

In that format:

• Replace mod with act, wcag, or best.
• Replace m, n, o, p, etc. with the 0 or more integers that identify rules.

For example, 'best:6,11' would specify that QualWeb is to test for best-practices rules QW-BP6 and QW-BP11, but not for any other best-practices rules.

When a string contains only a module prefix and no integers, such as best:, it specifies that the module is not to be run at all.

When no string pertains to a module, then QualWeb will test for all of the rules in that module.

Thus, when the rules argument is omitted, QualWeb will test for all of the rules in all of these modules.

The target can be provided to QualWeb either as HTML or as a URL. Experience indicates that the results can differ between these methods, with each method reporting some rule violations or some instances that the other method does not report. For at least some cases, more rules are reported violated when HTML is provided (withNewItems: false).

QualWeb creates sandboxed Playwright pages to perform its tests on. Therefore, the host must permit sandboxed browsers to be launched. See the discussion above about browser security.

SureA11y

The surea11y rule engine makes use of the surea11y/surea11y.browser.js file, the standalone browser bundle that @surea11y/core publishes, vendored verbatim. That package is licensed under the Mozilla Public License 2.0, unlike the MIT-licensed or Apache-licensed rule engines; the vendored file must not be modified. The surea11y/README.md file documents the pinned upstream version and the result shape. Testaro injects the bundle into the page and runs every automatic rule. The engine reports an outcome per rule (pass, fail, cantTell, or notApplicable) and may grade individual occurrences of a fail rule as cantTell. Occurrences graded fail become standard instances with outcome failed; occurrences graded cantTell become instances with outcome cantTell, carrying the engine's uncertainty code and needed guidance when it supplies them. Rules of type manual and rules with outcome pass or notApplicable are tallied in the act's data property and never become instances.

Testaro

The rules that Testaro can test for are implemented in files within the testaro directory.

The Testaro rules are classified by an allRules array defined in the tests/testaro.ts file. Each item in that array is an object with these properties:

• id: the rule ID.
• what: a description of the rule.
• contaminates: whether the test for the rule modifies the page, requiring the next test to launch a new browser for test isolation
• needsAccessibleName: whether the rule requires an added script adding an accessible-name computation method to window
• timeOut: the maximum time in seconds allowed for a test of the rule
• defaultOn: whether the rule is to be tested for by default

If you do not specify rules when using the testaro rule engine, Testaro will test for its default rules, in the order in which they appear in the array.

The optional rules argument for a testaro test act is an array whose first item is either 'y' or 'n' and whose remaining items are rule IDs. If 'y', then only the specified rules’ tests are performed. If 'n', then all the default rules are tested for, except for the specified rules.

The testaro rule engine (like the ibm rule engine) has a withItems property. If you set it to false, the standardResult object will contain an instances property with summaries that identify issues and instance counts. If you set it to true, some of the instances will be itemized.

Unlike any other rule engine, the testaro rule engine requires a stopOnFail property, which specifies whether a failure to conform to any rule (i.e. any value of totals other than [0, 0, 0, 0]) should terminate the execution of tests for the remaining rules.

Tests of the testaro tests (i.e. validation) could previously be performed as documented in the VALIDATION.md file. This functionality has broken and its redesign is planned.

One Testaro rule, allCaps, is currently being tested for in part with the assistance of the Claude Haiku artificial intelligence (AI) model. To obtain that assistance, you need an Anthropic API key, and its value must be assigned to the ANTHROPIC_API_KEY environment variable in the .env file. If no valid API key is set there, the rule will be tested for, but without AI assistance.

WAVE

If a wave test act is included in the job, the WAVE tests will be performed either by the subscription API or by the stand-alone API.

If you want the subscription API to perform the tests, you must get a WAVE API key from WebAIM and assign it as the value of an environment variable named WAVE_KEY. The subscription API does not accept a transmitted document for testing. WAVE must be given only a URL, which it then visits to perform its tests. Therefore, you cannot manipulate a page and then have WAVE test it, or ask WAVE to test a page that cannot be reached directly with a URL.

If you want the stand-alone API to perform the tests, you need to have that API installed and running, and the wave test act needs to define the URL of your stand-alone API. The test act can also define a prescript script and/or a postscript script.

Contribution

You can define additional Testaro rules and functionality. Contributions are welcome.

Please report any issues, including feature requests, at the repository.

Accessibility principles

The rationales motivating the Testaro-defined tests can be found in comments within the files of those tests, in the testaro directory. Unavoidably, each test is opinionated. Testaro itself, however, can accommodate other tests representing different opinions. Testaro is intended to be neutral with respect to questions such as the criteria for accessibility, the severities of accessibility defects, whether accessibility is binary or graded, and the distinction between usability and accessibility.

Challenges

Abnormal termination

On some occasions a test throws an error that cannot be handled with a try-catch structure. It has been observed, for example, that the ibm test does this when the page content, rather than the page URL, is given to getCompliance() and the target is https://globalsolutions.org, https://monsido.com, or https://www.ambetterhealth.com/.

Some rule engines take apparently infinite time to perform their tests on some pages. One website whose pages prevent 5 of the rule engines from ever completing their tests is the site of BrowserStack.

To handle such fatal errors and stalls, Testaro runs the tests of each rule engine in a separate forked child process that executes the procs/doTestAct.js module. The parent process subjects each rule engine to a time limit and kills the child if the time limit expires.

Activation

Testing to determine what happens when a control or link is activated is straightforward, except in the context of a comprehensive set of tests of a single page. There, activating a control or link can change the page or navigate away from it, interfering with the remaining planned tests of the page.

The Playwright “Receives Events” actionability check does not check whether an event is dispatched on an element. It checks only whether a click on the location of the element makes the element the target of that click, rather than some other element occupying the same location.

Test prevention

Test targets employ mechanisms to prevent scraping, multiple requests within a short time, automated form submission, and other automated actions. These mechanisms may interfere with testing. When a test act is prevented, Testaro reports this prevention.

Some targets prohibit the execution of alien scripts unless the client can demonstrate that it is the requester of the page. Failure to provide that evidence results in the script being blocked and an error message being logged, saying “Refused to execute a script because its hash, its nonce, or unsafe-inline does not appear in the script-src directive of the Content Security Policy”. This mechanism affects rule engines that insert scripts into a target in order to test it. To comply with this requirement, Testaro obtains a nonce from the response that serves the target. Then the file that runs the rule engine adds that nonce to the script as the value of a nonce attribute when it inserts its script into the target.

Some targets have been found erratically to prevent the creation of page images. When page images have been created, during the motion test in testaro some targets have been found to prevent their comparison by BlazeDiff, but comparison by pixelmatch has succeeded. For this reason, although reportedly slower, pixelmatch is the library used for image comparison.

Rule-engine duplicativity

Rule engines sometimes do redundant testing, in that two or more rule engines test for the same defects, although such duplications are not necessarily perfect. This fact creates problems:

• One cannot be confident in excluding some tests of some rule engines on the assumption that they perfectly duplicate tests of other rule engines.
• The Testaro report from a job documents each rule engine’s results separately, so a single defect may be documented in multiple locations within the report, making the direct consumption of the report inefficient.
• An effort to aggregate the results into a single score may distort the scores by inflating the weights of defects that happen to be discovered by multiple rule engines.
• Rule engines use different methods for identifying the locations of elements that violate rule-engine rules.

Rule-engine malfunctions

Rule engines can become faulty. For example, Alfa stopped reporting any rule violations in mid-April 2024 and resumed doing so at the end of April. In some cases, such as this, the rule-engine maker corrects the fault. In others, the rule engine changes and forces Testaro to change its handling of the rule engine.

Dependency deployment

The behavior of Testaro as a dependency of an application deployed on a virtual private server has been observed to be vulnerable to slower performance and more frequent test preventions than when Testaro is deployed as a stand-alone application on a workstation. The configuration of Testaro has been tuned for mitigation of such behaviors.

Containerized deployment

A reference container image for stand-alone deployment, in which all rule engines run, is defined by the Dockerfile and docker-compose.yml files at the project root and documented in CONTAINERS.md.

Headless browser fidelity

Testaro normally performs tests with headless browsers. Some experiments appear to have shown that some test results are inaccurate with headless browsers, but this has not been replicated. The launch function in the run module accepts a headEmulation argument with 'high' and 'low' values. Its purpose is to permit optimizations of headless browsers to be turned off, so browsers behave and appear more similar to headed browsers. Observation has failed to show any performance cost, so 'high' is the default value.

Repository exclusions

Any files in the temp or tmp directory are presumed ephemeral and are not tracked by git. Jobs create temporary files in subdirectories of tmp and delete those subdirectories on termination.

Related work

Testilo

Testilo is an application that:

• converts lists of targets and lists of issues into jobs
• produces scores and adds them to the raw reports of Testaro
• produces human-oriented HTML digests from scored reports
• produces human-oriented HTML comparisons of the scores of targets

Testilo contains procedures that reorganize report data by issue and by element, rather than rule engine, and that compensate for duplicative tests when computing scores.

Report standardization could be performed by other software rather than by Testaro. That would require sending the original reports to the server. They are typically larger than standardized reports. Whenever users want only standardized reports, the fact that Testaro standardizes them eliminates the need to send the original reports anywhere.

Kilotest

Kilotest is an application that offers a simplified interface to Testaro. At present it is deployed as a public service.

Code style

The JavaScript code in this project generally conforms to the ESLint configuration file .eslintrc.json. However, the htmlcs/HTMLCS.js file implements an older version of JavaScript. Its style is regulated by the htmlcs/.eslintrc.json file.

History

Work on the testaro tests in this package began in 2017, and work on the multi-package ensemble that Testaro implements began in early 2018. These two aspects were combined into an “Autotest” package in early 2021 and into the more single-purpose packages, Testaro and Testilo, in January 2022.

On 12 February 2024 ownership of the Testaro repository was transfered from the personal account of contributor Jonathan Pool to the organization account cvs-health of CVS Health. The MIT license of the repository did not change, but the copyright holder changed to CVS Health.

Maintenance of the repository owned by CVS Health came to an end on 30 September 2025. The current repository was forked from the cvs-health repository in October 2025 and then unlinked from the fork network, by agreement with CVS Health.

Version change notes

These notes cover versions from 73.0.0 (May 2026) onward. The date of each version is its npm publication date, except that versions marked “unpublished” were committed but not published to npm, and their changes first reached npm in the next published version. Changes in earlier versions are recorded in the commit history.

78

• 78.4.0 (2 September 2026). The first npm publication since 78.0.8; it includes the changes of the unpublished versions 78.1.0 through 78.3.0. Deployment options, each a job property when set, else an environment variable, else the earlier behavior: the load state, time limit, and fail-fast-on-4xx policy of navigation (navigation; NAV_WAIT_UNTIL, NAV_TIMEOUT, NAV_FAIL_FAST_4XX), a scanner identity request header (scannerId; SCANNER_ID), a full-page scroll before testing (scroll; PRESCAN_SCROLL), a branded Chromium channel (browserChannel), launch retries per rule of the testaro tool (act retries; TESTARO_RULE_RETRIES), and stealth and ad blocking for the qualWeb tool (act stealth and adBlock; QUALWEB_STEALTH, QUALWEB_ADBLOCK), which the QualWeb driver had ignored and which are now provided by playwright-extra and @ghostery/adblocker-playwright. A GitHub Actions workflow publishes the package to npm when a GitHub release is created whose tag matches the package version.
• 78.3.0 (2 September 2026, unpublished). Checkpoints (see Checkpoints and docs/checkpoint-scanning.md): a checkpoint act snapshots a page state reached by the acts of a job, and test acts test the latest checkpoint, reaching it by replaying the recorded interaction acts; an isolation job property (process, browser, or page) governs how test acts are isolated; when a job has two or more checkpoints, the report gains a flow property listing the issues added, persisted, removed, and not retested between consecutive checkpoints, with catalog-structure and ARIA-snapshot diffs; a test act may have scope: 'changed' to test only the subtrees changed since the previous checkpoint; interaction acts may identify elements by a Playwright selector; catalog entries not cited by any test act are pruned incrementally; procs/userPath.js converts recorded user paths into acts. Repairs to the act loop (next, failed launches, url, state).
• 78.2.0 (1 September 2026, unpublished). Two new rule engines, pour (Pour Engine 1.37.0) and surea11y (SureA11y core 1.7.0), each vendored as a browser bundle, bringing the count of rule engines from 10 to 12. All rule validators pass again (47 had been quarantined in 78.0.8), legacy expectation paths are resolved from the catalog, and a harness in validation/act/ scores engines against the W3C ACT test cases. Dependency updates.
• 78.1.0 (1 September 2026, unpublished). Certainty on standard instances: every instance carries outcome (failed or cantTell), optionally uncertainty and needed, and the standard result gains outcomeTotals; ordinalSeverity and totals are unchanged, and a later major version will redefine ordinalSeverity as impact only (see docs/standard-result-outcome.md). TypeScript: the rule modules, tool adapters, core procs, and types.ts are TypeScript sources whose emitted .js and .d.ts files are committed, the package declares types, a generated registry (testaro/registry.ts) maps rule IDs to rule modules, and the typecheck, build:ts, build:registry, and lint scripts are added. The required Node.js version becomes >=22.12, and dependencies are pinned to caret ranges instead of *. Polling a server (netWatch): authentication by a Basic authorization header (NETWATCH_AUTH_TYPE, NETWATCH_WORKER_ID, NETWATCH_WORKER_SECRET), job and report URLs no longer assume fixed paths, TLS certificates are verified by default, HTTP statuses are checked, and reports whose delivery fails are saved under REPORTDIR/netWatchFailed; the AGENT and NETWATCH_URL_AUTH variables are deprecated but still honored. Fixes: ordinalSeverity is numeric in summary instances of doTest rules, getBasicResult awaits the catalog index, styleDiff chooses the first list link deterministically, htmlcs leaves standard totals empty when standard results are off, ed11y reports a prevented run instead of crashing, browserClose abandons a close that has not settled after 10 seconds, the elements rule loads again (a duplicate parameter name had made it a syntax error), and executionTimeStamp is validated. Repository URLs point to YRA-Tech/testaro, and CI runs the linter, the type check, and the rule validators.
• 78.0.8 (20 August 2026). Copyright and license notices updated; a nonstandard directory removed.
• 78.0.7 (unpublished). getAttributeXPath tolerates Nu Html Checker messages without an extract, which had thrown and lost the standardization of the whole act.
• 78.0.6 (20 August 2026). nuVal and nuVnu: the extract of a message that identifies no element is preserved in the what property of the instance.
• 78.0.5 (20 August 2026). htmlcs: the define, exports, and module globals are hidden while HTMLCS.js is injected, so pages that expose an AMD loader no longer prevent the tool.
• 78.0.4 (19 August 2026). focAll reports the specific elements not reached by tabbing and those reached unexpectedly, itemized when withItems is true.
• 78.0.3 (unpublished). The accessibility-checker dependency is required to be at least 4.0.29.
• 78.0.2 (19 August 2026). browserClose actually closes the browser context and browser; previously every browser outlived its act.
• 78.0.1 (19 August 2026). The time budget of the motion rule rises from 5 to 30 seconds.
• 78.0.0 (19 August 2026). New optional imageScale job property: when it is greater than 1, the catalog page is rendered at that device scale factor and a second page image, at device-pixel scale, becomes the second item of images; the first item stays at CSS-pixel scale. The major version was incremented because of the added job property.

77

• 77.2.1 (19 August 2026). The catalog records no boxID for elements that are laid out but not painted (visibility: hidden and content-visibility: hidden subtrees).
• 77.2.0 (19 August 2026). During a job the map from path IDs to catalog indexes lives at report.pathIDs instead of catalog.pathID, so the catalog contains only element entries; the shipped report is unchanged.
• 77.1.0 (19 August 2026). Closed details elements are expanded before the page image and the box measurements are made, so the two agree.
• 77.0.4 (18 August 2026). Validation: expectations of test acts are resolved against act.result, so standardResult expectation paths work.
• 77.0.3 (18 August 2026). normalizeURL resolves relative path segments, so strict jobs with file: targets no longer report spurious bad redirections.
• 77.0.2 (18 August 2026). deSlash is defined, so strict jobs no longer abort on their first navigation.
• 77.0.1 (18 August 2026). autocomplete: its label arrays are spliced correctly, so the name, given-name, family-name, and email heuristics fire. The order in which navigation load-wait conditions are relaxed is improved. Dependency updates.
• 77.0.0 (28 July 2026). Observability: doJob accepts a second argument with an onProgress callback and emits jobStart, catalogStart, catalogEnd, actStart, actEnd, and jobEnd events (see Execution with observability).

76

• 76.2.2 (28 July 2026). Container versioning requirements documented; dependency updates.
• 76.2.1 (28 July 2026). The catalog no longer fails on pages whose scripts extend Array.prototype.
• 76.2.0 (27 July 2026). When a target repeatedly rejects a job, the job's browserID is changed.
• 76.1.2 (27 July 2026). The launch context honors a user agent supplied by the job for device emulation instead of overriding it.
• 76.1.1 (26 July 2026). Corrections related to containerization.
• 76.1.0 (25 July 2026). The qualWeb tool uses the QualWeb Playwright driver instead of Puppeteer. Documentation and dependency updates.
• 76.0.0 (18 July 2026). Containerized deployment: the TESTARO_CHROMIUM_NO_SANDBOX environment variable and a reference container image (Dockerfile, docker-compose.yml, CONTAINERS.md). Archived jobs and reports are named by ID in run and dirWatch, and a dirWatch file-deletion bug is corrected. The validation/watch directory is removed.

75

• 75.2.2 (18 July 2026). The call commands tolerate absent netWatch environment variables instead of failing.
• 75.2.1 (20 June 2026). A failure to launch a browser for the catalog is fatal to the job.
• 75.2.0 (20 June 2026). When a browser crashes on launch, the launch is retried with a different browser type, and browser-type changes are logged.
• 75.1.0 (6 June 2026). New optional imageColor job property: when a catalog is created, a page image with that color type is made the first item of an images array in the report. The motion rule makes its own screenshot and compares it with that image, using pixelmatch instead of BlazeDiff.
• 75.0.0 (5 June 2026). Breaking: the shoot act creates a full-page screenshot as a base64-encoded PNG and, with its exclusionSelector, colorType, and action properties, decides what to mask, which color type to use, and whether to return the image in the act result, add it to the images array of the report, or save it as a file; the screenshot option of the launch act is removed.

74

• 74.2.3 (3 June 2026). axe: the full data-xpath of each flagged element is resolved from the live DOM, because axe truncates the HTML it reports. Dependency updates.
• 74.2.2 (29 May 2026). Additions to UPGRADES.md.
• 74.2.1 (26 May 2026). WebKit and Firefox are imported from playwright-extra again; 74.2.0 had imported only Chromium.
• 74.2.0 (25 May 2026). The stealth plugin is applied only to Chromium; WebKit and Firefox had rejected a Chromium-only launch option and failed every job.
• 74.1.3 (25 May 2026). The TESTARO_NU_URL environment variable overrides the URL of the Nu Html Checker, for a self-hosted checker without the body-size limit of the public one.
• 74.1.2 (25 May 2026). nuVal survives empty responses from the W3C validator instead of losing the underlying error.
• 74.1.1 (25 May 2026). The exclude-list form of the testaro rule specification (['n', …]) actually excludes the listed rules.
• 74.1.0 (25 May 2026). Job-level stealth toggle.
• 74.0.1 (25 May 2026). Copyright notices updated.
• 74.0.0 (25 May 2026). New shoot act type, making full-page screenshots.

73

• 73.0.0 (25 May 2026). Temporary files are kept in a per-job directory under tmp at the project root, falling back to the operating system's temporary directory; the TMPDIRNAME environment variable and procs/config.js are removed.

Contributing

From 12 February 2024 through 30 September 2025, contributors of code to Testaro executed a CVS Health OSS Project Contributor License Agreement for Testaro before any pull request was approved and merged.

Future work

Future work contemplated for this project is described in its issues and also discussed in the UPGRADES.md file.

Etymology

“Testaro” means “collection of tests” in Esperanto.

License

© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved. © 2026 Jeff Witt. © 2025–2026 Jonathan Robert Pool.

Licensed under the MIT License. See LICENSE file at the project root for details.

SPDX-License-Identifier: MIT