Post reports as a single PR comment by gohabereg · Pull Request #151 · editor-js/document-model

gohabereg · 2026-05-31T19:16:24Z

Now only a single comment with test reports generated and updated for all workflows. Some minor workflow changes as well. Coverage comparison would be enabled after merge as base branch doesn't generate summary at the moment

Suggestions on the format are welcome.
Current format for a package section (warning are displayed conditionally):

<package-name>

Unit tests report

<number> tests passed in <number> suites.
Branches coverage: <number>%

Caution

Coverage for <category> dropped by <number>%
Coverage for <another-category> dropped by <number>%

Mutation tests report

Mutation tests run with mutation score <number>%.
Survived mutants: <number>
Not covered mutants: <number>

Caution

Mutation score is below the low threshold of <number>%

Warning

Mutation score is below the low threshold of <number>%

Dashboard URL: <URL>

This reverts commit 8904eb5.

…tion

Copilot

Pull request overview

This PR updates the CI pipelines to generate unit-test coverage + mutation-test artifacts per workflow and aggregate them into a single, continuously-updated PR comment (one section per package/workflow), while also adding workflow concurrency controls and bumping multiple GitHub Actions versions.

Changes:

Generate/upload Jest JSON + coverage summary artifacts (head + base) and Stryker JSON artifacts for later aggregation.
Add an aggregate-report composite action plus scripts to assemble Markdown and update a single PR comment.
Add concurrency to multiple workflows and introduce a new report job that posts/updates the aggregated comment.

Reviewed changes

Copilot reviewed 34 out of 35 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
packages/sdk/jest.config.ts	Adds coverage reporters needed for summary extraction.
packages/ot-server/package.json	Updates coverage script to emit Jest JSON output.
packages/ot-server/jest.config.ts	Adds coverage reporters for JSON summary + text summary.
packages/model/stryker.conf.mjs	Adds JSON reporter for mutation results.
packages/model/package.json	Updates coverage script to emit Jest JSON output.
packages/model/jest.config.ts	Adds coverage reporters for JSON summary + text summary.
packages/dom-adapters/package.json	Updates coverage script to emit Jest JSON output (currently with trailing whitespace bug).
packages/dom-adapters/jest.config.ts	Adds coverage reporters for JSON summary + text summary.
packages/core/stryker.conf.mjs	Adds JSON reporter for mutation results.
packages/core/package.json	Updates coverage script to emit Jest JSON output.
packages/core/jest.config.ts	Adds coverage reporters for JSON summary + text summary.
packages/collaboration-manager/package.json	Updates coverage script to emit Jest JSON output.
packages/collaboration-manager/jest.config.ts	Adds coverage reporters for JSON summary + text summary.
.gitignore	Ignores generated `jest-report.json`.
.github/workflows/sdk.yml	Adds concurrency + bumps checkout action version.
.github/workflows/playground.yml	Adds concurrency + bumps checkout action version.
.github/workflows/ot-server.yml	Adds concurrency, bumps checkout version, adds `report` job (needs PR-only gating).
.github/workflows/npm-publish.yml	Bumps checkout/setup-node action versions.
.github/workflows/model.yml	Adds concurrency, bumps checkout version, adds `report` job (needs PR-only gating).
.github/workflows/dom-adapters.yml	Adds concurrency, bumps checkout version, adds `report` job (needs PR-only gating; mutation JSON reporter mismatch).
.github/workflows/core.yml	Adds concurrency, bumps checkout version, adds `report` job.
.github/workflows/collaboration-manager.yml	Adds concurrency, bumps checkout version, adds `report` job (needs PR-only gating).
.github/workflows/build-and-push-docker-image.yml	Bumps checkout action version.
.github/scripts/update-aggregated-comment.js	Implements “create/update single aggregated PR comment” logic (export format currently incompatible with require).
.github/scripts/process-mutation.js	Parses mutation JSON + changed-files list into Markdown (module format + logic issues).
.github/scripts/generate-report.js	Generates per-package Markdown section (export format currently incompatible with require).
.github/scripts/compare-coverage.js	Compares head/base coverage + extracts Jest counts into Markdown (module format + formatting issues).
.github/actions/unit-tests/action.yml	Runs coverage, uploads artifacts, triggers base coverage (needs PR-only gating).
.github/actions/setup/action.yml	Bumps cache action version.
.github/actions/mutation-tests-changed-files/action.yml	Uploads mutation artifacts and changed-files list for aggregation.
.github/actions/mutation-tests-all-files/action.yml	Bumps setup-node/changed-files versions.
.github/actions/lint/action.yml	Bumps setup-node version.
.github/actions/build/action.yml	Bumps setup-node version.
.github/actions/base-coverage/action.yml	New composite action to run coverage on base ref and upload artifacts.
.github/actions/aggregate-report/action.yml	New composite action to download artifacts, build Markdown, and update the aggregated PR comment (coverage path handling needed).

Comments suppressed due to low confidence (1)

.github/scripts/compare-coverage.js:118

After removing the ESM export keyword, processReports still needs to be exported for the require(...).processReports call site. Add a CommonJS export at the end of the file.

  return message;
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    "lint:fix": "yarn lint --fix",
    "test": "jest",
-    "test:coverage": "yarn test --coverage=true",
+    "test:coverage": "yarn test --coverage=true --json --outputFile=jest-report.json ",


+  const score = (killed + timedOut) / (killed + timedOut + notCovered + survived).toFixed(2);
+
+  return {
+    total,
+    score,
+    survived,
+    notCovered,
+    thresholds: obj.thresholds,
+  };


+    - name: Run base coverage action
+      uses: ./.github/actions/base-coverage
+      with:
+        package-name: ${{ inputs.package-name }}
        working-directory: ${{ inputs.working-directory }}


+  let message = `${tests.passedTests} tests passed in ${tests.passedSuites} suites.\n`
+
+  message += `Branches coverage: ${categories.branches.head}%\n`;
+
+  let warning = '';
+
+  for (const cat in categories) {
+    if (categories[cat].delta < 0) {
+      warning += `Coverage for ${cat} dropped by ${categories[cat].delta.toFixed(2)}%\n`;
+    }
+  }
+
+  if (warning.length > 0) {
+    message += `> [!WARNING]\n> ${message}\n`;
+  }


+  if (!fs.existsSync(reportFile) && !fs.existsSync(changedFilesFile)) {
+    return '';
+  }
+  const changedFiles = JSON.parse(fs.readFileSync(changedFilesFile, 'utf8'));
+
+  if (changedFiles.length === 0) {
+    return 'No files to mutate found.';
+  }
+
+  const raw = fs.readFileSync(reportFile, 'utf8');
+  const obj = JSON.parse(raw);
+


+  let message = `Mutation tests run with mutation score ${(metrics.score * 100).toFixed(2)}%.\n`;
+
+  if (metrics.survived) {
+    message += `Survived mutants: ${metrics.survived}\n`;
+  }
+
+  if (metrics.notCovered) {
+    message += `Not covered mutants: ${metrics.notCovered}\n`;
+  }
+
+  if (metrics.score < metrics.thresholds.low) {
+    message += `> [!WARNING]\n> Mutation score is below the low threshold of ${metrics.thresholds.low}%\n`;
+  } else if (metrics.score < metrics.thresholds.high) {
+    message += `> [!CAUTION]\n> Mutation score is below the high threshold of ${metrics.thresholds.low}%\n`;
+  }


neSpecc · 2026-06-01T14:29:31Z

@editorjs/model

Unit tests report

563 tests passed in 26 suites. Branches coverage: 98.43%

Mutation tests report

No files to mutate found.

@editorjs/core

Unit tests report

143 tests passed in 8 suites. Branches coverage: 91.3%

Mutation tests report

No files to mutate found.

@editorjs/collaboration-manager

Unit tests report

130 tests passed in 7 suites. Branches coverage: 85.81%

@editorjs/ot-server

Unit tests report

4 tests passed in 1 suites. Branches coverage: 20%

@editorjs/dom-adapters

Unit tests report

11 tests passed in 2 suites. Branches coverage: 86.95%

Mutation tests report

No files to mutate found.

Why do we need to see tests result in a comment when we can see it in Checks section?
It will show reports for all packages every time? Or only for those which was affected in a PR?

gohabereg · 2026-06-01T15:05:49Z

We can remove test results, the main info is coverage anyway. Maybe instead we can include coverage on changed or added files
For now it shows all packages every time. I am not sure how we can selectively run the workflows and guarantee there won't be integration problems. Will look into it

…e or dependencies have changed

neSpecc · 2026-06-01T17:12:23Z

We can remove test results, the main info is coverage anyway. Maybe instead we can include coverage on changed or added files

For now it shows all packages every time. I am not sure how we can selectively run the workflows and guarantee there won't be integration problems. Will look into it

maybe it would be better to display the result as a simple table

package	coverage	coverage change
@editorjs/model	98.43%	increased
@editorjs/core	99.11%	—
@editorjs/ot-server	80.12%	⚠️ decreased

e11sy · 2026-06-01T16:38:57Z

+
+    - name: Find current PR's number
+      uses: jwalton/gh-find-current-pr@v1
+      id: findPr


Suggested change

id: findPr

id: find_pr

for ids consistency if possible

e11sy · 2026-06-01T16:46:44Z

+  if (fs.existsSync(report)) {
+    try {
+      return JSON.parse(fs.readFileSync(report, 'utf8'));
+    } catch (report) {
+      // continue
+    }
+  }
+


would be nice if eslint checked this scripts too, but for now i would suggest logging of the report or explicitly dismissing with void report

e11sy · 2026-06-01T16:49:10Z

+  } catch (e) {
+    // ignore parse errors
+  }


same, dismiss or log

e11sy · 2026-06-01T16:50:55Z

+// prefer head counts; if missing, fallback to base counts; otherwise null
+  const tests = {


Suggested change

// prefer head counts; if missing, fallback to base counts; otherwise null

const tests = {

// prefer head counts; if missing, fallback to base counts; otherwise null

const tests = {

e11sy · 2026-06-01T16:53:58Z

+
+  for (const cat in categories) {
+    if (categories[cat].delta < 0) {
+      warning += `\n> Coverage for ${cat} dropped by ${categories[cat].delta.toFixed(2)}%\n`;


if delta is 'N/A' — it has no toFixed() method, otherwise delta is already .toFixed(2)

e11sy · 2026-06-01T17:04:10Z

+  elif git rev-parse --verify "origin/master" >/dev/null 2>&1; then
+    RESOLVED_BASE_REF="origin/master"
+    debug "Resolved base ref as: origin/master"
+  elif git rev-parse --verify "master" >/dev/null 2>&1; then
+    RESOLVED_BASE_REF="master"
+    debug "Resolved base ref as: master"


redundant

Suggested change

elif git rev-parse --verify "origin/master" >/dev/null 2>&1; then

RESOLVED_BASE_REF="origin/master"

debug "Resolved base ref as: origin/master"

elif git rev-parse --verify "master" >/dev/null 2>&1; then

RESOLVED_BASE_REF="master"

debug "Resolved base ref as: master"

e11sy · 2026-06-01T17:11:05Z

+  let md = '';
+
+  md += `<!-- ${pkg} REPORT -->\n`;
+  md += `## ${pkg}\n\n`;


lets start with separator

Suggested change

md += `## ${pkg}\n\n`;

md += '---'

md += `## ${pkg}\n\n`;

github-actions · 2026-06-01T18:51:09Z

Unit Tests

Package	Coverage	Delta
@editorjs/dom-adapters	86.95%	N/A
@editorjs/core	91.3%	N/A
@editorjs/ot-server	20%	N/A
@editorjs/model	98.43%	N/A
@editorjs/collaboration-manager	85.81%	N/A

Mutation Tests

Package	Mutation score	Dashboard URL
@editorjs/dom-adapters	No files to mutate found.
@editorjs/core	No files to mutate found.
@editorjs/model	No files to mutate found.

gohabereg and others added 30 commits May 31, 2026 20:16

Post reports as a single PR comment

2ce87da

Merge branch 'main' into chore/pr-reporting

d8edf3a

update all actions

88f1dd4

Remove core require

c175508

build packages for base branch test run

8b117ee

Replace require with import

8904eb5

Revert "Replace require with import"

2f21c6c

This reverts commit 8904eb5.

Replace require only inside js scripts

6642696

Fixes

1898afa

Remove unused code

1b629a0

Pass core and github as parameters

f6382d4

Remove require core and github

6834c3e

revert github require

1cca3ed

Pass context separately

e962bcc

Remove turnstyle initial wait, update cache action

7910489

Add reporting for unit tests to each package, update changed-files ac…

d8bf2c1

…tion

Use coverage json-summary reporter

9197c9b

Upload mutation artifact even if no files changed

ff2acc8

Send message about files not found if the files list is empty

7aa89ef

Add concurrency

00051da

Test coverage drop and mutation tests report

acb8efd

Add at least one test

e41ee6e

fixes

b3be451

update files list formatting

3865c3c

Update artifact paths

389e2b5

Test coverage drop

d5e785f

Always run report, add dashboard url to mutation report

7b9bb7e

fix base branch tests checkout

952d1e8

Test dashboard url

eca9bfe

Add json report for dom-adapters

72a3a1c

gohabereg added 2 commits June 1, 2026 14:55

Mutation score to percent conversion

9da66ec

Switch warning and caution

1685a8c

gohabereg marked this pull request as ready for review June 1, 2026 13:59

gohabereg requested a review from Copilot June 1, 2026 13:59

Copilot started reviewing on behalf of gohabereg June 1, 2026 13:59 View session

Copilot AI reviewed Jun 1, 2026

View reviewed changes

resolve review comments

b30e1fd

gohabereg added 4 commits June 1, 2026 17:34

Extract duplication to a common workflow. Run workflow only if packag…

488bf4c

…e or dependencies have changed

resolve base ref for shallow clone

8f2da60

Update script

fbe7bd8

Update path to script

9533fee

e11sy reviewed Jun 1, 2026

View reviewed changes

Remove some lint warnings

0d24791

editor-js deleted a comment from github-actions Bot Jun 1, 2026

gohabereg added 3 commits June 1, 2026 19:25

Update format

962b1b0

Fix typo

a08ce4a

Get info from computed data

d0bb75a

editor-js deleted a comment from github-actions Bot Jun 1, 2026

gohabereg added 2 commits June 1, 2026 19:42

Add pipes at the end of the rows

6461226

Remove empty lines between rows

1515928

editor-js deleted a comment from github-actions Bot Jun 1, 2026

gohabereg added 2 commits June 1, 2026 19:52

test updates and mutation report

c13ff85

Revert test

605d146

neSpecc approved these changes Jun 1, 2026

View reviewed changes

		// prefer head counts; if missing, fallback to base counts; otherwise null
		const tests = {

Conversation

gohabereg commented May 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

<package-name>

Unit tests report

Mutation tests report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

neSpecc commented Jun 1, 2026

@editorjs/model

Unit tests report

Mutation tests report

@editorjs/core

Unit tests report

Mutation tests report

@editorjs/collaboration-manager

Unit tests report

@editorjs/ot-server

Unit tests report

@editorjs/dom-adapters

Unit tests report

Mutation tests report

Uh oh!

gohabereg commented Jun 1, 2026

Uh oh!

neSpecc commented Jun 1, 2026

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

e11sy Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Tests

Mutation Tests

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gohabereg commented May 31, 2026 •

edited

Loading

github-actions Bot commented Jun 1, 2026 •

edited

Loading