Skip to content

fix: stream compose logs in post-job cleanup - #298

Draft
neilime with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-docker-compose-logs-issue
Draft

fix: stream compose logs in post-job cleanup#298
neilime with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-docker-compose-logs-issue

Conversation

Copilot AI commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Large docker compose logs output could crash the post-job hook before cleanup completed because log collection buffered the full command output in memory. As a result, docker compose down could be skipped when service logs were very large.

  • Post-job log collection

    • Replace the docker-compose package logs() helper with a direct docker compose logs process spawn.
    • Stream stdout/stderr chunks to the configured service logger instead of assembling a single in-memory string.
    • Remove the extra full-output re-log in post-runner, since logs are now emitted incrementally.
  • Cleanup resilience

    • Treat log collection failures as non-fatal in the post hook.
    • Continue to docker compose down even when log collection exits non-zero, is terminated by signal, or cannot be spawned.
  • Failure reporting

    • Return concise diagnostic messages for abnormal log-command termination without reintroducing output buffering.
    • Handle edge cases where the child process cannot provide output streams.
const childProcess = spawn("docker", commandArgs, { cwd });

childProcess.stdout.on("data", (chunk) => {
  serviceLogger(chunk.toString());
});

childProcess.stderr.on("data", (chunk) => {
  serviceLogger(chunk.toString());
});

Copilot AI linked an issue Aug 7, 2026 that may be closed by this pull request
5 tasks
Copilot AI changed the title [WIP] Fix issue with collecting docker compose logs output fix: stream compose logs in post-job cleanup Aug 7, 2026
Copilot AI requested a review from neilime August 7, 2026 09:28

@neilime neilime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot handle feedbacks

Comment thread src/post-runner.ts
);
}

loggerService.debug(`docker compose logs:\n${output}`);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, removing this is introducing a breaking change

Copilot AI commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

@copilot handle feedbacks

Addressed in 6100198. I restored the explicit docker compose logs: debug message in post-runner without bringing back buffered log aggregation, updated the related test, and revalidated with lint, build, test:ci, package, review, and CodeQL.

Copilot AI requested a review from neilime August 16, 2026 09:51
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 6100198 to 6f7e04a Compare August 18, 2026 13:03
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Fail ❌
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Fail ❌
TRIVY Pass ✅

Super-linter detected linting errors

For more information, see the GitHub Actions workflow run

Powered by Super-linter

BIOME_FORMAT
Checked 26 files in 179ms. No fixes applied.
Found 2 errors.src/services/docker-compose.service.test.ts format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Formatter would have printed the following content:

    381 381 │         const logsPromise = service.logs(logsInputs);
    382 382 │
    383     │ - ······expect(spawnMock).toHaveBeenCalledWith("docker",·[
    384     │ - ········"compose",
    385     │ - ········"-f",
    386     │ - ········"docker-compose.yml",
    387     │ - ········"logs",
    388     │ - ········"helloworld2",
    389     │ - ········"helloworld3",
    390     │ - ······],·{
    391     │ - ········cwd:·"/current/working/dir",
    392     │ - ······});
        383 │ + ······expect(spawnMock).toHaveBeenCalledWith(
        384 │ + ········"docker",
        385 │ + ········[
        386 │ + ··········"compose",
        387 │ + ··········"-f",
        388 │ + ··········"docker-compose.yml",
        389 │ + ··········"logs",
        390 │ + ··········"helloworld2",
        391 │ + ··········"helloworld3",
        392 │ + ········],
        393 │ + ········{
        394 │ + ··········cwd:·"/current/working/dir",
        395 │ + ········},
        396 │ + ······);
    393 397 │
    394 398 │         stdout.emit("data", Buffer.from("logs"));
    ······· │
    421 425 │         const logsPromise = service.logs(logsInputs);
    422 426 │
    423     │ - ······expect(spawnMock).toHaveBeenCalledWith("docker",·[
    424     │ - ········"--context",
    425     │ - ········"dev",
    426     │ - ········"compose",
    427     │ - ········"--profile",
    428     │ - ········"ci",
    429     │ - ········"-f",
    430     │ - ········"docker-compose.yml",
    431     │ - ········"logs",
    432     │ - ······],·{
    433     │ - ········cwd:·"/current/working/dir",
    434     │ - ······});
        427 │ + ······expect(spawnMock).toHaveBeenCalledWith(
        428 │ + ········"docker",
        429 │ + ········[
        430 │ + ··········"--context",
        431 │ + ··········"dev",
        432 │ + ··········"compose",
        433 │ + ··········"--profile",
        434 │ + ··········"ci",
        435 │ + ··········"-f",
        436 │ + ··········"docker-compose.yml",
        437 │ + ··········"logs",
        438 │ + ········],
        439 │ + ········{
        440 │ + ··········cwd:·"/current/working/dir",
        441 │ + ········},
        442 │ + ······);
    435 443 │
    436 444 │         childProcess.emit("close", 1);
    ······· │
    517 525 │
    518 526 │         await expect(service.logs(logsInputs)).resolves.toEqual({
    519     │ - ········error:·"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
        527 │ + ········error:
        528 │ + ··········"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
    520 529 │           output: "",
    521 530 │         });


src/services/docker-compose.service.ts format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Formatter would have printed the following content:

     86  86 │           settled = true;
     87  87 │           resolve({
     88     │ - ··········error:·"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
         88 │ + ··········error:
         89 │ + ············"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
     89  90 │             output: "",
     90  91 │           });
    ······· │
    104 105 │           settled = true;
    105 106 │           resolve({
    106     │ - ··········error:
    107     │ - ············signal
    108     │ - ··············?·`Docker·Compose·logs·command·failed·with·signal·${signal}`
    109     │ - ··············:·exitCode·!==·null·&&·exitCode·!==·0
    110     │ - ················?·`Docker·Compose·logs·command·failed·with·exit·code·${exitCode}`
    111     │ - ················:·"",
        107 │ + ··········error:·signal
        108 │ + ············?·`Docker·Compose·logs·command·failed·with·signal·${signal}`
        109 │ + ············:·exitCode·!==·null·&&·exitCode·!==·0
        110 │ + ··············?·`Docker·Compose·logs·command·failed·with·exit·code·${exitCode}`
        111 │ + ··············:·"",
    112 112 │             output: "",
    113 113 │           });


format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Some errors were emitted while running checks.

SPELL_CODESPELL
/github/workspace/dist/index.js:748: ECT ==> ETC
/github/workspace/dist/index.js:790: TE ==> THE, BE, WE, TO
/github/workspace/dist/index.js:12049: contructor ==> constructor
/github/workspace/dist/index.js:19691: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:19698: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:21152: alse ==> also, else, false
/github/workspace/dist/index.js:21625: alse ==> also, else, false
/github/workspace/dist/index.js:22267: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:22269: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:28705: fpr ==> for, far, fps
/github/workspace/dist/index.js:28712: fpr ==> for, far, fps
/github/workspace/dist/index.js:30336: boolen ==> boolean
/github/workspace/dist/index.js:30421: collapsable ==> collapsible
/github/workspace/dist/index.js:30424: collapsable ==> collapsible
/github/workspace/dist/post.js:748: ECT ==> ETC
/github/workspace/dist/post.js:790: TE ==> THE, BE, WE, TO
/github/workspace/dist/post.js:12049: contructor ==> constructor
/github/workspace/dist/post.js:19691: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:19698: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:21152: alse ==> also, else, false
/github/workspace/dist/post.js:21625: alse ==> also, else, false
/github/workspace/dist/post.js:22267: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:22269: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:26885: boolen ==> boolean
/github/workspace/dist/post.js:26970: collapsable ==> collapsible
/github/workspace/dist/post.js:26973: collapsable ==> collapsible

@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 6f7e04a to 05978b5 Compare August 18, 2026 13:14
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.07692% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.45%. Comparing base (e56276d) to head (6f330cd).

Files with missing lines Patch % Lines
src/services/docker-compose.service.ts 71.11% 6 Missing and 7 partials ⚠️
src/post-runner.ts 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##              main     #298      +/-   ##
===========================================
- Coverage   100.00%   93.45%   -6.55%     
===========================================
  Files            9        9              
  Lines          173      214      +41     
  Branches        37       50      +13     
===========================================
+ Hits           173      200      +27     
- Misses           0        6       +6     
- Partials         0        8       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI and others added 2 commits August 18, 2026 15:39
Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 05978b5 to 5e62815 Compare August 18, 2026 13:39
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
Signed-off-by: Emilien Escalle <emilien.escalle@escemi.com>
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 5e62815 to 6f330cd Compare August 18, 2026 14:29
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue with collecting docker compose logs output in post job cleanup

2 participants