Skip to content

fix(gh,github,junit): attach test:log to the test that emitted it - #275

Merged
MoLow merged 2 commits into
mainfrom
fix/buffer-test-log-ordering
Aug 3, 2026
Merged

MoLow merged 2 commits into
mainfrom
fix/buffer-test-log-ordering

Conversation

@MoLow

@MoLow MoLow commented Aug 3, 2026

Copy link
Copy Markdown
Owner

test:log is execution-ordered: Node forwards it the moment t.log() runs, bypassing the per-file declaration-order buffer that test:diagnostic waits in (kExecutionOrderedEvents). gh, github and junit structure their output by test, so rendering a log on arrival files it under whichever test happens to be printing.

Two concurrent files, @reporters/gh under Actions — before:

A1 log before
B1 log                     <- file B's log, before A reports at all
A1 log after
::group::✔ A one
A1 diagnostic              <- correct, Node ordered it
A2 log                     <- A two's log, inside A one's group
::endgroup::
::group::✔ A two
B2 log late                <- B two's log, inside A two's group
::endgroup::

after:

::group::✔ A one
A1 log before
A1 log after
A1 diagnostic
::endgroup::
::group::✔ A two
A2 log
::endgroup::

@reporters/junit had it worse — the misplacement is in the document, not just the display:

<!-- A1 log before -->
<!-- B1 log -->
<testcase name="A one" .../>
<!-- A2 log -->
<testcase name="A two" .../>

What changed

Each log is held keyed by (entryFile ?? file, testId) and emitted when its owner reports — the position test:diagnostic already occupies.

  • @reporters/github — new log_buffer.js, exported as @reporters/github/log_buffer. Annotations flush at test:pass/test:fail, including the subtestsFailed path that emits no annotation of its own.
  • @reporters/gh — its own buffer instance; flushed lines land after the group header. Buffers in and out of GitHub Actions so local output matches CI.
  • @reporters/junit — its own copy of the map, so the package stays dependency-free. Logs attach after the testcase/testsuite decision, so a logged test can't become a suite.

Logs whose test never reports — an interrupted run, or a t.log() that outlived its test — are emitted at the end rather than dropped.

Two bugs fixed in the same code path: a log with no location crashed @reporters/github in extractLocation (Cannot read properties of undefined (reading 'startsWith')), and the log's data payload was dropped from the annotation (gh rendered it, github didn't).

Not changed

live, web and testwatch stream their output — real-time logging is what test:log exists for, and they're the reporters built for it.

junit logs stay XML comments. <system-out> would be more useful to CI parsers, but it's a format change, and comments are deliberately excluded from the nonCommentChildren count that decides testcase vs testsuite.

Known limitation

Under process isolation, (file, testId) collides across processes when tests are declared in a shared helper file — the same defect already tracked for the tree store. Reading entryFile first makes this correct automatically once nodejs/node#64309 lands.

Upstream

Node's built-in junit and tap reporters have the identical defect; a fix is in progress separately. spec is left live by design.

Verification

436 tests pass, 100% statements/lines, branch coverage 95.38% → 95.79%, lint clean. New regression tests in all three packages cover: a log flushing inside its owner's group, a sibling's log not leaking into a foreign group, cross-file interleaving, multiple logs keeping their order, an orphan drained at end of run, gh rendering a log exactly once (it wraps upstream spec, which would double-print if the event were forwarded), and github tolerating a location-less log.

Also verified end-to-end against real Node v26.6.0 runs — nested, concurrent, in and out of Actions — and the suite re-run with GITHUB_ACTIONS=1 to exercise the CI paths.

🤖 Generated with Claude Code

`test:log` is execution-ordered: Node forwards it the moment `t.log()` runs,
bypassing the per-file declaration-order buffer that `test:diagnostic` waits
in (`kExecutionOrderedEvents` in lib/internal/test_runner/runner.js). These
three reporters structure their output by test, so rendering a log on arrival
files it under whichever test happens to be printing.

Two concurrent files, before:

    A1 log before
    B1 log                     <- file B's log, before A reports at all
    A1 log after
    ::group::✔ A one
    A1 diagnostic              <- correct, Node ordered it
    A2 log                     <- A two's log, inside A one's group
    ::endgroup::
    ::group::✔ A two
    B2 log late                <- B two's log, inside A two's group
    ::endgroup::

junit was worse: the misplacement is in the document, not just the display.

Hold each log until its owner reports, keyed by (entryFile ?? file, testId),
and emit it with that test - the position `test:diagnostic` already occupies.
gh and github share a LogBuffer from `@reporters/github/log_buffer`; junit
keeps its own copy so the package stays dependency-free. gh buffers in and out
of GitHub Actions, so local output matches CI. Logs whose test never reports,
from an interrupted run or a `t.log()` that outlived its test, are emitted at
the end rather than dropped.

Two bugs fixed in the same path: a log with no location crashed the github
reporter in `extractLocation`, and its `data` payload was dropped from the
annotation.

`live`, `web` and `testwatch` are unchanged - real-time output is what
`test:log` exists for, and they are the reporters built for it.
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.83%. Comparing base (d2ba502) to head (fe15bac).

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #275    +/-   ##
========================================
  Coverage   99.83%   99.83%            
========================================
  Files          40       41     +1     
  Lines        4129     4294   +165     
  Branches      604      604            
========================================
+ Hits         4122     4287   +165     
  Misses          5        5            
  Partials        2        2            
Flag Coverage Δ
v22 99.58% <100.00%> (+0.01%) ⬆️
v24 99.53% <100.00%> (+0.01%) ⬆️
v26 99.53% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The reporter reads GITHUB_ACTIONS once, at construction, so the drain test
covered whichever branch the environment selected: locally the plain one,
and in Actions - where CI runs - the grouped one, leaving the other two
lines uncovered. Pin the variable when loading the module instead.
@MoLow
MoLow merged commit 4ae4b04 into main Aug 3, 2026
6 checks passed
@pr-opener-bot pr-opener-bot Bot mentioned this pull request Aug 3, 2026
@MoLow
MoLow deleted the fix/buffer-test-log-ordering branch August 9, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant