Skip to content

[v18.x backport] test_runner features #46839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MoLow wants to merge 20 commits into from
Closed

[v18.x backport] test_runner features #46839

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
9ec0a4e
test_runner: avoid swallowing of asynchronously thrown errors
fossamagna Nov 7, 2022
7201845
test_runner: add reporters
MoLow Dec 19, 2022
a34809a
module: move test reporter loading
GeoffreyBooth Dec 22, 2022
b628161
test_runner: report `file` in test runner events
MoLow Jan 2, 2023
6d9ec6c
test_runner: make built in reporters internal
cjihrig Jan 6, 2023
6cb8f02
test_runner: add initial code coverage support
cjihrig Dec 21, 2022
8ad7668
doc,lib,src,test: rename --test-coverage
cjihrig Jan 8, 2023
95250e1
test: fix tap escaping with and without --test
pulkit-30 Jan 28, 2023
e1f5a3a
test_runner: top-level diagnostics not ommited when running with --test
pulkit-30 Feb 2, 2023
f4264f0
test_runner: fix missing test diagnostics
MoLow Feb 2, 2023
9c49554
test: fix tap parser fails if a test logs a number
pulkit-30 Feb 6, 2023
1502aca
test_runner: allow nesting test within describe
MoLow Feb 17, 2023
f7d8bed
test_runner: flatten TAP output when running using `--test`
MoLow Feb 18, 2023
730f11f
test_runner: parse non-ascii character correctly
mertcanaltin Feb 18, 2023
4a63d24
test_runner: display skipped tests in spec reporter output
richiemccoll Feb 18, 2023
32a0802
test_runner: centralize CLI option handling
cjihrig Feb 20, 2023
b50cfb3
test_runner: emit test-only diagnostic warning
richiemccoll Feb 21, 2023
69100c9
test_runner: bootstrap reporters before running tests
MoLow Feb 19, 2023
6ef4886
test_runner: add `describe.only` and `it.only` shorthands
richiemccoll Feb 21, 2023
b7aefcb
test_runner: better handle async bootstrap errors
cjihrig Feb 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test_runner: display skipped tests in spec reporter output 
PR-URL: #46651
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
  • Loading branch information
@richiemccoll @MoLow
richiemccoll authored and MoLow committed Feb 25, 2023
commit 4a63d24c8b9f2c56398da474e3c72ff4dd8719a2
14 changes: 10 additions & 4 deletions lib/internal/test_runner/reporter/spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ const symbols = {
'test:pass': '\u2714 ',
'test:diagnostic': '\u2139 ',
'arrow:right': '\u25B6 ',
'hyphen:minus': '\uFE63 ',
};
class SpecReporter extends Transform {
#stack = [];
Expand Down Expand Up @@ -60,8 +61,8 @@ class SpecReporter extends Transform {
return `\n${indent} ${message}\n`;
}
#handleEvent({ type, data }) {
const color = colors[type] ?? white;
const symbol = symbols[type] ?? ' ';
let color = colors[type] ?? white;
let symbol = symbols[type] ?? ' ';

switch (type) {
case 'test:fail':
Expand All @@ -81,15 +82,20 @@ class SpecReporter extends Transform {
ArrayPrototypeUnshift(this.#reported, msg);
prefix += `${this.#indent(msg.nesting)}${symbols['arrow:right']}${msg.name}\n`;
}
const skippedSubtest = subtest && data.skip && data.skip !== undefined;
const indent = this.#indent(data.nesting);
const duration_ms = data.details?.duration_ms ? ` ${gray}(${data.details.duration_ms}ms)${white}` : '';
const title = `${data.name}${duration_ms}`;
const title = `${data.name}${duration_ms}${skippedSubtest ? ' # SKIP' : ''}`;
if (this.#reported[0] && this.#reported[0].nesting === data.nesting && this.#reported[0].name === data.name) {
// If this test has had children - it was already reporter, so slightly modify the output
// If this test has had children - it was already reported, so slightly modify the output
ArrayPrototypeShift(this.#reported);
return `${prefix}${indent}${color}${symbols['arrow:right']}${white}${title}\n\n`;
}
const error = this.#formatError(data.details?.error, indent);
if (skippedSubtest) {
color = gray;
symbol = symbols['hyphen:minus'];
}
return `${prefix}${indent}${color}${symbol}${title}${error}${white}\n`;
}
case 'test:start':
Expand Down
20 changes: 10 additions & 10 deletions test/message/test_runner_output_spec_reporter.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
*
*

sync skip pass (*ms)
sync skip pass with message (*ms)
sync skip pass (*ms) # SKIP
sync skip pass with message (*ms) # SKIP
sync pass (*ms)
this test should pass
sync throw fail (*ms)
Expand All @@ -34,7 +34,7 @@
*
*

async skip pass (*ms)
async skip pass (*ms) # SKIP
async pass (*ms)
async throw fail (*ms)
Error: thrown from async throw fail
Expand All @@ -46,7 +46,7 @@
*
*

async skip fail (*ms)
async skip fail (*ms) # SKIP
Error: thrown from async throw fail
*
*
Expand Down Expand Up @@ -129,8 +129,8 @@
top level (*ms)

invalid subtest - pass but subtest fails (*ms)
sync skip option (*ms)
sync skip option with message (*ms)
sync skip option (*ms) # SKIP
sync skip option with message (*ms) # SKIP
sync skip option is false fail (*ms)
Error: this should be executed
*
Expand All @@ -146,13 +146,13 @@
<anonymous> (*ms)
test with only a name provided (*ms)
<anonymous> (*ms)
<anonymous> (*ms)
test with a name and options provided (*ms)
functionAndOptions (*ms)
<anonymous> (*ms) # SKIP
test with a name and options provided (*ms) # SKIP
functionAndOptions (*ms) # SKIP
escaped description \ # *
*
(*ms)
escaped skip message (*ms)
escaped skip message (*ms) # SKIP
escaped todo message (*ms)
escaped diagnostic (*ms)
#diagnostic
Expand Down