Skip to content

Tokenization: give the stylesheet a token layer, zero visual change (refs #439) - #455

Merged
tylervick merged 1 commit into
mainfrom
tylervick/style-tokens-439
Aug 13, 2026
Merged

tylervick merged 1 commit into
mainfrom
tylervick/style-tokens-439

Conversation

@tylervick

Copy link
Copy Markdown
Member

Refs #439 (option C direction — this is the token layer the C-refresh lands on). Zero visual change beyond the () removal below.

What

  1. 33 CSS custom properties on :root, at their current values, grouped and commented (brand / text / status / surfaces / borders / typography / spacing / component sizes). Every colour literal in the stylesheet now lives in the token block — #1780FA--color-accent, red--status-failed, the grays split by role (text vs border vs surface). The "SF Pro Display" stack is pinned exactly as-is (--font-family-base); the font fix is C-refresh scope. The px type scale (11/12/13/16/17/20), font weights (400/500), icon sizes (10/14/24), the preview height (600px), the one border-radius, and the two spacing steps that actually repeat (4px, 10px) are tokens too.
  2. ([[TIME]]) removed from the activity template — the empty () on every activity row left behind when 5b emptied the duration (the migration's logged follow-up, now that Delete stale HTML/ copy — HTMLTemplates.swift is the template source of truth (#434) #452 settled the template source of truth). The dead "TIME": "" supply in Activity.swift is gone with it (its only consumer).

Judgment calls (one-offs left inline)

  • Structural one-offs stay literal: header height 70px, sidebar widths 200/400px, screenshot-flow/tail heights 200/350px, indent offsets (22/28/52px, 24px padding-left), resizer geometry (6px, −4px), 1px border widths, 0.8em, and the 14×11px preview-icon (non-square, so not coupled to --icon-size).
  • #DDD/#ddd (same colour, two spellings) collapse into one --color-border-light; white/#FFF/#FFFFFF split semantically into --color-surface vs --color-on-accent.
  • No new tokens for future values, no dark mode.

The gate (byte-honest, one fixture generation)

BaselineCaptureTests + XCHR_BASELINE_DIR, before vs after, all three fixtures. The diff contains only the two permitted classes, enumerated:

Fixture <style> lines (−/+) activity rows losing exactly () any other line
TestResults −92/+146 47 0
SanityResults −92/+146 4 0
RetryResults −92/+146 23 0

The +54 style-line delta is the :root block itself. Value round-trip proven mechanically: substituting every var() back with its definition and dropping :root reproduces the old stylesheet — all 103 rules value-identical (modulo colour spelling #ddd#DDD and one width:24pxwidth: 24px colon-spacing normalization). Every var() has a definition; no definition is unused.

Tests

  • Legacy leg: swift test — 100 tests, 0 failures (2 env-gated skips)
  • Forced-modern leg: XCHR_RESULT_READER=modern swift test — 100 tests, 0 failures
  • DifferentialTests 6/6 — both backends share the markup, parity confirmed, not assumed

Hook note

First real consumer of the template-aware pre-commit hook (#452): a mixed commit (exempt HTMLTemplates.swift + regular Activity.swift) worked with zero friction — swiftformat skipped the template, linted the Swift file.

🤖 Generated with Claude Code

… (refs #439)

Every repeated or semantic hardcoded value in the report stylesheet moves
into a grouped, commented :root block — colours (incl. #1780FA), the
SF Pro Display font stack (pinned exactly as-is), the px type scale,
font weights, icon sizes, and the two spacing steps that repeat (4px,
10px). Rules reference them via var(); a retheme now edits one block.

Also removes the ([[TIME]]) placeholder from the activity template — the
stray empty parens on every activity row left when 5b emptied the
duration — and the now-dead "TIME": "" supply in Activity.swift, its
only consumer.

Gate (BaselineCaptureTests, one fixture generation, all three fixtures):
the before/after render diff contains only <style> lines (-92/+146 in
each fixture) and activity rows losing exactly " ()" (47/4/23 rows in
TestResults/SanityResults/RetryResults). A var()-substitution round-trip
proves all 103 CSS rules value-identical; every token is defined and
used. Suite green on both backend legs (100 tests, 0 failures each);
DifferentialTests 6/6.

Co-Authored-By: Claude Fable 5 <[email protected]>
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@tylervick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 16 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 23a027f2-05e9-4029-af57-65bca3560166

📥 Commits

Reviewing files that changed from the base of the PR and between 8a8a63b and 1b04a06.

📒 Files selected for processing (2)
  • Sources/XCTestHTMLReportCore/Classes/HTMLTemplates.swift
  • Sources/XCTestHTMLReportCore/Classes/Models/Activity.swift

Comment @coderabbitai help to get the list of available commands.

@tylervick
tylervick merged commit fdbe78b into main Aug 13, 2026
8 checks passed
@tylervick
tylervick deleted the tylervick/style-tokens-439 branch August 13, 2026 08:33
tylervick added a commit that referenced this pull request Aug 13, 2026
…identity title (refs #439) (#456)

* Theme the token layer: dark mode, WCAG floors, system font, run-identity title (refs #439)

Part 1 of 2 of the Option C conservative refresh. Today's three-pane
identity is kept; this is the theme half only, and the structural half
(responsive collapse, status icons for mixed/unknown, attachment labels)
is deliberately left to PR 2.

Every selector stays shared between light and dark — the dark block
overrides token *values* and nothing else, which is exactly what
tokenizing in #455 bought.

- viewport meta, `color-scheme: light dark`, and an explicit background
  on html/body. `body` previously set no background at all, so the white
  canvas was a browser default — the thing that made dark mode
  impossible to bolt on.
- system-ui font stack replacing the pinned "SF Pro Display", with a
  generic family last so the chain always terminates.
- Contrast: 11 failing fg/bg pairs go to 0 in both themes, verified by
  computing WCAG 2.1 ratios for every pairing the changed tokens
  produce. `#1780FA` -> `#1163CC`, bare `red` -> `#D70015`, `#777`
  pills -> `#555`, and the placeholder off `#AAA`. White-on-hover-tint
  was the worst at 1.55:1 and now uses primary text.
- `--color-accent-text` splits the blue's two roles. One value cannot
  serve as both a fill under white text and as text on a surface once
  dark mode exists.
- The tab title carries run identity instead of the constant
  `XCHTMLReport`. It derives from the bundle *file name* — an input
  path, never anything read out of a bundle — so both readers emit the
  same bytes and the differential holds by construction.
- `.toolbar`/`.table-header` take min-height rather than a fixed
  height. The viewport meta stops phones rendering this page zoomed out
  at desktop width, which had been hiding the pills overlapping the
  table header once they wrap. Containment only; the narrow-screen
  layout is PR 2.
- Corrects the false spacing rationale carried over from the #455
  review: 2px and 6px each repeat four times, so "the only two steps
  that repeat" was not why they were left inline.

Both legs green (auto and modern), DifferentialTests included. No test
needed modifying.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Unclamp header height too (CodeRabbit on #456)

The review's stated failure does not occur: the pills that wrap live in
.tests-header, not in <header>, which holds only the title band and the
two-item Tests/Logs row.

The underlying point stands though. 70px is about 4px of slack over those
two rows, so browser text scaling clipped the header, and leaving it
clamped is inconsistent with the sibling toolbars this branch already
moved to min-height. Geometry is unchanged at 1440: header contents,
both sidebars and the first summary row all keep their previous heights
and offsets.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Lift the dark selection fill to #2170D6 so it clears 3:1 on every backdrop (review on #456)

#1E65C2 cleared the 3:1 UI floor against --color-surface (3.18) but not
against --color-bg-sidebar (2.76), where the default-selected device card
actually sits, nor --color-bg-group-header (2.86). The PR's contrast table
listed only the surface pairing, which is how the gap went unnoticed.

#2170D6 keeps the hue (214deg) and saturation (73%) and raises lightness
43.9% -> 48.4%, clearing every backdrop while white-on-fill stays over the
4.5:1 text floor:

  fill vs sidebar     #232327   2.76 -> 3.25
  fill vs surface     #161619   3.18 -> 3.74
  fill vs group hdr   #202024   2.86 -> 3.37
  white text on fill            5.68 -> 4.82

One token still covers every role. The selected device card carries white
text on the fill, so the 3:1 and the 4.5:1 floors bind on the same element
and a separate --color-accent-fill token could not relax either of them. At
this hue no value clears both by more than ~7%; this one sits at the centre
of that window.

Verified by re-deriving the ratios from the template source, by a dark DOM
walk over all three fixtures at 1440 and 375 (374 text samples, 0 failures),
and by sampling the painted pixels back out of the renders (#2170D6 on
#232327 = 3.25). The dark block is still tokens-only: one :root, 18 custom
properties, no structural declarations.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
tylervick added a commit that referenced this pull request Aug 13, 2026
…ix status glyphs, type-derived attachment labels (refs #439) (#459)

* C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439)

The structural half of option C, on top of the theme half (#456) and the
token layer (#455). Desktop at 1024 and 1440 is unchanged to the pixel:
every layout box and every row box measures identical to the pre-change
render, and the only geometry that moves is the two expected-failure rows
gaining the status icon they never had.

Responsive collapse. One `@media (max-width: 700px)` block at the end of
the sheet; nothing above it changed. The device sidebar becomes a
horizontal strip, and the attachment pane a bottom sheet that is in the
layout only while an attachment is selected. The DOM is untouched, so the
filter and collapse scripts run the same code in both layouts; the only
script change is two lines toggling an `attachment-open` body class from
the functions that already tracked exactly that state.

SVG icons. Every glyph is a one-colour SVG applied as a CSS mask, coloured
by `background-color` underneath — a data-URI image cannot see the token
layer, which is why the sheet used to ship a second white copy of four
PNGs just for selected rows. `currentColor` now covers that for free.
`.bundle-icon` deleted: 26KB of base64 no markup ever referenced. A
linking-mode TestResults report goes 453,337 -> 93,963 bytes (-79.3%);
the same absolute saving applies inline.

Status glyphs. `Status` gains `expectedFailure`, and `unknown` gains a
class instead of the empty string. All six states get a distinct shape,
not just a colour, at a 3:1 non-text contrast floor in both themes. Both
readers already mapped xcresult's `Expected Failure`, so no backend moved
and the differential needed no allow-list change. Status selectors use
`>`: matched loosely, a mixed row's rule repainted the icons of the
iterations nested inside it.

Attachment labels. `kXCT…` names are classified internal by prefix rather
than by enumerating one constant, so `kXCTAttachmentScreenRecording` stops
reaching the UI and so does whatever Apple adds next. This removes a
divergence rather than adding one — legacy's internal name and modern's
nil name now both fall to the same type-derived "Video". Derivation reads
`type` alone, a fact both backends set the same way.

One test pin edited: `testExpectedFailureStillRendersAsUnknown` asserted
the flattening this change removes, and is rewritten to assert the state
survives to the renderer while neither backend moves.

Refs #439.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Propagate expectedFailure through the suite-row aggregate (refs #439)

`TestGroup.status` walks a fixed precedence list, and #439's new status was
missing from it: a suite whose every test is an expected failure fell
through to `.unknown` -- the one value that means "we could not tell" --
while every row inside it said otherwise. Added last in the list, so a
suite holding a real failure is still a failed suite.

Invisible today, because group rows draw no status glyph and no fixture
builds such a suite, which is also why nothing would have caught it. The
new StatusAggregationTests constructs the cases from crafted inputs
instead of waiting for a fixture to grow them, and pins the two boundaries
either side of the new entry: a real outcome still outranks an expected
failure, and an expected failure still keeps a suite off the all-passed
shortcut.

Raised by CodeRabbit on #459.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Fix the two review findings on the C-refresh structure half (refs #439)

F1 — `.device-identifier { display: none }` was dead below 700px. Inside the
same media query `#info-sections ul li` sets `display: inline`, and one ID
beats one class, so the field kept rendering: 299px wide with its right edge
at x=708 on a 375 viewport, scrolling the device strip 303px further than
designed and clipping `Model:` at the edge. Qualifying the selector with the
same ID wins the cascade. CDP `CSS.getMatchedStylesForNode` at 375 now
reports computed `display: none` (was `inline`) and the strip measures 346px
instead of 649px; at 701/1024/1440 no media rule applies and the row's box is
unchanged at [4,177,196,18].

F2 — `Status.unknown`'s new `"unknown"` class had neither a fixture nor a
test, the only mapping in the PR with no coverage of any kind. `Status` is
now `CaseIterable` so the new table test can assert it covers every case:
a status added without a `cssClass` fails the test instead of shipping a row
that draws no glyph.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
tylervick added a commit that referenced this pull request Aug 13, 2026
Nothing here has ever asserted anything about how the report looks. The
three existing comparisons -- DifferentialTests, ReproducibilityTests,
BaselineCaptureTests -- all compare one render to another render, and
none of them knows what any of it means. A token that resolves to
nothing, a dark-mode palette that fails contrast, a filter tab that
stops filtering: all pass silently.

That became load-bearing when #439 started redesigning the UI. #455
tokenized the stylesheet and claimed "zero visual change" with no test
able to confirm it; #456 adds dark mode and asserts WCAG floors. Those
are claims a browser checks mechanically and a reviewer cannot.

Three layers over one synthetic ParsedResult fixture: unit tests on
model logic, per-template HTML goldens, and Playwright for the facts
only a browser knows. Layers 1 and 2 add no toolchain and need neither
simulator nor .xcresult.

The design turns on a constraint BaselineCaptureTests already documents:
fixtures regenerate, so a golden keyed to one cannot be checked in. The
way out is to stop feeding renders from generated fixtures -- hence the
synthetic fixture, and hence the one production change, a Summary
initialiser taking pre-parsed runs.

Also records a defect found while writing this: TestScreenshotFlow
discards its tailCount parameter and hardcodes suffix(3).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
tylervick added a commit that referenced this pull request Aug 13, 2026
* Design: test coverage for the rendered report

Nothing here has ever asserted anything about how the report looks. The
three existing comparisons -- DifferentialTests, ReproducibilityTests,
BaselineCaptureTests -- all compare one render to another render, and
none of them knows what any of it means. A token that resolves to
nothing, a dark-mode palette that fails contrast, a filter tab that
stops filtering: all pass silently.

That became load-bearing when #439 started redesigning the UI. #455
tokenized the stylesheet and claimed "zero visual change" with no test
able to confirm it; #456 adds dark mode and asserts WCAG floors. Those
are claims a browser checks mechanically and a reviewer cannot.

Three layers over one synthetic ParsedResult fixture: unit tests on
model logic, per-template HTML goldens, and Playwright for the facts
only a browser knows. Layers 1 and 2 add no toolchain and need neither
simulator nor .xcresult.

The design turns on a constraint BaselineCaptureTests already documents:
fixtures regenerate, so a golden keyed to one cannot be checked in. The
way out is to stop feeding renders from generated fixtures -- hence the
synthetic fixture, and hence the one production change, a Summary
initialiser taking pre-parsed runs.

Also records a defect found while writing this: TestScreenshotFlow
discards its tailCount parameter and hardcodes suffix(3).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Plan: report visual test coverage, 13 tasks in three phases

Phase 1 builds the Swift foundation -- a StubPayloadProvider, a synthetic
ParsedResult covering every rendered status, and a Summary initialiser
taking pre-parsed runs. Phase 2 adds golden snapshots keyed to that
fixture rather than to a generated .xcresult, which is what makes them
committable at all. Phase 3 adds the browser layer.

Task 1 opens RED on the tailCount defect the spec recorded. Task 12
hands the finished suite to #456 and unskips the dark-mode assertion
there, which is the sequencing argument's payoff: the first claim about
how this report looks that a machine, not a reviewer, checks.

Three assertions in the plan are verified by deliberately breaking
something and confirming the failure -- an undeclared token reference, a
contrast floor, and a changed golden -- because an assertion that has
never failed is one nobody should trust.

Selectors are taken from HTMLTemplates.swift rather than guessed:
`.selected` is the live selection class, group rows sit under
`.run.active`, and the filter tabs are `<li>` elements carrying counts.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* TestScreenshotFlow honours tailCount instead of hardcoding 3

* Synthetic ParsedResult fixture covering every rendered state

* Fix PayloadProviding member count in the visual-test-coverage design spec

The spec called PayloadProviding a three-member protocol, but it declares
five: url, exportPayload, exportPayloadData, exportLogs, exportLogsData.
Task 2's SyntheticResultTests fixture is what surfaced the discrepancy.

* Deduplicate SyntheticResultTests' ParsedTestCase extraction, widen filename assertion

Review fix round 1: the three tests repeated the same 11-line
runs→testables→groups→ParsedNode extraction chain verbatim. Extracted a
private allTestCases() helper. Also widened
testCoversHostileAttachmentFilename to require all five hostile
characters the fixture filename actually carries ("'<>&), not just two
of them, so the assertion pins what the fixture provides.

* Summary initialiser taking pre-parsed runs, for fixture-free rendering

Adds an internal Summary.init(parsedRuns:payloads:renderingMode:
downsizeImagesEnabled:downsizeScaleFactor:faultCollector:bundleNames:),
alongside the existing public path-based initialiser, so tests can render
a full report page from SyntheticResult's fixture with no .xcresult and
no simulator. This is the injection point #391 made possible: ParsedRun
is now the boundary between reading and rendering, so tests can construct
one directly instead of reading it out of a bundle.

Internal, not public, because PayloadProviding is internal;
@testable import reaches it from tests while library consumers keep the
resultPaths-based initialiser untouched.

SummarySeamTests exercises the new seam: a full page renders
(<!doctype html>, the fixture's SyntheticSuite group, the :root token
layer), the complete synthetic tree produces zero faults, and two
renders of the same fixture are byte-identical.

* Give the synthetic fixture a real log reference, so Run.init? stops degrading

SyntheticResult.parsedRun passed logReference: nil, which sent Run.init?
down the else branch: a warning to the console, and logContent = .none —
no logs section at all in the rendered page. Every golden HTML file from
Task 4 onward would have pinned a report missing that pane, and
StubPayloadProvider.exportLogs/exportLogsData would have stayed dead code
no test ever reached. Fixing this now, before any goldens exist, is the
cheapest point to do it.

StubPayloadProvider gains a logText constant and real exportLogs/
exportLogsData implementations that mirror exportPayload/exportPayloadData:
resolve from the same exports map, touch no filesystem. SyntheticResult
registers a logReference in payloads and wires it into parsedRun.

SummarySeamTests.testRendersAFullPageWithoutAnXcresult now asserts the
logs iframe doesn't degrade to an empty src. A new test,
testInlineRenderingEmbedsTheActualLogBytes, proves the fixture's log
bytes reach the rendered page for real: under .inline rendering the
iframe src is a data: URI, so the exact base64 encoding of logText is
a verifiable substring of the output. .linking mode (used elsewhere in
this file) only yields a content-free relative file name, so it cannot
by itself prove real bytes made the trip — the .inline mode assertion is
required to confirm the fix's actual purpose, not just that the warning
went away.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* Snapshot harness and the index golden, keyed to the synthetic fixture

* Inline-mode golden alongside the linking-mode one

* Correct the no-fixtures verification claim: park only the .xcresult bundles

Parking the entire Resources directory also removes differential-allowlist.json,
the fourth Package.swift-declared resource. With zero resolvable resources
SwiftPM synthesizes no Bundle.module, breaking the whole test target's build —
a packaging artifact, not evidence the synthetic-fixture suites need .xcresult
fixtures. Parking only the three .xcresult bundles isolates the actual claim.

* Dump the synthetic render for the browser suite

* Playwright scaffold and token-resolution assertions

Introduces the browser test layer: a pinned Playwright + axe-core
devDependency set, chromium-only config with retries disabled, and two
assertions over the report's computed CSS custom properties — every
declared :root token resolves to a non-empty value, and no rule
references a token that was never declared.

Verified the second assertion actually bites: temporarily broke a
var(--color-text-primary) reference in HTMLTemplates.swift to
var(--color-text-nonexistent), re-dumped the fixture, and watched
`no rule references an undeclared token` fail naming the bogus token.
The template edit was reverted before committing.

* Recurse into grouping rules so dark-mode :root is not blind spot

Both CSS scanners in tokens.spec.ts filtered on `rule instanceof
CSSStyleRule`, which is false for CSSMediaRule — so the
@media (prefers-color-scheme: dark) :root block PR #456 added was
invisible to both assertions. A stylesheet with a token declared or
referenced only inside that block would pass "no rule references an
undeclared token" vacuously; the name was inaccurate.

Walk CSSGroupingRule.cssRules recursively (covers @media, @supports,
@layer alike) in both evaluate bodies. Verified the walk now visits 2
:root rules instead of 1, and that a var() reference broken *inside*
the dark block is caught post-fix but was silently missed by the
pre-fix scanner against the same fixture.

* WCAG contrast and dark-mode assertions, dark mode unskipped (#456 landed)

* axe-core runs without gating; all findings filed on #440

* Behavioural assertions for filters and keyboard navigation

* Run the browser assertions in CI

Adds a two-job workflow: a macOS job dumps the synthetic render fixtures
via VisualFixtureDumpTests (no simulator, no .xcresult, no
prepareTestResults.sh needed — Package.swift's differential-allowlist.json
resource is enough to synthesize Bundle.module), then an Ubuntu job
downloads those fixtures and runs the Playwright suite against them.

setup-node and download-artifact are pinned to their actual latest
releases (v7.0.0 and v8.0.1) rather than the older SHAs from the initial
draft; download-artifact's SHA matches the one already pinned in
release.yml.

* Fix effectiveBackground() to skip masked icon fills

The WCAG contrast test's effectiveBackground() walk treated any element's
non-transparent computed background-color as a real painted surface. The
#439 icon refresh (merged to main via #459, which this branch predates)
switched .preview-icon and friends to `mask-image` + `background-color:
currentColor` — the standard single-colour tintable-icon technique. That
background-color is clipped to the icon's silhouette by the mask; it is
never a rectangle behind text. For any such element, currentColor makes
the "background" trivially equal the "foreground" by construction, so the
walk found a spurious 1.00:1 wherever a masked icon happened to pick up
sampleable text.

That happened here because of a real (separate, out of scope for this
fix) HTML-escaping bug: the synthetic fixture's edge-case filename
containing a raw `"` breaks out of `data="[[FILENAME]]"`, leaking markup
as text children of the otherwise-empty .preview-icon span and an
unrecognized <greaterthan> element the broken parse produces. That leak is
real and made both elements eligible for sampling; skipping masked
elements in the background walk makes the walk find the same true
ancestor surface every other passing text element already clears against.

Diagnosed by downloading the exact fixture CI's dump job produced
(GH Actions run 31745292239) and reproducing the 1.00:1 locally against
that fixture with plain diagnostic instrumentation: this branch's own
fixture predates #459's icon refresh entirely, which is why "8/8 green
locally" and "2/8 red in CI" were both true — pull_request's default
checkout tests the PR merged with current main, not the branch alone.

* Fix undeclared-token scan to see shorthand var() references

rule.style iterated longhand-by-longhand: Chromium stores a shorthand
containing var() (e.g. border: 1px solid var(--color-border-strong)) as a
pending-substitution value, so every expanded longhand serialises to "".
Scanning rule.style.cssText instead sees the literal var(...) text
regardless of shorthand expansion.

Confirmed by mutation: before the fix the scan found 28 of 34 var()
references; after, all 34. Deleting --color-border-strong from
HTMLTemplates.swift's :root (both light and dark blocks) now makes the
test fail, naming the token; previously it stayed green.

* Fail a snapshot refresh run instead of reporting it green

The XCHR_UPDATE_SNAPSHOTS=1 branch wrote the golden and returned, so a
refresh run was unconditionally green — writing bytes proves nothing about
whether the new content is correct. XCTFail after a successful write so
CI can never pass while the env var is set, and a developer refreshing
locally is prompted to re-run without it for a real verdict.

* Rename a11y test to state what it actually checks

GATING_IMPACTS = [] (correctly, per #440) makes the assertion
expect([]).toEqual([]) — it cannot fail regardless of what axe finds, but
the old name ("report has no critical or serious accessibility
violations") claimed a guarantee the test does not provide. Renamed to
describe the gate's actual state and expanded the comment to record all
six known findings (image-alt critical x6, frame-title serious,
heading-order/landmark-one-main/region moderate, empty-heading minor) so
the debt is legible without re-running the suite.

* Add expected-failure regression test for hostile-filename escaping

SyntheticResultTests.testCoversHostileAttachmentFilename asserts the
fixture contains a filename with " ' < > & — but nothing asserted the
render escapes it, and it does not: HTMLTemplates.swift interpolates the
raw filename into onclick="showText('[[SOURCE]]')" unescaped, so the
embedded double quote breaks out of the attribute. The committed goldens
already contain this broken markup.

Adds a regression test wrapped in XCTExpectFailure recording the defect
(HTMLTemplates.swift / Attachment.swift, out of scope here) without
turning the suite red. The moment the escaping is fixed, XCTExpectFailure
flips this into a hard failure — an unexpectedly-passing expected
failure — which is the loud signal that the test should be promoted out
of the wrapper.

* Amend the design spec to match what actually shipped

Whole-branch review found five claims in the spec the delivered work does
not satisfy. Recorded as a dated Amendments section rather than editing
history: fixture attachment coverage (PNG/text only, not video/HTML),
behavioural assertions (2 of 3 shipped — no attachment-click-populates-
preview test), contrast pairing discovery (sampled from one static DOM
state, not the cascade; --color-text-secondary and --color-accent-soft
are never exercised as text colour in the fixture's default render), axe
moderate/minor reporting (console.log, not $GITHUB_STEP_SUMMARY), and
Sequencing (PR #456 merged first and is this branch's merge-base, so the
suite did not land before #456 as planned — #456's claims were verified
retroactively instead, and confirmed sound).

* Document the visual test layer in CONTRIBUTING.md

"CI runs exactly these two commands, so a green run locally means a green
run on your pull request" stopped being true once the Visual workflow
landed. Corrected the claim and documented the two layers CONTRIBUTING.md
was silent on: template snapshots (XCHR_UPDATE_SNAPSHOTS=1 to refresh,
and why a refresh run always fails) and the Playwright browser layer
(dump via XCHR_VISUAL_DIR swift test --filter VisualFixtureDumpTests,
then npm ci + npx playwright test in visual/).

* Drop the test-fixture default from Summary's seam initialiser

bundleNames: [String] = ["Synthetic"] put a test fixture's name into
production source as a default value. Made it a required parameter and
updated the three callers (SummarySeamTests, TemplateSnapshotTests,
VisualFixtureDumpTests) to pass "Synthetic" explicitly — the goldens'
<title> is unchanged because the value itself didn't change, only where
it's supplied from.

Also mirrored the path-based initialiser's identifier shape —
.appending("bundle\(index)").appending("action0") — instead of the
unrelated run-\(index), so the goldens pin an identifier set a real
report could actually produce. This does change the goldens: every
derived id/toggle/activities hash shifts with the path string. Regenerated
via XCHR_UPDATE_SNAPSHOTS=1 and diffed to confirm every changed line is
only a 32-hex-char identifier, nothing else.

* Exclude committed Snapshots goldens from the test target's resources

swift build --build-tests warned "found 2 file(s) which are unhandled"
for Tests/XCTestHTMLReportTests/Snapshots/*.html once those goldens
existed: SwiftPM saw them under the target's source directory but they
are read directly by path (SnapshotSupport.swift), not vended as
Bundle.module resources. exclude: ["Snapshots"] tells SwiftPM to leave
the directory alone.

* Regenerate goldens against PR #459's icon/structure refresh

Rebased onto origin/main (f442e8e), which merged #459 mid-execution of
this branch's plan: an icon refresh adding status-glyph and icon tokens
(SVG masks replacing base64 PNGs), a responsive @media (max-width: 700px)
breakpoint, and type-derived attachment labels. TemplateSnapshotTests
correctly caught the drift (Snapshot index/index-inline changed).

Refreshed with XCHR_UPDATE_SNAPSHOTS=1 swift test --filter
TemplateSnapshotTests (which itself fails on purpose per the FIX-2
change), then re-ran clean and read both goldens in full before
committing: well-formed, properly closed, the hostile-filename markup
this branch's expected-failure regression test depends on is still
present and unescaped.

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
tylervick added a commit that referenced this pull request Aug 15, 2026
The one literal A3b left outside it. The sheet has had zero hardcoded font
stacks since #455 gave it a token layer, and the log body had no business
being the exception.

Refs #439.
tylervick added a commit that referenced this pull request Aug 15, 2026
…ixes #460) (#486)

* Give every status a filter, and the log something to filter it with (fixes #460)

The last of #439's redesign PRs. A1 gave the report a summary header, A2 gave
the tree Xcode's outline, A3a gave each view its own surface and reserved a
slot in each toolbar. This fills the slots.

## The expected-failure bucket (#460)

The model has named `.expectedFailure` since #443 and A1's tally has counted
it in a bucket of its own since the header landed. The filter row was the last
reading of a run that still had it in no bucket at all: five functions, each
naming the four row classes it shows or hides, and an expected failure carries
none of them. So "Passed" left one on screen — a reader asking for the tests
that passed was shown one that did not — and "All" could not put it back,
because "All" only ever set `display: block` on those same four. The group
pass then read the row as hidden, so a suite of nothing but expected failures
vanished from the tree the moment the reader touched any filter, "All"
included.

**Decision: they leave `unknown` entirely, and nothing in the header moves.**
That is the whole point — A1 already put them where they belong, and the
coordination this needed was to make the toolbar agree rather than to move a
count. `CoreTests` has carried the disagreement as a comment since A1 ("the two
readings disagree until A3 rebuilds the filters"); it now asserts the partition
instead.

**The pills are the legend, made operable.** Both are rendered from the run's
`Tally`: same buckets, same order, same drop-the-empty-ones rule. A leading
"All", then one pill per outcome the run produced. That is what closes the
class of defect rather than the instance — `unknown` had exactly the same gap
and no issue filed against it, and it is filtered now for the same reason
`expectedFailure` is: a status gets a pill *because* the run produced it.

Three consequences worth stating, since each was a filter that did not finish:

- **A suite goes when its last visible row does.** The pass this replaces
  bailed out on any group holding a sub-group, so the legacy backend's two
  wrapper levels — "Selected tests" and "<target>.xctest" — survived every
  filter. Two selected rows sat under three empty headings. Deciding
  deepest-first covers both cases with one rule.
- **A filtered-out test takes its screenshot with it.** `.screenshot-tail` is
  emitted *between* rows rather than inside one (A2), so it was outside
  everything the filters touched.
- **A childless suite is a row.** `Run.allTests` counts one as a leaf — the
  shape a crashed target leaves behind, which `TruncationFaultTests` is built
  from — so the pills count it and now filter it.

## Tests and runs

Xcode's toolbar states 21 tests and 23 runs on `TestResults.xcresult`; ours
knew only tests. The difference is `parameterizedAddition(value:)`, which ran
once per argument set.

Neither `iterations` nor `arguments` could carry that: both readers
deliberately collapse argument executions into one iteration (that is what
makes the two backends agree on the rows), and only the modern format names
the argument values. The *count* is in both bundles — legacy has one sibling
metadata entry per execution, modern one `Arguments` child — and legacy was
throwing it away in `mergingArgumentExecutions`. `ParsedTestCase` gains
`executionCount`, so this converges the readers rather than reading anything
new, and `DifferentialTests.testExecutionCountsAgreeAcrossBackends` holds them
converged, with a non-vacuity check because equality is free if nothing ever
ran twice.

The toolbar reads `23 executions`, not "23 runs": this report already spends
the word "run" on a destination, which the picker labels `Run 1`, `Run 2`.
It is `aria-live`, because a filter changes it. Each row that accounts for a
difference says so — `3 arguments`, the option-A mockup's own tag — and only
the count is stated, never the values, which one backend cannot see.

No per-run rows in the tree: counts and filter semantics only.

## A name filter, in both views

Xcode puts a Filter field at the trailing end of both toolbars. So does this,
in the slot A3a reserved. On Tests it is a name-substring filter over the tree
that *composes* with the pills — "Failed" and "one" together mean the failing
tests whose names contain "one". On Logs it filters the log's lines.

**The log moves into the page to make that possible.** It was an
`<iframe src>` pointing at a `file://` sibling or a `data:` URI — a foreign
origin either way — and two things followed that the report shipped: nothing
in the page could filter, search or scroll it, which is why A3a's toolbar
carried an inert "All Messages" label where a control belongs; and it could
not see the token layer, so in dark mode the Logs view was a white slab with
black text, the same class of defect as the base64 status PNGs #459 replaced.
`logs-1440-dark.png` in the A3a evidence is that slab; the one here is the
same view themed.

A single `<pre>` rather than an element per line: a filter that rebuilds one
text node is a string join, where a six-figure log would otherwise be a
six-figure DOM. The `.log` export is untouched — it is the artifact #480 fixed
and `DifferentialLogTests` compares — and inline mode gets *smaller*, since
escaped text costs less than the same bytes base64'd into a URI.

## Substitution order, closed rather than ordered

`Run` fills its two templates from two ordered lists instead of one
dictionary. `HTML.html` reduces over a dictionary, so it fills placeholders in
hash order and fills the ones an earlier replacement *inserted* as readily as
the ones the template author wrote — the hazard the A3a review found in the
picker. Two per-view templates make it closable: each list holds only what its
own template needs, so a test named `[[LOG_TEXT]]` is not merely filled late
but unfillable, and a log line reading `[[TEST_SUMMARIES]]` likewise. Both
directions pinned in `PlaceholderOrderTests`.

## Verification

- `swift test` — 193 tests, 0 failures, 3 skipped, on both legs (default/auto
  and `XCHR_RESULT_READER=modern`).
- `DifferentialTests` green on both legs; the allow-list is untouched.
- `visual` — 48 Playwright tests, 0 failures: six axe states (two new — the
  filtered tree and the filtered log), contrast over both themes, both
  fixtures and now both *views*, the filter behaviour, and 375px.
- Masked duration shapes untouched.
- `swiftformat --lint` clean; no new SwiftLint warnings.

Refs #439.

* Match suite names in the tree filter too

A filter over a tree that only ever reads its leaves is one a reader finds out
about the hard way: typing the name of a suite they can see emptied the pane.
Matching an ancestor keeps that suite and the tests inside it, which is what
the same query does in Xcode's outline.

The status filter stays a leaf question. A suite's own status is folded from
its children, so filtering on it would put rows of every outcome under a
heading claiming one.

Gated with its own precondition: the test asserts that no row carries the
suite's name itself, or it would be measuring substring matching rather than
ancestry.

Refs #439.

* Put the log's monospace stack in the token layer

The one literal A3b left outside it. The sheet has had zero hardcoded font
stacks since #455 gave it a token layer, and the log body had no business
being the exception.

Refs #439.

* Carry every tail screenshot with its row, not just the nearest

Review finding, verified before fixing. `TestScreenshotFlow` emits a test's
*last three* screenshots (`suffix(tailCount)`), so a test with several is
preceded by several — and the filter walked back exactly one sibling, leaving
the outer ones on screen with no row to belong to.

The synthetic fixture already contains such a row (`testFails()`, whose
standard activities and failure activity each carry a screenshot), so the gate
was passing on a one-tail row while the two-tail one went unchecked. It now
counts every tail rather than the first, and asserts the fixture actually holds
a run of more than one — mutation-tested against the one-sibling version, which
it fails.

Refs #439.

* Keep the executions count level with the rows on screen

Review finding, reproduced before fixing. Filter the real two-bundle report to
`Passed` and the toolbar reads `14 executions`; jump to a failing test from the
digest and a thirteenth row appears — 15 executions on screen — while the
toolbar still says 14. `digestJump` cleared `display` on the row and its
ancestors itself and never recomputed, so it walked around the single writer
the file's own contract names ("One writer, so the rendered value and every
recomputation of it are the same sentence"). The element is `aria-live`, so the
notice a screen-reader reader gets when a row arrives was silence.

The count is now taken off the rows rather than accumulated while the filter
decides them: `countExecutionsOnScreen` sums the visible rows' `data-runs` and
writes through `setCount`, and both the filter and the jump call it. Anything
that changes what the pane holds can restate the figure the same way.

The gate is the reviewer's scenario exactly — `Passed`, then the digest jump —
asserting the toolbar equals the executions actually visible, summed from the
rows, with a precondition that the jump really did reveal one. It fails on the
old code (`4 executions` where 5 are showing) and holds on the new. The goldens
carry only this script change.

Refs #439.

* Say only what the fixtures show about the combined shape

Two documentation nits from the review of #486.

The cross-reference named `testExecutionCountsMatchAcrossBackends`; the test is
`testExecutionCountsAgreeAcrossBackends`. A stale name in a repo that otherwise
keeps them exact is a reader sent looking for a test that does not exist.

And the note on a test that is both parameterized and repeated asserted more
than anyone here can know: "legacy counts the product, modern the larger of the
two". No fixture exercises that shape, which is the whole reason it is written
down — and modern's answer in particular turns on where the `Repetition` nodes
sit, `R` if they nest under `Arguments` rather than `max(R, A)`. The note now
says the shape is unmeasured and names the gate that would catch it, which is
what the reasoning behind the field actually supports.

Refs #439.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant