Skip to content

ModernResultReader, ModernPayloadStore, and extension-based attachment typing (#391, Tasks 8–10) - #447

Merged
tylervick merged 5 commits into
mainfrom
tylervick/modern-reader-391
Aug 13, 2026
Merged

tylervick merged 5 commits into
mainfrom
tylervick/modern-reader-391

Conversation

@tylervick

Copy link
Copy Markdown
Member

Part of #391 (milestone 4.0), phase 2 of the migration plan: the modern backend itself. Follows #441 (client + schema) and #443 (the port). The reader is reachable only through direct construction in tests — backend selection is Task 11, the next PR. Do not expect render changes from this PR; nothing routes through the new code yet.

What's here

  • ModernResultReader (Task 8): builds ParsedResult from get test-results. Statuses map into the neutral enum (Expected Failure still flattens to unknown downstream, deliberately); multi-repetition status derives from Repetition children, ignoring the parent Test Case's own resultRetryResults reports Passed there for a Failed-then-Passed test, so taking it would turn mixed tests green. The tree renders flat (no synthesized All tests / *.xctest wrappers). One activities subprocess per exact test-case id; a failed query records the new .missingActivities fault rather than shipping a silently gutted report.
  • ModernPayloadStore (Task 9): one export attachments call per bundle + the manifest.json join (activity attachment uuid ↔ basename of exportedFileName). No per-payload subprocess, no lock table — only the one-shot export is guarded. Run logs come from get log --type action, rendered from structured messages (the format has no emittedOutput).
  • Attachment typing (Task 10): the failable AttachmentType(filenameExtension:) landed in The port: ParsedResult model, LegacyResultReader, and the renderer migration (#391, Tasks 3–5b) #443; this PR adds the cross-backend agreement test and fixes the carried review note — public.data was missing from the legacy UTI fallback table, degrading extensionless data attachments to .unknown.
  • Review notes from XCResultToolClient and new-format schema structs (#391) #441/The port: ParsedResult model, LegacyResultReader, and the renderer migration (#391, Tasks 3–5b) #443: XCResultToolError now conforms to LocalizedError (faults and warnings printed NSError boilerplate before); the client's unconditional --path/--schema-version is verified safe for all three invocation shapes used (get test-results, export attachments, get log) — no client change needed.
  • Sample app: a parameterized @Test(arguments: [1, 2, 3]) so Arguments nodes stop being fixture-unexercised (measured: 3 nodes in the regenerated fixture). Its argument sets merge into one rendered row; the TestResults header total moves 17 → 18 in CoreTests.

Plan deviation, coordinator-approved: failure text is a join

Measured on Xcode 26.2 fixtures: the activities document carries every assertion failure as a failure-flagged activity row — positioned, timestamped, nested inside the user's activity when the assertion fired there — with only the file:line prefix missing; the tests document's Failure Message node keeps the prefix but has no timestamp. The plan's anti-double-count guard kept the activity title and dropped the message, losing file:line from every modern failure render — contradicting the spec's failureTitlePrefix allow-list entry and the "failure text sources from Failure Message nodes" rule.

As ruled: each message retitles the first unclaimed failure-flagged activity whose title is an exact suffix of the message (document order, first-unmatched-first, recursive so the nested RetryResults case retitles in place), keeping position, nesting, timestamp, attachments, and children. Unmatched messages (skip reasons, expected-failure notes) append as before. No regex extraction of file/line — the string renders as given. Plan and spec amended in this PR; regression tests pin file:line presence on both the plain and the nested case.

Fault discipline

testModernRenderOfEveryFixtureRecordsNoFaults does a full modern-path render of all three fixtures — reader + store, attachments and logs resolved in linking mode — and asserts zero faults and zero unresolved attachments. Structurally absent fields (display names, activity types, suite durations, finish times) never fault; a failed activities query and a failed export still do.

Verification

  • Full suite: 65 tests, 0 failures, 2 pre-existing skips (CoreTests.testRetryFunctionalityJunit expectations drift with Xcode version #378 JUnit drift, baseline capture) — legacy paths (LegacyResultReaderTests, CoreTests, SanityTests, fault tests) and modern paths (ModernResultReaderTests ×11, ModernPayloadStoreTests ×5, AttachmentTypeTests ×3) both green.
  • Fixtures regenerated via ./prepareTestResults.sh on Xcode 26.2 (17C52), xcresulttool 24514, schema 0.1.0 — the spec's measurement environment.

Refs #391.

🤖 Generated with Claude Code

tylervick and others added 5 commits August 12, 2026 17:06
…e legacy backend

Task 10 of the migration plan. The AttachmentType(filenameExtension:)
initializer itself landed with #443; what was missing is the cross-backend
agreement pin and one table entry: LegacyResultReader's UTI fallback omitted
public.data, so an extensionless public.data attachment degraded
.data -> .unknown on the legacy backend only.

Co-Authored-By: Claude Fable 5 <[email protected]>
Carried forward from the #441 review: consumers now exist (faults and
warnings format these errors), and localizedDescription printed the generic
NSError boilerplate rather than the command, exit status, and stderr.

Co-Authored-By: Claude Fable 5 <[email protected]>
Task 9 of the migration plan. One `export attachments` call per bundle plus
the manifest.json join (activity attachment uuid <-> basename of
exportedFileName) replaces the legacy per-payload export and its lock table;
only the one-shot export is guarded. Run logs come from `get log --type
action` and render from the structured messages, since the new format has
no emittedOutput.

Committed ahead of Task 8 so each commit builds: the reader's payloadStore
property references this type.

Co-Authored-By: Claude Fable 5 <[email protected]>
Task 8 of the migration plan. Parity rules carried exactly: statuses map
into the neutral enum (Expected Failure still flattens to unknown
downstream); multi-repetition status derives from the Repetition children
and ignores the parent Test Case's own result (RetryResults reports Passed
there for a Failed-then-Passed test); the tree renders flat, without the
legacy wrapper groups.

Failure text joins the two documents rather than choosing one: measured on
Xcode 26.2, every assertion failure appears both as a failure-flagged
activity row (positioned and timestamped, but stripped of file:line) and as
a Failure Message node (file:line kept, no timestamp). Each message retitles
the first unclaimed failure activity whose title is its exact suffix, in
document order, nested rows included; unmatched messages (skip reasons,
expected-failure notes) append. The plan's original guard kept the activity
title instead, which shipped strictly less information than the spec's
failureTitlePrefix entry documents — coordinator-approved correction,
amended in the plan.

A failed activities query records the new .missingActivities fault; fields
the format structurally lacks never fault, pinned by a full modern-path
render of all three fixtures asserting zero faults.

The sample app gains a parameterized @test so Arguments nodes stop being
fixture-unexercised; its argument sets merge into one rendered row, moving
the TestResults header total from 17 to 18.

Co-Authored-By: Claude Fable 5 <[email protected]>
The load-bearing correction: failure text is a join of the two documents,
not a choice between them. Measured on Xcode 26.2, the activities document
carries every assertion failure as a positioned, timestamped, sometimes
nested failure row missing only the file:line prefix; the plan's original
anti-double-count guard would have dropped the prefixed Failure Message and
shipped less than the spec's failureTitlePrefix entry documents.
Coordinator-approved retitle-join recorded in both documents.

Mechanical notes: Task 9 commits before Task 8 (type dependency), TestRun is
nested in TestActivities, the AttachmentType initializer shipped failable in
443, and regenerating fixtures after the sample-app change collapses the
plan's 6-of-7 intermediate state.

Co-Authored-By: Claude Fable 5 <[email protected]>
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@tylervick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0c810b2d-99b2-45ae-86a8-862489fdb830

📥 Commits

Reviewing files that changed from the base of the PR and between 9ed0905 and ff5e7b8.

📒 Files selected for processing (14)
  • Sources/XCTestHTMLReportCore/Classes/Helpers/FaultCollector.swift
  • Sources/XCTestHTMLReportCore/Classes/ResultReading/Legacy/LegacyResultReader.swift
  • Sources/XCTestHTMLReportCore/Classes/ResultReading/Modern/ModernPayloadStore.swift
  • Sources/XCTestHTMLReportCore/Classes/ResultReading/Modern/ModernResultReader.swift
  • Sources/XCTestHTMLReportCore/Classes/ResultReading/Modern/XCResultToolClient.swift
  • Tests/XCTestHTMLReportTests/AttachmentTypeTests.swift
  • Tests/XCTestHTMLReportTests/CoreTests.swift
  • Tests/XCTestHTMLReportTests/LegacyResultReaderTests.swift
  • Tests/XCTestHTMLReportTests/ModernPayloadStoreTests.swift
  • Tests/XCTestHTMLReportTests/ModernResultReaderTests.swift
  • Tests/XCTestHTMLReportTests/XCResultToolClientTests.swift
  • XCTestHTMLReportSampleApp/SampleAppUnitTests/SwiftTestingSuite.swift
  • docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md
  • docs/superpowers/specs/2026-08-10-xcresulttool-legacy-migration-design.md

Comment @coderabbitai help to get the list of available commands.

@tylervick
tylervick merged commit 3441ff3 into main Aug 13, 2026
7 checks passed
@tylervick
tylervick deleted the tylervick/modern-reader-391 branch August 13, 2026 00:48
tylervick added a commit that referenced this pull request Aug 13, 2026
…elease notes (#391, Tasks 14–15) (#451)

* feat!: emit our own schema from --json instead of the legacy object graph

BREAKING: --json previously dumped xcresulttool's legacy object graph verbatim.
That graph is Apple's internal shape and disappears with the legacy commands,
so --json now emits a documented schema, identical on both backends.

docs/json-schema.md is the contract, written before the encoder: field names
and nesting with a complete worked example, enum spellings, one uniform null
rule, duration and timestamp formats, ordering guarantees, and a semver
schemaVersion policy. The encoder (JsonReport.swift) is an explicit layer
rather than a synthesized Encodable on the internal model, so renaming a
Parsed* property breaks compilation instead of silently renaming public
output. ResultFile.exportJson() — the last exportRecursiveJson() call, kept
since Task 5a — is deleted; XCResultKit is confined to ResultReading/Legacy/.

JsonReportTests holds the output to the contract across both backends:
recursive schema identity, values deeply equal outside the two permitted
difference classes (JsonClassMask masks exactly the declared losses, with
non-vacuity stats), and arguments compared per class 2 — legacy asserted
== [] explicitly, modern asserted non-empty for the parameterized fixture.

The value differential surfaced one reader-parity gap the HTML differential
cannot see: under a retry-enabled plan, legacy stamps every summary
"iteration 1" while modern reports repetition info only for real
repetitions, and nothing renders a lone iteration number. The legacy reader
now strips it (strippingLoneIterationNumber) — a single execution carries no
repetition information on either backend. Recorded as a Task 14 execution
rule in the spec.

Refs #391 (Task 14).

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix: fold in the carried review findings from #447 and #450

- Pin the no-startTime keep-guard (#450 review): a no-start activity row
  carrying an attachment, a failure flag, or surviving children is kept by
  the symbol-annotation drop; only the contentless row is an annotation.
  Crafted-document test, since no fixture produces the shape.
- Suffix the failure-artifact name per matrix leg in test.yml (#450 review):
  upload-artifact v4 refuses duplicate names, so a run where both legs fail
  would have lost the second upload.
- ModernPayloadStore takes XCResultToolInvoking instead of the concrete
  client (#447 review), so export-failure faulting is provable in-suite; the
  new test also pins that a failed one-shot export no longer leaks its temp
  directory (removed in the catch — deinit never saw it).
- An out-of-range repetition index no longer falls back silently to
  runs.first (#447 review): the iteration renders no activities and records
  .missingActivities, instead of borrowing repetition 1's rows.

Refs #391.

Co-Authored-By: Claude Fable 5 <[email protected]>

* docs: document --result-reader and the --json schema change

README gains the --result-reader section, the --json break with a
before/after snippet, and the known backend differences drawn from the
allow-list. The 4.0 release notes are drafted at docs/release-notes/4.0.0.md
in the 3.0.0 narrative style for the release to source, covering the full
Task 15 checklist including the items added by rulings R1/R5/R7 and the
#443/#450 reviews. The spec gains a status header (phases 1-5 landed, phase
6 deliberately deferred) and the Task 14 execution rules; the plan's Tasks
14-15 are ticked with implementation amendments recorded.

Refs #391 (Task 15).

Co-Authored-By: Claude Fable 5 <[email protected]>

* docs: state schema identity, not value identity, in the README

The --json section claimed the file is identical across readers; the
contract's own headline is schema identity, with declared value differences.
Point at those and at testCase.arguments, the modern-only capability.
Addresses the valid half of CodeRabbit's review on #451.

Refs #391.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant