Add agent-agnostic doctor preflight to e2e SetupRepo#1162
Draft
Soph wants to merge 1 commit into
Draft
Conversation
After `entire enable` and any agent-specific tweaks, run `entire doctor --force` and fail the test if the output contains an actionable hook-drift marker (currently OUT OF DATE or REVIEW NEEDED). The full output is dumped to <artifact-dir>/doctor-preflight.log either way so failure traces include the diagnosis. Catches regressions like an incomplete hooks.json from `entire enable` before any prompt runs and the test fails in confusing ways. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Entire-Checkpoint: e88d4d18a0af
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an agent-agnostic E2E preflight check to catch actionable hook drift immediately after entire enable, improving failure clarity and avoiding later confusing agent failures.
Changes:
- Run
entire doctor --forceduringSetupRepoand fail fast if hook-drift markers are present; always write the doctor output to an artifact log. - Add
DoctorOutputhelper to capturedoctoroutput without forcing the caller to fail the test on non-zero exit. - Add a focused unit test for hook-drift marker detection logic.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| e2e/testutil/repo.go | Adds doctor preflight execution, artifact logging, and hook-drift marker scanning during repo setup. |
| e2e/testutil/preflight_test.go | Adds unit coverage for the hook-drift marker detection helper. |
| e2e/entire/entire.go | Adds DoctorOutput helper to capture entire doctor --force output for preflight/grepping scenarios. |
Comment on lines
+204
to
+207
| out, _ := entire.DoctorOutput(dir) | ||
| if artDir != "" { | ||
| _ = os.WriteFile(filepath.Join(artDir, "doctor-preflight.log"), []byte(out), 0o644) | ||
| } |
Comment on lines
+56
to
+58
| // output, ignoring the exit code. Useful for preflight scans where the | ||
| // caller wants to grep for specific findings without failing on | ||
| // unrelated non-zero exits (e.g. v2 generation health drift). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After
entire enableand any agent-specific tweaks, runentire doctor --forceand fail the test if the output contains an actionable hook-drift marker (currently OUT OF DATE or REVIEW NEEDED). The full output is dumped to /doctor-preflight.log either way so failure traces include the diagnosis. Catches regressions like an incomplete hooks.json fromentire enablebefore any prompt runs and the test fails in confusing ways.Entire-Checkpoint: e88d4d18a0af
Note
Medium Risk
Moderate risk because it changes
SetupRepoto add a new early-failure gate based on CLI output matching, which could introduce new E2E failures/flakiness if doctor output formats change.Overview
Adds a post-
entire enablepreflight inSetupRepothat runsentire doctor --force, writes the full output todoctor-preflight.log, and fails early if the output indicates hook drift (currently matchingOUT OF DATEorREVIEW NEEDED).Introduces
entire.DoctorOutputto capture doctor output without failing on non-zero exit codes, and adds a small unit test for the hook-drift marker matching logic (findHookDrift).Reviewed by Cursor Bugbot for commit 8e0a274. Configure here.