Skip to content

refactor: Change some validation.cpp methods to return BlockValidationState - #35570

Open
optout21 wants to merge 12 commits into
bitcoin:masterfrom
optout21:2605-validation-state-return
Open

optout21 wants to merge 12 commits into
bitcoin:masterfrom
optout21:2605-validation-state-return

Conversation

@optout21

Copy link
Copy Markdown
Contributor

Summary. Refactor validation result to be a return value instead of an output parameter in several validation.cpp methods.

Motivation. The benefits of the change are:

  • Exclude the potentially inconsistent case when the bool return value and the returned state are inconsistent
  • Exclude the ambiguity whether the passed in value of state is used or not (not obvious in chained calls)
  • Remove the possibility of unintuitive interaction between subsequent calls with the same state. In case of a validation error, a failure reason is set if the state was valid, but not if it was already invalid.
  • Slightly simpler: It's more evident which is the result; one less parameters.

This has grown out from #33856, mentioned in comment here and here.

Details. Many methods follow the scheme where the validation state is returned in an output parameter (BlockValidationState& state), and and additional bool return value indicating success. In success case the convention is that state.IsValid() and the return value are both true. After the change there is only a BlockValidationState return value, which is either success (state.IsValid() == true), or an invalid/error case.

This change is a highly localized refactor, but touching a sensitive file.

Relevant methods called by ProcessNewBlockHeaders and AcceptBlock (directly and indirectly) are touched.

Changes are separated into commits by touched methods, ordered by bottom-to-top in the call hierarchy.

@DrahtBot

DrahtBot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35570.

Reviews

See the guideline and AI policy for information on the review process.

Type Reviewers
Concept ACK w0xlt, purpleKarrot, stringintech, yuvicc, nervana21
Stale ACK arejula27

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #36244 (validation, net: Process blocks asynchronously and reduce cs_main contention by w0xlt)
  • #36066 (validation: Separate check-only version of ConnectBlock by optout21)
  • #35906 (First steps towards a stateless, side-effect free validation library by purpleKarrot)
  • #35793 (Implement BIP 54 (Consensus Cleanup) without mainnet activation by darosior)
  • #35751 (validation: use parallel input prevout fetching in TestBlockValidity by andrewtoth)
  • #35646 (RFC: Separate out runtime errors from BlockValidationState using util::Expected by yuvicc)
  • #35569 (Encapsulation for CTransaction by purpleKarrot)
  • #35557 (kernel, validation: Add btck_chainstate_manager_set_clock_time by ryanofsky)
  • #35502 (refactor: extract per-message helpers from ProcessMessage (move-only) by w0xlt)
  • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible places where named args for integral literals may be used (e.g. func(x, /*named_arg=*/0) in C++, and func(x, named_arg=0) in Python):

  • CheckBlock(block, chainparams.GetConsensus(), false, false) in src/bench/duplicate_inputs.cpp
  • AcceptBlock(new_block, &new_block_index, true, nullptr, nullptr, true) in src/test/baseindex_tests.cpp
  • AcceptBlock(pblock, nullptr, true, dbp, nullptr, true) in src/validation.cpp
  • AcceptBlock(pblockrecursive, nullptr, true, &it->second, nullptr, true) in src/validation.cpp

2026-09-19 05:15:48

@optout21
optout21 force-pushed the 2605-validation-state-return branch 2 times, most recently from cc4bac4 to 8963bb6 Compare June 20, 2026 07:11
@DrahtBot

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task tidy: https://github.com/bitcoin/bitcoin/actions/runs/27862735029/job/82461396003
LLM reason (✨ experimental): CI failed because clang-tidy reported readability-const-return-type errors in src/validation.cpp (e.g., static const BlockValidationState ... return types), causing the clang-tidy step to exit with failure.

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@optout21

optout21 commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

While working on this PR, one instance was identified where the invariant return_value == state.IsValid() was not guaranteed:
At the end of ChainstateManager::AcceptBlock(), FlushStateToDisk() was called, and its return value discarded, but it could have a side effect in state. In turn, this could make a difference in ChainstateManager::LoadExternalBlockFile().

Options to resolve this:

A. Since the return value is ignored, ignore the state returned as well.
B. Handle the error from FlushStateToDisk(), and pass the error.

The ignoring of flush result (A.) is also proposed in #29700 (cc: @ryanofsky ).

The minor behavior-change could be also omitted from this PR, limiting strictly to a no-behavior-change refactor:

  • Do not change AcceptBlock, keep the dual value & state return values
  • Restrict the scope of the PR and drop all commits 5-13
  • Defer this PR until this issue is solved separately first

@w0xlt

w0xlt commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Concept ACK

@purpleKarrot

Copy link
Copy Markdown
Contributor

Concept ACK. Preferring the return value over output parameters is a useful and I would say necessary improment.

But could we avoid using BlockValidationState as a state machine, or mutable local variable? That means, instead of code like:

{
  BlockValidationState state;
  if (...) {
    state.Invalid(...);
  }
  return state;
}

I would prefer something like:

{
  if (...) {
    return BlockValidationState::Invalid(...);
  }
  return BlockValidationState::Valid();
}

@optout21

Copy link
Copy Markdown
Contributor Author

could we avoid using BlockValidationState as a state machine, or mutable local variable

I fully agree, shorter-scoped variables mean less chance of potential interplay between calls, less complexity. I did change this in several places, but not everywhere. In some places there was a real chance for interaction between subsequent calls (e.g. error reason is set conditional of already set value), and I kept it to be safe (w.r.t. behavior changes). But I should review and change it more aggressively.

@optout21
optout21 force-pushed the 2605-validation-state-return branch 2 times, most recently from 4626ef2 to 8207ac1 Compare June 22, 2026 16:38
@optout21

Copy link
Copy Markdown
Contributor Author

Applied some improvements:

  • Got rid of method-wide BlockValidationState state variables, use only restricted scope state variables, optimally in "if (const auto state = Xxx; !state.IsValid())" construct. Thanks @purpleKarrot for the emphasis!
  • Added InvalidState static helper to create invalid BlockValidationState instance. In error branches, instead of 3 statements (declaration, setting, and return) now one is enough (return&construction). Added as new first commit.

@arejula27 arejula27 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concept ACK 8207ac1bb79f76c9005e8e2d695b725539b3c5c0

Looks good overall. I'd push this one step further and use util::Expected here, it's a natural fit and follows the PR's own motivation. The PR already cleans up the bool + state out-param inconsistency by making state the return value; util::Expected<T, BlockValidationState> does the same thing but at the type level instead of by convention. Success and failure become distinct alternatives of the type.

Concretely, a bare BlockValidationState still carries all three modes (M_VALID/M_INVALID/M_ERROR) in the return value even though M_VALID is now redundant with "the call succeeded". With util::Expected that mode collapses into has_value() and is no longer stored, the failure modes move to the error channel, and out-params like ppindex fold into the success channel. util/result.h also explicitly steers low-level functions toward util::Expected.

// pindex folded into the success channel
[[nodiscard]] util::Expected<CBlockIndex*, BlockValidationError> AcceptBlockHeader(
    const CBlockHeader& block, bool min_pow_checked);

auto res = AcceptBlockHeader(header, /*min_pow_checked=*/true);
if (!res) {
    if (res.error().IsInvalid()) MaybePunishNodeForBlock(...); // peer's fault
    return util::Unexpected{std::move(res).error()};
}
CBlockIndex* pindex = res.value(); // only reachable on success

One open question (happy to leave it as discussion): where does M_INVALID fit best? Above I put it in the error channel next to M_ERROR, but it's arguably a successful, expected outcome of validation, the function did run and produced a verdict ("this block is invalid"), whereas M_ERROR is a genuine runtime failure that prevented producing a result. So an alternative split would keep Valid/Invalid in the value channel and reserve the error channel for runtime errors only. I don't have a strong preference.

Comment thread src/validation.h Outdated
* @param[out] ppindex If set, the pointer will be set to point to the last new block index object for the given headers
* @returns false if AcceptBlockHeader fails on any of the headers, true otherwise (including if headers were already known)
* @returns BlockValidationState indicating the result. IsValid() returns true if all headers
* were accepted. On failure, IsInvalid() is false and the state contains the specific

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On failure, IsInvalid() is false
Would not be "true"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. On failure IsValid() is false, moreover IsInvalid() is true. I state: "On failure, IsValid() is false and the state contains the specific validation failure reason. Never returns Error state.". From the last bit it follows that IsInvalid() is true, but I don't mention it here, only IsValid(), as that should be used to check for normal vs. exceptional case.

@stringintech

stringintech commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Concept ACK

One open question (happy to leave it as discussion): where does M_INVALID fit best? Above I put it in the error channel next to M_ERROR, but it's arguably a successful, expected outcome of validation, the function did run and produced a verdict ("this block is invalid"), whereas M_ERROR is a genuine runtime failure that prevented producing a result. So an alternative split would keep Valid/Invalid in the value channel and reserve the error channel for runtime errors only. I don't have a strong preference.

There is a thread in #33856 where this was also brought up. I included a POC in the last comment of that thread which keeps the value channel for M_VALID/M_INVALID and the error channel for M_ERROR, which I think makes more sense as you're suggesting: M_INVALID is not an operation failure.

In general, I think stripping the runtime failure from BlockValidationState could also leave the door open for preferring exceptions over error values in the future. That said, this should be orthogonal to this PR and could be addressed in a follow-up if desired.

@optout21
optout21 force-pushed the 2605-validation-state-return branch from 9819124 to 312c4e7 Compare September 16, 2026 13:59
@DrahtBot

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/35105544002/job/104825688520
LLM reason (✨ experimental): CI failed because IWYU reported include fixes were needed and intentionally exited non-zero (“Failure generated from IWYU”).

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@optout21

Copy link
Copy Markdown
Contributor Author

Fixed IWYU CI failure (missing include for new Assume).
Additionally added comments to CheckBlockHeader and ContextualCheckBlockHeader (that they cannot return Error), and also fixed an outdated comment in src/net_processing.cpp (AcceptBlockHeader/ProcessNewBlockHeaders).

@optout21
optout21 force-pushed the 2605-validation-state-return branch from 312c4e7 to a92cd68 Compare September 16, 2026 16:50

@hodlinator hodlinator left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed a92cd68

The way this PR transforms pre-existing functions one at a time is something I wish #35646 did as well.

Agree that it complements #35646, but think it should go in after that one (see inline comment), so I'll defer my Concept A-C-K until something like that lands. Please don't see my reservations as disinterest, I spend time on it because I agree the general direction is worthwhile.

Branch incorporating my suggestions into existing commits: https://github.com/hodlinator/bitcoin/tree/pr/35570_suggestions

Comment thread src/consensus/validation.h Outdated
public:
//! Factory helper method to create an Invalid BlockValidationState
static BlockValidationState InvalidState(BlockValidationResult result,
const std::string& reject_reason = "", const std::string& debug_message = "")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thread #35570 (comment):

Please also fix MakeError(const std::string& reject_reason = "").

Comment thread src/kernel/bitcoinkernel.cpp Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be dropped in the "Refactor CheckBlock signature" commit so we only assign state once.

Suggested change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, MakeInvalid can be used, and then there is no need for the state variable, only further down. Done.

Comment thread src/consensus/validation.h Outdated
return state;
}
//! Factory helper method to create an Error BlockValidationState
static BlockValidationState ErrorState(const std::string& reject_reason = "")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thread #35570 (comment):

What I meant is to add both MakeInvalid() and MakeError() helpers in the first commit. Then in the "Refactor FatalError signature" and "Refactor AcceptBlock signature" commits you can switch directly to MakeError() instead of the intermediate hop through state.Error() ... return state which those commits currently do.

Comment thread src/validation.cpp Outdated
// Check that the header is valid (particularly PoW). This is mostly
// redundant with the call in AcceptBlockHeader.
if (!CheckBlockHeader(block, state, consensusParams, fCheckPOW))
state = CheckBlockHeader(block, consensusParams, fCheckPOW);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thread #35570 (comment):

I know many checks away in later commits but still think it's good practice to add them in intermediate commits when you are changing behavior of such critical code.

At the very least the "test ancestor commits" CI job can churn through them.

You are changing the local code behavior from maybe changing state when passed as an argument, to definitely stomping it with a return value.

I went through and added Assumes in my suggestions branch and many of them survive until the end, although I admit some are overly paranoid.

Comment thread src/validation.cpp
}

if (!CheckBlockHeader(block, state, GetConsensus())) {
if (const auto state = CheckBlockHeader(block, GetConsensus()); !state.IsValid()) {
assert(state.IsInvalid());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thread #35570 (comment):

Following the call tree on the current PR base is good to do, but not the best we can do. Adding asserts and now comments about current behavior feels like duct tape to me.

As the PRs currently stand, I much prefer #35646 be merged before this PR so invariants are enforced at compile time through types rather than by the duct tape that is runtime failures and comments. That makes it much harder for silent merge conflicts and the like to cause issues.

Comment thread src/validation.cpp Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

470b305 Internal simplification in TestBlockValidity:

This state should be removed if we keep this commit where we call CheckBlock() etc which return it's own instances instead of overwriting state. That means we also refactor the next if-block:

    if (block.hashPrevBlock != *Assert(tip->phashBlock)) {
        return BlockValidationState::MakeInvalid({}, "inconclusive-not-best-prevblk");
    }

That also necessitates re-introducing state before calling ConnectBlock(), unless that method is also refactored to return BlockValidationState:

    {
        BlockValidationState state;
        // Set fJustCheck to true in order to update, and not clear, validation caches.
        if (!chainstate.ConnectBlock(block, state, &index_dummy, view_dummy, /*fJustCheck=*/true)) {
            if (state.IsValid()) NONFATAL_UNREACHABLE();
            return state;
        } else {
            if (!state.IsValid()) NONFATAL_UNREACHABLE();
        }
    }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Realized BlockValidationState::MakeInvalid({}, ... triggers the Assume(result != BlockValidationResult::BLOCK_RESULT_UNSET), so might be best to skip adding the latter for now.

@optout21 optout21 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New round of changes following new review comments.

Comment thread src/consensus/validation.h Outdated
public:
//! Factory helper method to create an Invalid BlockValidationState
static BlockValidationState InvalidState(BlockValidationResult result,
const std::string& reject_reason = "", const std::string& debug_message = "")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, the fell through the cracks; done.

Comment thread src/kernel/bitcoinkernel.cpp Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, MakeInvalid can be used, and then there is no need for the state variable, only further down. Done.

Comment thread src/consensus/validation.h Outdated
return state;
}
//! Factory helper method to create an Error BlockValidationState
static BlockValidationState ErrorState(const std::string& reject_reason = "")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken. Through reordering, the double touching of FatalError and AcceptBlock can be avoided.

Comment thread src/validation.cpp Outdated
// Check that the header is valid (particularly PoW). This is mostly
// redundant with the call in AcceptBlockHeader.
if (!CheckBlockHeader(block, state, consensusParams, fCheckPOW))
state = CheckBlockHeader(block, consensusParams, fCheckPOW);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, in the previous form with input-output parameters method had the chance to overwrite the status, but did so only in error case. I've added the proposed Assume's in every place where we overwrite the status always. Some of these Assume's disappear in later commits, but not all.

optout21 and others added 6 commits September 18, 2026 16:56
Add `MakeInvalid` static helper to create invalid `BlockValidationState` instance,
for easier construction in error branches. Instead of the 3 statements of
declaration, setting, and return with the exisiting non-static method, now
return&construction is possible in one statement.
Change the (internal) method `CheckBlockHeader` to return the validation
result in the return value instead of an output parameter.
Change the (internal) method `ContextualCheckBlockHeader` to return the validation
result in the return value instead of an output parameter.
Change the method `AcceptBlockHeader` to return the `BlockValidationState`
validation result in the return value instead of an output parameter.
Return BlockValidationState by value instead of using an out-parameter,
similar to the TestBlockValidity refactoring in 74690f4.

Remove redundant int return from btck_chainstate_manager_process_block_header.
Previously returned both an int result and an output validation state parameter, creating ambiguity
where non-zero could mean either invalid header or processing failure. Since ProcessNewBlockHeaders already provides complete validation info, the int return was redundant.

Co-authored-by: stringintech <[email protected]>
Co-authored-by: stickies-v <[email protected]>
Change the (internal) method `CheckMerkleRoot` to return the `BlockValidationState`
validation result in the return value instead of an output parameter.
@optout21

Copy link
Copy Markdown
Contributor Author

Rebased ot master, hoping to fix the mining_template_verification.py failure in CI (could not reproduce locally).

@optout21
optout21 force-pushed the 2605-validation-state-return branch from 1de8a2e to 6294fb5 Compare September 18, 2026 14:59
@optout21

Copy link
Copy Markdown
Contributor Author

Test failure is due to this if in TestBlockValidity:

    if (block.hashPrevBlock != *Assert(tip->phashBlock)) {
        return BlockValidationState::MakeInvalid({}, "inconclusive-not-best-prevblk");
    }

which creates an Invalid state with result == BlockValidationResult::BLOCK_RESULT_UNSET, which violates a newly added Assume.

@optout21

Copy link
Copy Markdown
Contributor Author

Small change to fix CI failure.
CI caught a violation of the newly introduced Assume: #31981 introduced a usage of BlockValidationState where the result is discarded, and Unset result is used.

git range-diff 6294fb57f8f8db077d22ec294633e7e78762d5b1...0e8dc6b9dd7f07e756dcfe0b8146cc2daa17008e

@optout21
optout21 force-pushed the 2605-validation-state-return branch from 6294fb5 to 0e8dc6b Compare September 19, 2026 04:54
optout21 and others added 6 commits September 19, 2026 07:15
Change the method `CheckBlock` to return the `BlockValidationState`
validation result in the return value instead of an output parameter.
Change the (internal) method `CheckWitnessMalleation` to return the
`BlockValidationState` validation result in the return value
instead of an output parameter.
Change the (internal) method `ContextualCheckBlock` to return the
`BlockValidationState` validation result in the return value
instead of an output parameter.
Change the method `FatalError` to return the constructed `BlockValidationState`
object in a return value instead of an output parameter.
Add a helper `MakeError` factory method to the `BlockValidationState`, similar to `MakeInvalid`.

Co-authored-by: Íñigo Aréjula Aísa <[email protected]>
Change the method `FlushStateToDisk` to return the `BlockValidationState`
validation result in the return value instead of an output parameter.
Change the method `AcceptBlock` to return the `BlockValidationState`
validation result in the return value instead of an output parameter.

Co-authored-by: Íñigo Aréjula Aísa <[email protected]>
@optout21
optout21 force-pushed the 2605-validation-state-return branch from 0e8dc6b to 9e3c5ba Compare September 19, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants