227 consensus validation / Flight Control #262

sandtreader · 2025-10-21T16:41:21Z

Resolves issue #227

This PR implements the skeleton of the "Flight Control" Go/NoGo mechanism for validation:

Merges BlockHeader and BlockBody messages into RawBlock, switches handling in EpochState, SPOState
Definition of BlockValidation messages which contain a ValidationResult (in common::validation)
New consensus module which interposes in the (merged) block message stream, accepting cardano.block.available from Mithril/Upstream and sending on cardano.block.proposed (which downstreams all now switch to)
Configuration of validators in consensus - currently empty because there aren't any. VRF Validator (VRF validation #197) can be the first one.

For now consensus can't do anything with a validation failure other than log it, because it doesn't have multiple chains to choose from. #205 will extend this.

I've requested lots of reviewers because (a) this is pretty central to a lot we are doing, and comments welcome and (b) almost everyone will be involved in implementing validators which need to connect to it.

Rationale: Within memory there is no extra cost to sending the full block body in the two cases (EpochState, SPOState) which subscribe only to headers, and having separate header messages complicates the forthcoming insertion of consensus into the block flow. The only case where this would add overhead would be if either EpochState and/or SPOState were remote while all the other consumers of block bodies were still internal. This seems very unlikely...

Consensus listens on cardano.block.available, and resends as cardano.block.proposed. Downstreams BlockUnpacker, EpochsState, SPOState now subscribe for cardano.block.proposed. Side-effect: Removed info-level filter from omnibus main to allow debug output again - needs investigation what it was for (ajw)

Reads BlockValidation messages from all configured validators, fails the block if any say NoGo. All must respond for it to continue. Note: Validation failure is only logged so far, we don't have multiple upstreams to run consensus on. Note: There are no configured topics yet because no validators yet issue messages!

modules/consensus/src/consensus.rs

buddhisthead · 2025-10-21T20:44:48Z

modules/consensus/src/consensus.rs

+
+                            // Read validation responses from all validators in parallel
+                            let results: Vec<_> =
+                                try_join_all(validator_subscriptions.iter_mut().map(|s| s.read()))


Does this deal with a validator that doesn't respond, like a timeout?

Good point. It will indeed lock up if a validator doesn't respond, or if a validator is configured which doesn't exist. I did wonder about a timeout but actually a non-response is a system failure - we can't safely either accept or reject the block. If we did had a short timeout, there's always a risk that something gets slowed down and it gets falsely triggered. If it's long, it's game over anyway because we'll miss the slot.

But very open to suggestions!

When we validate block header, will we have several modules to validate each category?
e.g. vrf validator module and kes validator module?

Or will we have them all in one module (Will this reduce some duplication? since these validations can share same state)

Yes that's right - either of which can flag a failure.

I think because we're listening for messages, it's best to have a timeout and a log message, otherwise it will be very difficult to debug the system.

Yes, agreed - I'll add a timeout.

Added, but see input-output-hk/caryatid#13

buddhisthead · 2025-10-21T20:46:16Z

@sandtreader this is really nice! Super clean design and easy to follow. Hope you don't mind - I turned on the co-pilot reviewer just to see what it would find.

Copilot

Pull Request Overview

This PR implements the "Flight Control" validation mechanism by introducing a consensus module that acts as a validation gateway between block sources and downstream consumers. The key changes consolidate block message handling and establish a validation framework where multiple validators can provide Go/NoGo decisions on proposed blocks.

Key Changes:

Merged BlockHeader and BlockBody messages into a single RawBlockMessage containing both header and body data
Created new consensus module that receives blocks on cardano.block.available and publishes validated blocks on cardano.block.proposed
Defined validation framework with ValidationStatus and ValidationError types for Go/NoGo decisions

Reviewed Changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
modules/consensus/src/consensus.rs	New consensus module implementing validation orchestration
common/src/validation.rs	Validation types defining Go/NoGo status and error categories
common/src/messages.rs	Consolidated block messages and added BlockValidation message type
modules/upstream_chain_fetcher/src/utils.rs	Updated to publish single RawBlockMessage instead of separate header/body
modules/mithril_snapshot_fetcher/src/mithril_snapshot_fetcher.rs	Updated to publish consolidated RawBlockMessage
modules/epochs_state/src/epochs_state.rs	Updated subscription from header-only to full block messages
modules/spo_state/src/spo_state.rs	Updated subscription from header-only to full block messages
modules/block_unpacker/src/block_unpacker.rs	Updated to consume RawBlockMessage instead of BlockBody
processes/omnibus/src/main.rs	Registered consensus module and modified logging filter

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

modules/consensus/src/consensus.rs

common/src/validation.rs

processes/omnibus/omnibus.toml

lowhung

This approach feels intuitive to me 👍

Co-authored-by: Copilot <[email protected]>

golddydev

This all looks good to me 👍🏼

Note ChainStore now listens on cardano.block.proposed

modules/consensus/src/consensus.rs

alexwoods · 2025-10-22T14:02:48Z

processes/omnibus/src/main.rs

-        .with_filter(filter::filter_fn(|meta| meta.is_event()));
+    let fmt_layer = fmt::layer().with_filter(EnvFilter::from_default_env());
+
+    // TODO disabled this filter because it prevents debugging - investigate


Was it intentional to check this in?

Yes, I was going to ask you about that... Maybe shouldn't go in this PR though.

Now reverted and opened #273 for it

Routes blocks to consensus

Note: Triggers a Caryatid bug where we don't handle timeouts on the future which produces a panic in select_all() - should be fine once this is fixed (and this is a Black Swan safety catch anyway)

Still needs fixing to allow DEBUG but left for future PR

sandtreader added 7 commits October 21, 2025 11:53

Emergency fix of omnibus.toml

73101f0

Add skeleton validation result

639b133

Use correct NASA terminology!

e725294

cargo fmt

850cd19

sandtreader requested review from alexwoods, buddhisthead, golddydev, lowhung, shd and whankinsiv October 21, 2025 16:41

Typo

561e57d

buddhisthead reviewed Oct 21, 2025

View reviewed changes

modules/consensus/src/consensus.rs Outdated Show resolved Hide resolved

buddhisthead reviewed Oct 21, 2025

View reviewed changes

modules/consensus/src/consensus.rs Outdated Show resolved Hide resolved

buddhisthead reviewed Oct 21, 2025

View reviewed changes

buddhisthead requested a review from Copilot October 21, 2025 20:45

Copilot AI reviewed Oct 21, 2025

View reviewed changes

modules/consensus/src/consensus.rs Outdated Show resolved Hide resolved

common/src/validation.rs Outdated Show resolved Hide resolved

common/src/validation.rs Outdated Show resolved Hide resolved

processes/omnibus/omnibus.toml Outdated Show resolved Hide resolved

lowhung approved these changes Oct 21, 2025

View reviewed changes

sandtreader and others added 6 commits October 22, 2025 11:41

Fix typo in config property

404bffc

Co-authored-by: Copilot <[email protected]>

Comment /// fix

b6dbe76

Co-authored-by: Copilot <[email protected]>

Comment /// fix

99d9514

Co-authored-by: Copilot <[email protected]>

Fix confusing comment re downstreams

6d37a74

Set spdd-retention-epochs to 0 by default

d236fea

Merge branch 'main' into prc/227-consensus-validation

fbc6faf

golddydev approved these changes Oct 22, 2025

View reviewed changes

sandtreader added 2 commits October 22, 2025 14:20

Merge branch 'main' into prc/227-consensus-validation

06c4238

Note ChainStore now listens on cardano.block.proposed

cargo fmt

0d7697e

Merge branch 'main' into prc/227-consensus-validation

06d969e

alexwoods reviewed Oct 22, 2025

View reviewed changes

sandtreader added 4 commits October 23, 2025 11:54

Merge branch 'main' into prc/227-consensus-validation

d50cf31

Fix publication topic for Mithril fetcher, log it

e045a64

Routes blocks to consensus

Add timeout to consensus validation reads

0e38076

Note: Triggers a Caryatid bug where we don't handle timeouts on the future which produces a panic in select_all() - should be fine once this is fixed (and this is a Black Swan safety catch anyway)

Restore fmt filtering

231f0ca

Still needs fixing to allow DEBUG but left for future PR

sandtreader merged commit 651746a into main Oct 23, 2025
2 checks passed

sandtreader deleted the prc/227-consensus-validation branch October 23, 2025 12:47

This was referenced Oct 27, 2025

📊 Weekly Status - Week of 2025-10-27 #300

Open

📊 Weekly Status - Week of 2025-10-31 #331

Open

227 consensus validation / Flight Control #262

227 consensus validation / Flight Control #262

Uh oh!

Conversation

sandtreader commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

golddydev Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buddhisthead commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lowhung left a comment

Choose a reason for hiding this comment

Uh oh!

golddydev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sandtreader commented Oct 21, 2025 •

edited

Loading

golddydev Oct 22, 2025 •

edited

Loading

buddhisthead commented Oct 21, 2025 •

edited

Loading