Skip to content

Commit ae74336

Browse files
committed
TQ: Async Nodes and P2P connections
Builds on #9232 This is the first step in wrapping the `trust_quorum::Node` so that it can be used in an async context and integrated with sled-agent. Only the sprockets networking has been fully integrated so far such that each `NodeTask` has a `ConnMgr` that sets up a full mesh of sprockets connections. A test for this connectivity behavior has been written but the code is not wired into the production code yet. Messages can be sent between `NodeTasks` over sprockets connections. Each connection exists in it's own task managed by an `EstablishedConn`. The main `NodeTask` task sends messages to and receives messages from this task to interact with the outside world via sprockets. Currently only `Ping` messages are sent over the wire as a means to keep the connections alive and detect disconnects. A `NodeHandle` allows one to interact with the `NodeTask`. Currently only three operations are implemented with messages defined in `NodeApiRequest`. The user can instruct the node who it's peers are on the bootstrap network to establish connectivity, can poll for connectivity status, and can shutdown the node. All of this functionality is used in the accompanying test. It's important to re-iterate that this code only implements connectivity between trust quorum nodes and no actual trust quorum messages are sent. They can't be as a handle can not yet initiate a reconfiguration or LRTQ upgrade. That behavior will come in a follow up. This PR is large enough. A lot of this code is similar to the LRTQ connection management code, except that it operates over sprockets rather than TCP channels. This introduces some complexity, but it is mostly abstracted away into the `SprocketsConfig`.
1 parent 08cd293 commit ae74336

File tree

8 files changed

+1754
-22
lines changed

8 files changed

+1754
-22
lines changed

Cargo.lock

Lines changed: 212 additions & 22 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -368,6 +368,7 @@ assert_matches = "1.5.0"
368368
assert_cmd = "2.0.17"
369369
async-bb8-diesel = "0.2"
370370
async-trait = "0.1.89"
371+
attest-mock = { path = "../dice-util/attest-mock" }
371372
atomicwrites = "0.4.4"
372373
authz-macros = { path = "nexus/authz-macros" }
373374
backoff = { version = "0.4.0", features = [ "tokio" ] }

sled-agent/src/bootstrap/config.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@
77
pub const BOOTSTRAP_AGENT_HTTP_PORT: u16 = 80;
88
pub const BOOTSTRAP_AGENT_RACK_INIT_PORT: u16 = 12346;
99
pub const BOOTSTORE_PORT: u16 = 12347;
10+
pub const TRUST_QUORUM_PORT: u16 = 12349;

trust-quorum/Cargo.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,13 @@ workspace = true
1111
anyhow.workspace = true
1212
bcs.workspace = true
1313
bootstore.workspace = true
14+
bytes.workspace = true
1415
camino.workspace = true
1516
chacha20poly1305.workspace = true
17+
ciborium.workspace = true
1618
daft.workspace = true
1719
derive_more.workspace = true
20+
futures.workspace = true
1821
gfss.workspace = true
1922
hex.workspace = true
2023
hkdf.workspace = true
@@ -28,6 +31,7 @@ sha3.workspace = true
2831
sled-agent-types.workspace = true
2932
slog.workspace = true
3033
slog-error-chain.workspace = true
34+
sprockets-tls.workspace = true
3135
static_assertions.workspace = true
3236
subtle.workspace = true
3337
thiserror.workspace = true
@@ -38,12 +42,14 @@ omicron-workspace-hack.workspace = true
3842

3943
[dev-dependencies]
4044
assert_matches.workspace = true
45+
attest-mock.workspace = true
4146
dropshot.workspace = true
4247
omicron-test-utils.workspace = true
4348
proptest.workspace = true
4449
serde_json.workspace = true
4550
test-strategy.workspace = true
4651
trust-quorum-test-utils.workspace = true
52+
sprockets-tls-test-utils.workspace = true
4753

4854
[features]
4955
# Impl `PartialEq` and `Eq` for types implementing `subtle::ConstantTimeEq` when

0 commit comments

Comments
 (0)