Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add Subscription RPC for Grandpa Finality #5732

Merged
75 commits merged into from
Aug 10, 2020
Merged

Add Subscription RPC for Grandpa Finality #5732

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
2f5367f
Rough skeleton for what I think the RPC should look like
HCastano Apr 8, 2020
9d369ac
Create channel for sending justifications
HCastano Apr 10, 2020
0095b78
WIP: Add subscribers for justifications to Grandpa
HCastano Apr 18, 2020
7719292
WIP: Add a struct for managing subscriptions
HCastano Apr 21, 2020
8c35b1e
Make naming more clear and lock data in Arc
HCastano Apr 21, 2020
07ef5fa
Rough idea of what RPC would look like
HCastano Apr 21, 2020
c9aa9e2
Remove code from previous approach
HCastano Apr 22, 2020
78250b7
Missed some things
HCastano Apr 22, 2020
fe31500
Update client/rpc-api/src/chain/mod.rs
HCastano Apr 22, 2020
ba13ee6
Update client/rpc-api/src/chain/mod.rs
HCastano Apr 22, 2020
fe360b9
Split justification subscription into sender and receiver halves
HCastano Apr 26, 2020
254bebe
Replace RwLock with a Mutex
HCastano Apr 26, 2020
9c996ea
Add sample usage from the Service's point of view
HCastano Apr 26, 2020
8090161
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano Apr 29, 2020
1150e5d
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano May 7, 2020
130a871
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano May 15, 2020
f42b6c3
Remove code that referred to "chain_" RPC
HCastano May 15, 2020
d61d7ec
Use the Justification sender/receivers from Grandpa LinkHalf
HCastano May 17, 2020
56e716b
Add some PubSub boilerplate
HCastano May 18, 2020
c0c6508
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano May 18, 2020
1bc9103
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano Jun 1, 2020
30e3831
Merge branch 'master' into hc-add-subscription-rpc-for-finality
HCastano Jun 3, 2020
8b0850a
Add guiding comments
HCastano Jun 3, 2020
cf07f2f
TMP: comment out to fix compilation
octol Jun 8, 2020
ab64ecb
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jun 9, 2020
b1c04ba
Return MetaIoHandler from PubSubHandler in create_full
octol Jun 10, 2020
ebad4d7
Uncomment pubsub methods in rpc handler (fails to build)
octol Jun 10, 2020
caaaa61
node/rpc: make Metadata concrete in create_full to fix compilation
octol Jun 11, 2020
f3cc272
node: pass in SubscriptionManger to grandpa rpc handler
octol Jun 11, 2020
3c4505a
grandpa-rpc: use SubscriptionManger to add subscriber
octol Jun 12, 2020
9a225ac
grandpa-rpc: attempt at setting up the justification stream (fails to…
octol Jun 16, 2020
a85dd53
grandpa-rpc: fix compilation of connecting stream to sink
octol Jun 16, 2020
1b0344f
grandpa-rpc: implement unsubscribe
octol Jun 16, 2020
12dd5df
grandpa-rpc: update older tests
octol Jun 18, 2020
bca6a13
grandpa-rpc: add full prefix to avoid confusing rust-analyzer
octol Jun 22, 2020
efaa0d1
grandpa-rpc: add test for pubsub not available
octol Jun 22, 2020
9632988
grandpa-rpc: tidy up leftover code
octol Jun 22, 2020
964f1e3
grandpa-rpc: add test for sub and unsub of justifications
octol Jun 22, 2020
82004cb
grandpa-rpc: minor stylistic changes
octol Jun 23, 2020
e526f95
grandpa-rpc: split unit test
octol Jun 23, 2020
6c71b8f
grandpa-rpc: minor stylistic changes in test
octol Jun 23, 2020
8bfb560
grandpa-rpc: skip returning future when cancelling
octol Jun 25, 2020
be7cb78
grandpa-rpc: reuse testing executor from sc-rpc
octol Jun 25, 2020
ac81cdf
grandpa-rpc: don't need to use PubSubHandler in tests
octol Jun 25, 2020
3f8630d
node-rpc: use MetaIoHandler rather than PubSubHandler
octol Jun 26, 2020
3b387e6
grandpa: log if getting header failed
octol Jun 26, 2020
bf4b19a
grandpa: move justification channel creation into factory function
octol Jun 26, 2020
4c96b73
grandpa: make the justification sender optional
octol Jun 26, 2020
12158a2
grandpa: fix compilation warnings
octol Jun 27, 2020
e2584e5
grandpa: move justification notification types to new file
octol Jun 27, 2020
dc3c888
grandpa-rpc: move JustificationNotification to grandpa-rpc
octol Jun 27, 2020
410af1f
grandpa-rpc: move JustificationNotification to its own file
octol Jun 27, 2020
fb3ad63
grandpa: rename justification channel pairs
octol Jun 27, 2020
5d80399
grandpa: rename notifier types
octol Jun 27, 2020
3f33fd2
grandpa: pass justification as GrandpaJustification to the rpc module
octol Jun 29, 2020
57f1963
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jun 29, 2020
90ae56e
Move Metadata to sc-rpc-api
tomusdrw Jun 30, 2020
be1df4f
grandpa-rpc: remove unsed error code
octol Jun 29, 2020
d4cddf3
grandpa: fix bug for checking if channel is closed before sendind
octol Jul 1, 2020
8bd9e5c
grandpa-rpc: unit test for sending justifications
octol Jun 30, 2020
127c19e
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jul 2, 2020
f43ee9b
grandpa-rpc: update comments for the pubsub test
octol Jul 2, 2020
39640c1
grandpa-rpc: update pubsub tests with more steps
octol Jul 2, 2020
f3c6c54
grandpa-rpc: fix pubsub test
octol Jul 2, 2020
44ba73e
grandpa-rpc: minor indendation
octol Jul 3, 2020
01f7d73
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jul 22, 2020
3f3b136
grandpa-rpc: decode instead of encode in test
octol Jul 23, 2020
908cf15
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jul 27, 2020
bb9ad4b
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Jul 27, 2020
410ddea
grandpa: fix review comments
octol Jul 31, 2020
ca126fd
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Aug 4, 2020
452024f
grandpa: remove unused serde dependency
octol Aug 4, 2020
8b0cf76
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Aug 5, 2020
7ddffef
Merge branch 'master' into hc-add-subscription-rpc-for-finality
octol Aug 6, 2020
817b425
Merge remote-tracking branch 'upstream/master' into hc-add-subscripti…
octol Aug 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions bin/node-template/node/src/service.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
let client = client.clone();
let pool = transaction_pool.clone();

Box::new(move |deny_unsafe| {
Box::new(move |deny_unsafe, _| {
let deps = crate::rpc::FullDeps {
client: client.clone(),
pool: pool.clone(),
Expand Down Expand Up @@ -278,7 +278,7 @@ pub fn new_light(config: Configuration) -> Result<TaskManager, ServiceError> {
transaction_pool,
task_manager: &mut task_manager,
on_demand: Some(on_demand),
rpc_extensions_builder: Box::new(|_| ()),
rpc_extensions_builder: Box::new(|_, _| ()),
telemetry_connection_sinks: sc_service::TelemetryConnectionSinks::default(),
config,
client,
Expand Down
1 change: 1 addition & 0 deletions bin/node/cli/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ serde = { version = "1.0.102", features = ["derive"] }
futures = { version = "0.3.1", features = ["compat"] }
hex-literal = "0.2.1"
jsonrpc-core = "14.2.0"
jsonrpc-pubsub = "14.2.0"
log = "0.4.8"
rand = "0.7.2"
structopt = { version = "0.3.8", optional = true }
Expand Down
10 changes: 8 additions & 2 deletions bin/node/cli/src/service.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,10 @@ pub fn new_partial(config: &Configuration) -> Result<sc_service::PartialComponen
sp_consensus::DefaultImportQueue<Block, FullClient>,
sc_transaction_pool::FullPool<Block, FullClient>,
(
impl Fn(node_rpc::DenyUnsafe) -> node_rpc::IoHandler,
impl Fn(
node_rpc::DenyUnsafe,
jsonrpc_pubsub::manager::SubscriptionManager
) -> node_rpc::IoHandler,
(
sc_consensus_babe::BabeBlockImport<Block, FullClient, FullGrandpaBlockImport>,
grandpa::LinkHalf<Block, FullClient, FullSelectChain>,
Expand Down Expand Up @@ -101,6 +104,7 @@ pub fn new_partial(config: &Configuration) -> Result<sc_service::PartialComponen
let (rpc_extensions_builder, rpc_setup) = {
let (_, grandpa_link, babe_link) = &import_setup;

let justification_stream = grandpa_link.justification_stream();
let shared_authority_set = grandpa_link.shared_authority_set().clone();
let shared_voter_state = grandpa::SharedVoterState::empty();

Expand All @@ -114,7 +118,7 @@ pub fn new_partial(config: &Configuration) -> Result<sc_service::PartialComponen
let select_chain = select_chain.clone();
let keystore = keystore.clone();

let rpc_extensions_builder = move |deny_unsafe| {
let rpc_extensions_builder = move |deny_unsafe, subscriptions| {
let deps = node_rpc::FullDeps {
client: client.clone(),
pool: pool.clone(),
Expand All @@ -128,6 +132,8 @@ pub fn new_partial(config: &Configuration) -> Result<sc_service::PartialComponen
grandpa: node_rpc::GrandpaDeps {
shared_voter_state: shared_voter_state.clone(),
shared_authority_set: shared_authority_set.clone(),
justification_stream: justification_stream.clone(),
subscriptions,
},
};

Expand Down
20 changes: 10 additions & 10 deletions bin/node/rpc/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,25 @@ repository = "https://github.com/paritytech/substrate/"
targets = ["x86_64-unknown-linux-gnu"]

[dependencies]
sc-client-api = { version = "2.0.0-rc5", path = "../../../client/api" }
sc-rpc = { version = "2.0.0-rc5", path = "../../../client/rpc" }
jsonrpc-core = "14.2.0"
jsonrpc-pubsub = "14.2.0"
node-primitives = { version = "2.0.0-rc5", path = "../primitives" }
node-runtime = { version = "2.0.0-rc5", path = "../runtime" }
sp-runtime = { version = "2.0.0-rc5", path = "../../../primitives/runtime" }
sp-api = { version = "2.0.0-rc5", path = "../../../primitives/api" }
pallet-contracts-rpc = { version = "0.8.0-rc5", path = "../../../frame/contracts/rpc/" }
pallet-transaction-payment-rpc = { version = "2.0.0-rc5", path = "../../../frame/transaction-payment/rpc/" }
substrate-frame-rpc-system = { version = "2.0.0-rc5", path = "../../../utils/frame/rpc/system" }
sp-transaction-pool = { version = "2.0.0-rc5", path = "../../../primitives/transaction-pool" }
sc-client-api = { version = "2.0.0-rc5", path = "../../../client/api" }
sc-consensus-babe = { version = "0.8.0-rc5", path = "../../../client/consensus/babe" }
sc-consensus-babe-rpc = { version = "0.8.0-rc5", path = "../../../client/consensus/babe/rpc" }
sp-consensus-babe = { version = "0.8.0-rc5", path = "../../../primitives/consensus/babe" }
sc-keystore = { version = "2.0.0-rc5", path = "../../../client/keystore" }
sc-consensus-epochs = { version = "0.8.0-rc5", path = "../../../client/consensus/epochs" }
sp-consensus = { version = "0.8.0-rc5", path = "../../../primitives/consensus/common" }
sp-blockchain = { version = "2.0.0-rc5", path = "../../../primitives/blockchain" }
sc-finality-grandpa = { version = "0.8.0-rc5", path = "../../../client/finality-grandpa" }
sc-finality-grandpa-rpc = { version = "0.8.0-rc5", path = "../../../client/finality-grandpa/rpc" }
sc-keystore = { version = "2.0.0-rc5", path = "../../../client/keystore" }
sc-rpc-api = { version = "0.8.0-rc5", path = "../../../client/rpc-api" }
sc-rpc = { version = "2.0.0-rc5", path = "../../../client/rpc" }
sp-api = { version = "2.0.0-rc5", path = "../../../primitives/api" }
sp-block-builder = { version = "2.0.0-rc5", path = "../../../primitives/block-builder" }
sp-blockchain = { version = "2.0.0-rc5", path = "../../../primitives/blockchain" }
sp-consensus = { version = "0.8.0-rc5", path = "../../../primitives/consensus/common" }
sp-consensus-babe = { version = "0.8.0-rc5", path = "../../../primitives/consensus/babe" }
sp-transaction-pool = { version = "2.0.0-rc5", path = "../../../primitives/transaction-pool" }
substrate-frame-rpc-system = { version = "2.0.0-rc5", path = "../../../utils/frame/rpc/system" }
38 changes: 25 additions & 13 deletions bin/node/rpc/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,21 @@

use std::sync::Arc;

use jsonrpc_pubsub::manager::SubscriptionManager;
use node_primitives::{Block, BlockNumber, AccountId, Index, Balance, Hash};
use sp_api::ProvideRuntimeApi;
use sp_transaction_pool::TransactionPool;
use sp_blockchain::{Error as BlockChainError, HeaderMetadata, HeaderBackend};
use sp_consensus::SelectChain;
use sc_keystore::KeyStorePtr;
use sp_consensus_babe::BabeApi;
use sc_consensus_epochs::SharedEpochChanges;
use sc_consensus_babe::{Config, Epoch};
use sc_consensus_babe_rpc::BabeRpcHandler;
use sc_finality_grandpa::{SharedVoterState, SharedAuthoritySet};
use sc_consensus_epochs::SharedEpochChanges;
use sc_finality_grandpa::{SharedVoterState, SharedAuthoritySet, GrandpaJustificationStream};
use sc_finality_grandpa_rpc::GrandpaRpcHandler;
use sp_block_builder::BlockBuilder;
use sc_keystore::KeyStorePtr;
pub use sc_rpc_api::DenyUnsafe;
use sp_api::ProvideRuntimeApi;
use sp_block_builder::BlockBuilder;
use sp_blockchain::{Error as BlockChainError, HeaderMetadata, HeaderBackend};
use sp_consensus::SelectChain;
use sp_consensus_babe::BabeApi;
use sp_transaction_pool::TransactionPool;

/// Light client extra dependencies.
pub struct LightDeps<C, F, P> {
Expand Down Expand Up @@ -75,6 +76,10 @@ pub struct GrandpaDeps {
pub shared_voter_state: SharedVoterState,
/// Authority set info.
pub shared_authority_set: SharedAuthoritySet<Hash, BlockNumber>,
/// Receives notifications about justification events from Grandpa.
pub justification_stream: GrandpaJustificationStream<Block>,
/// Subscription manager to keep track of pubsub subscribers.
pub subscriptions: SubscriptionManager,
}

/// Full client dependencies.
Expand All @@ -97,9 +102,9 @@ pub struct FullDeps<C, P, SC> {
pub type IoHandler = jsonrpc_core::IoHandler<sc_rpc::Metadata>;

/// Instantiate all Full RPC extensions.
pub fn create_full<C, P, M, SC>(
pub fn create_full<C, P, SC>(
deps: FullDeps<C, P, SC>,
) -> jsonrpc_core::IoHandler<M> where
) -> jsonrpc_core::IoHandler<sc_rpc_api::Metadata> where
C: ProvideRuntimeApi<Block>,
C: HeaderBackend<Block> + HeaderMetadata<Block, Error=BlockChainError> + 'static,
C: Send + Sync + 'static,
Expand All @@ -109,7 +114,6 @@ pub fn create_full<C, P, M, SC>(
C::Api: BabeApi<Block>,
C::Api: BlockBuilder<Block>,
P: TransactionPool + 'static,
M: jsonrpc_core::Metadata + Default,
SC: SelectChain<Block> +'static,
{
use substrate_frame_rpc_system::{FullSystem, SystemApi};
Expand All @@ -125,6 +129,7 @@ pub fn create_full<C, P, M, SC>(
babe,
grandpa,
} = deps;

let BabeDeps {
keystore,
babe_config,
Expand All @@ -133,6 +138,8 @@ pub fn create_full<C, P, M, SC>(
let GrandpaDeps {
shared_voter_state,
shared_authority_set,
justification_stream,
subscriptions,
} = grandpa;

io.extend_with(
Expand Down Expand Up @@ -161,7 +168,12 @@ pub fn create_full<C, P, M, SC>(
);
io.extend_with(
sc_finality_grandpa_rpc::GrandpaApi::to_delegate(
GrandpaRpcHandler::new(shared_authority_set, shared_voter_state)
GrandpaRpcHandler::new(
shared_authority_set,
shared_voter_state,
justification_stream,
subscriptions,
)
)
);

Expand Down
13 changes: 13 additions & 0 deletions client/finality-grandpa/rpc/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,29 @@ edition = "2018"
license = "GPL-3.0-or-later WITH Classpath-exception-2.0"

[dependencies]
sc-rpc = { version = "2.0.0-rc5", path = "../../rpc" }
sp-runtime = { version = "2.0.0-rc5", path = "../../../primitives/runtime" }
sc-finality-grandpa = { version = "0.8.0-rc5", path = "../" }
finality-grandpa = { version = "0.12.3", features = ["derive-codec"] }
jsonrpc-core = "14.2.0"
jsonrpc-core-client = "14.2.0"
jsonrpc-derive = "14.2.1"
jsonrpc-pubsub = "14.2.0"
futures = { version = "0.3.4", features = ["compat"] }
serde = { version = "1.0.105", features = ["derive"] }
serde_json = "1.0.50"
log = "0.4.8"
derive_more = "0.99.2"
parity-scale-codec = { version = "1.3.0", features = ["derive"] }

[dev-dependencies]
sc-block-builder = { version = "0.8.0-rc5", path = "../../block-builder" }
sc-network-test = { version = "0.8.0-rc5", path = "../../network/test" }
sc-rpc = { version = "2.0.0-rc5", path = "../../rpc", features = ["test-helpers"] }
sp-blockchain = { version = "2.0.0-rc5", path = "../../../primitives/blockchain" }
sp-consensus = { version = "0.8.0-rc5", path = "../../../primitives/consensus/common" }
sp-core = { version = "2.0.0-rc5", path = "../../../primitives/core" }
sp-finality-grandpa = { version = "2.0.0-rc5", path = "../../../primitives/finality-grandpa" }
sp-keyring = { version = "2.0.0-rc5", path = "../../../primitives/keyring" }
substrate-test-runtime-client = { version = "2.0.0-rc5", path = "../../../test-utils/runtime/client" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is all of these really needed? Doesn't substrate-test-runtime-client re-export some of this stuff?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

substrate-test-client re-exports Ed25519Keyring, but that seems like an even bigger dependency? But maybe it's more correct to swap sp-keyring for substrate-test-client?

The sp-finality-grandpa dependency is for localized_payload which I'm not seeing being re-exported

lazy_static = "1.4"
Loading