From 5efc3f6083608ff5bb38f196fcad2a9b4248daf8 Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Mon, 14 Oct 2019 15:33:47 -0400 Subject: [PATCH 01/26] rename: weight -> fees --- docs/development/module/{weight.md => fees.md} | 0 website/sidebars.json | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) rename docs/development/module/{weight.md => fees.md} (100%) diff --git a/docs/development/module/weight.md b/docs/development/module/fees.md similarity index 100% rename from docs/development/module/weight.md rename to docs/development/module/fees.md diff --git a/website/sidebars.json b/website/sidebars.json index 0a0250d238..2aa4ce683c 100644 --- a/website/sidebars.json +++ b/website/sidebars.json @@ -69,7 +69,7 @@ "development/module/events", "development/module/errors", "development/module/traits", - "development/module/weight", + "development/module/fees", "development/module/tests" ] }, From f38ac5e4264ffc7616dd0d62b6784f1549bf5337 Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Mon, 14 Oct 2019 16:00:40 -0400 Subject: [PATCH 02/26] Braindump --- docs/development/module/fees.md | 40 ++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 50b91264b7..31cf041275 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -1,13 +1,41 @@ --- -title: Transaction Weight +title: Transaction Fees --- -* What is the transaction weight? +When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, the cost nodes incur to execute a transaction must be covered by the submitter of the transaction. The cost to execute transactions varies over orders of magnitude, and thus Substrate provides a flexible mechanism for characterizing the total cost of a transaction. -* How does it work? +## Formula +The fee to execute a transaction consists of three parts +``` +total_fee = base_fee + length_fee * length + weight_fee * weight +``` -* What kinds of weights can I add to my runtime functions? +* `base_fee` a fixed fee that is applied to every single transaction. +* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the serialized transaction +* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. Transaction weighing is highly customizable. -* What is the default weight if left undefined? +> Transactors may also include optional tips in transactions to incentivize validators to include the transactions faster. Tips are a separate concept and are not covered here. -* How should I measure the "weight" of a function? + +## Weight Basics +Transaction weights allow developers to express the cost of executing a transaction as a function of it's arguments. In many cases the computational complexity of a function can be captured in terms of the arguments. Some theoretical and practical advice for doing so is given below. + +How theoretical do we want to be vs how practical? + +## Examples +O(n) in an integer parameter +link to recipes + +## Limitations +some operations get more expensive as storage grows and shrinks. +Is the weighting function open to change through governance short of a runtime upgrade? + +## What is the default weight if left undefined? +I think it's `FreeNormal` + +## How should I measure the "weight" of a function? +@kianenigma Can you explain the fundamentals of what you did for https://docs.google.com/spreadsheets/d/1h0RqncdqiWI4KgxO0z9JIpZEJESXjX_ZCK6LFX6veDo/edit?ts=5d8b02b7#gid=765851977 + +Material to link +https://en.wikipedia.org/wiki/Analysis_of_algorithms +https://en.wikipedia.org/wiki/Computational_complexity From e6fcd72f1d79b3789105c527c86feb8e52f58867 Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Mon, 14 Oct 2019 16:10:50 -0400 Subject: [PATCH 03/26] More brain dump --- docs/development/module/fees.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 31cf041275..d426c0482d 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -4,10 +4,11 @@ title: Transaction Fees When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, the cost nodes incur to execute a transaction must be covered by the submitter of the transaction. The cost to execute transactions varies over orders of magnitude, and thus Substrate provides a flexible mechanism for characterizing the total cost of a transaction. -## Formula +## Fees Calculation The fee to execute a transaction consists of three parts + ``` -total_fee = base_fee + length_fee * length + weight_fee * weight +total_fee = base_fee + length_fee * length + weight_multiplier * weight ``` * `base_fee` a fixed fee that is applied to every single transaction. @@ -16,11 +17,20 @@ total_fee = base_fee + length_fee * length + weight_fee * weight > Transactors may also include optional tips in transactions to incentivize validators to include the transactions faster. Tips are a separate concept and are not covered here. +## Adjusting Multipliers +The `base_fee` and `byte_fee` are constants in the Transaction Fees module. The weight multiplier seems to adjust dynamically. @kainenigma can you explain that. ## Weight Basics -Transaction weights allow developers to express the cost of executing a transaction as a function of it's arguments. In many cases the computational complexity of a function can be captured in terms of the arguments. Some theoretical and practical advice for doing so is given below. +Transaction weights allow developers to express the cost of executing a transaction as a function of it's arguments. Some theoretical and practical advice for doing so is given below. + +## Dispatch Parameters +Every transaction is classified as either Normal +FixedNormal +FreeNormal +MaxNormal + +> Here we've presented the Normal variants. There is also `FixedOperational` and its relatives. Explain the difference and decide whether this article should be about dispatch info more generally. -How theoretical do we want to be vs how practical? ## Examples O(n) in an integer parameter From 3d7cc069241c03ba74f298964e5627446e1c4da9 Mon Sep 17 00:00:00 2001 From: kianenigma Date: Tue, 22 Oct 2019 20:48:53 +0200 Subject: [PATCH 04/26] A draft of content by kian --- docs/development/module/fees.md | 120 ++++++++++++++++++++++++++------ 1 file changed, 98 insertions(+), 22 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index d426c0482d..f9d8a1a990 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -1,50 +1,126 @@ --- -title: Transaction Fees +title: Transaction Weight and Fees --- -When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, the cost nodes incur to execute a transaction must be covered by the submitter of the transaction. The cost to execute transactions varies over orders of magnitude, and thus Substrate provides a flexible mechanism for characterizing the total cost of a transaction. +When transactions are submitted to a blockchain, they are executed by the nodes in the network. To +be economically sustainable, the cost nodes incur to execute a transaction must be covered by the +submitter of the transaction. The cost to execute transactions could vary over orders of magnitude, +and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, +cost of a transaction in order to be included into a block. + +Furthermore, a number of resources in any chain can be limited by nature such as time, memory and +computation. A mechanism should exist to prevent individual resource-consuming components of the +chain, including dispatchables, to consume too much of any of the mentioned resources. This concept +is represented in substrate by weights. + +## Weights Basics + +The fee system is dependent on the weight system. Hence, we will first briefly describe the weights. + +As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. However, +at the time of this writing, substrate weights are simply a numeric value (todo: link). Each +dispatchable function is given a weight using the `#[weight = $x]` annotation, where `$x` is some +function capable of determining the weight of the dispatch. `$x` can access and examine the +arguments of the dispatch. Nonetheless, a weight calculation should always: +- Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine + the weight of a dispatchable before actually deciding to accept it or not. +- While not enforced, calculating the weight should not consume much resource itself. It does not + make much sense to spend almost the same amount of time as dispatching a transaction to estimate + how much weight it will consume. Thus, weight computation should typically be very fast, much + faster than dispatching. This is analogous to a `poll()` function of the `Future` trait. It really + doesn't make sense to poll a future if it is going to stall us until the future itself is + resolved; A sensible `poll` must return fast. +- Finally, if a transaction is very variable in the amount of resources it consumes, it should + manually prevent restrict certain dispatches through any means that makes sense to that particular + chain. An example of such dispatches in substrate is the call to a smart contract (todo: link when + this is implemented). + +TODO: a very small paragraph linking to _how to denote the weights_ in your custom runtime either +via using the simple default stuff of substrate or Joshy's example or the `srml/example` also has +some good material. Link and don't write too much. + +The system module is responsible for storing the weight of each block and making sure that it does +not exceed the limit (todo: link to section down). The transaction-payment module is responsible for +interpreting these weights and deducting fees based upon them. ## Fees Calculation -The fee to execute a transaction consists of three parts -``` -total_fee = base_fee + length_fee * length + weight_multiplier * weight -``` +The fee to include a transaction consists of three parts: * `base_fee` a fixed fee that is applied to every single transaction. -* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the serialized transaction -* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. Transaction weighing is highly customizable. +* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded + transaction. +* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As + mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the + weight, it must be converted to a deductible `balance` type (typically denoted by a module that + implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a + `WeightToFee` type that makes the conversion. -> Transactors may also include optional tips in transactions to incentivize validators to include the transactions faster. Tips are a separate concept and are not covered here. +TODO: all three items need linkage to proper parameter type. Generally everything can use better linking. -## Adjusting Multipliers -The `base_fee` and `byte_fee` are constants in the Transaction Fees module. The weight multiplier seems to adjust dynamically. @kainenigma can you explain that. +> Transactors may also include optional tips in transactions to incentivize validators to include +> the transactions faster. Tips will include the chance of inclusion. More details are outside the +> scope of this writing. -## Weight Basics -Transaction weights allow developers to express the cost of executing a transaction as a function of it's arguments. Some theoretical and practical advice for doing so is given below. +based on the above, the final fee of a dispatchable is: -## Dispatch Parameters -Every transaction is classified as either Normal -FixedNormal -FreeNormal -MaxNormal +`fee = base_fee + len(tx) * length_fee + WeightToFee(weight)` -> Here we've presented the Normal variants. There is also `FixedOperational` and its relatives. Explain the difference and decide whether this article should be about dispatch info more generally. +## Adjusting Multiplier +The above formula gives a fee which is constant through time. Except for transaction version updates +or minor changes in length due to different arguments, any dispatchable, given the same inputs, +_will always incur the same cost_. This might not always be desirable. Chains might need to increase +or decrease fees based on sime circumstance. To fulfill this requirement, Substrate provides: + - a multiplier stored in the system module that is applied to the outcome of the above formula by + default (needless to say, the default value of which is `One`, meaning that it has no effect) + - a configurable parameter for a runtime to describe how this multiplier can change. + TODO more about it should be said + links + +## More on Weights: Block Weight and Length Limit +Must cover: +- dispatch class and how it must be implemented (briefly, similar to above just link to a + tutorial or srml/example) +- MaximumBlockWeight + Maximum Block Length + how a ratio of it is for operational ones. +- glue it together by linking to CheckWeight + +> Here we've presented the Normal variants. There is also `FixedOperational` and its relatives. +> Explain the difference and decide whether this article should be about dispatch info more +> generally. + +## Gluing it Together: How Things Work in Substrate +explain: +- the implementation of our targetedMultiplierAdjustment +- our weightToFee +- our SimpleDispatchInfo + +Basically until this point all links should be to traits, now we link them to implementations. +Again, a good place to link to an example of how you can ditch our SimpleDispatchInfo and make your +own. ## Examples O(n) in an integer parameter link to recipes ## Limitations -some operations get more expensive as storage grows and shrinks. -Is the weighting function open to change through governance short of a runtime upgrade? +Some operations get more expensive as storage grows and shrinks. + - Yes and a static weight will not help with that. As mentioned, weight should be known rapidly + ahead of dispatch. In this case the implementation of the dispatch should for example charge + manually more money etc. +Is the weighting function open to change through governance short of a + runtime upgrade? + - Yes, it is part of a runtime upgrade. A upgrade can just update weight functions. ## What is the default weight if left undefined? I think it's `FreeNormal` +Kian: can remove this and cover in the section `## Gluing it Together: How Things Work in Substrate` + ## How should I measure the "weight" of a function? -@kianenigma Can you explain the fundamentals of what you did for https://docs.google.com/spreadsheets/d/1h0RqncdqiWI4KgxO0z9JIpZEJESXjX_ZCK6LFX6veDo/edit?ts=5d8b02b7#gid=765851977 +@kianenigma Can you explain the fundamentals of what you did for +https://docs.google.com/spreadsheets/d/1h0RqncdqiWI4KgxO0z9JIpZEJESXjX_ZCK6LFX6veDo/edit?ts=5d8b02b7#gid=765851977 +Kian: I will fill this in later in the section `## Gluing it Together: How Things Work in +Substrate`. But I don't like to imply that everyone should do that, this is merely one example. Material to link https://en.wikipedia.org/wiki/Analysis_of_algorithms From ddcae11b0d12ac6f86939a438d2cd654d8cbbcf0 Mon Sep 17 00:00:00 2001 From: kianenigma Date: Thu, 24 Oct 2019 13:49:59 +0200 Subject: [PATCH 05/26] more sutff --- docs/development/module/fees.md | 187 ++++++++++++++++++-------------- 1 file changed, 107 insertions(+), 80 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index f9d8a1a990..8ec2d9d4d1 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -8,20 +8,24 @@ submitter of the transaction. The cost to execute transactions could vary over o and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -Furthermore, a number of resources in any chain can be limited by nature such as time, memory and -computation. A mechanism should exist to prevent individual resource-consuming components of the -chain, including dispatchables, to consume too much of any of the mentioned resources. This concept -is represented in substrate by weights. +Furthermore, a number of resources in any chain can be limited by definition such as time, memory +and computation. A mechanism should exist to prevent individual resource-consuming components of the +chain, including but not limited to dispatchables, to consume too much of any of the mentioned +resources. This concept is represented in substrate by weights. ## Weights Basics -The fee system is dependent on the weight system. Hence, we will first briefly describe the weights. - -As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. However, -at the time of this writing, substrate weights are simply a numeric value (todo: link). Each -dispatchable function is given a weight using the `#[weight = $x]` annotation, where `$x` is some -function capable of determining the weight of the dispatch. `$x` can access and examine the -arguments of the dispatch. Nonetheless, a weight calculation should always: +The fee system is dependent on the +[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. Hence, we will first +briefly describe how weights work. + +As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom +implementation may use complex structures to demonstrate this. At the time of this writing, +substrate weights are simply a [numeric +value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Ea1ch dispatchable function +is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of +determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. +Nonetheless, a weight calculation should always: - Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine the weight of a dispatchable before actually deciding to accept it or not. - While not enforced, calculating the weight should not consume much resource itself. It does not @@ -35,93 +39,116 @@ arguments of the dispatch. Nonetheless, a weight calculation should always: chain. An example of such dispatches in substrate is the call to a smart contract (todo: link when this is implemented). -TODO: a very small paragraph linking to _how to denote the weights_ in your custom runtime either -via using the simple default stuff of substrate or Joshy's example or the `srml/example` also has -some good material. Link and don't write too much. - -The system module is responsible for storing the weight of each block and making sure that it does -not exceed the limit (todo: link to section down). The transaction-payment module is responsible for +Given the above details, one can write their own weight computation function, `$x` in the above +listing, to determine the weight of dispatchable. The programming details of doing this is beyond +the scope of this document. For more information on that, refer to: +- the [weight module's](https://crates.parity.io/sr_primitives/weights/index.html) documentation in + substrate-primitives. +- the [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) + module and the inline documentation. Skim the code for `#[weight]` attributes and all the related + documentations. +- the section on weights in Substrate recipes. TODO + +The system module is responsible for accumulating the weight of each block as it gets executed and +making sure that it does not exceed the limit. The transaction-payment module is responsible for interpreting these weights and deducting fees based upon them. ## Fees Calculation The fee to include a transaction consists of three parts: -* `base_fee` a fixed fee that is applied to every single transaction. +* `base_fee` a fixed fee that is applied to every single transaction. See + [`TransactionBaseFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) * `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded - transaction. + transaction. See + [`TransactionByteFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) * `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it must be converted to a deductible `balance` type (typically denoted by a module that implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a - `WeightToFee` type that makes the conversion. - -TODO: all three items need linkage to proper parameter type. Generally everything can use better linking. - -> Transactors may also include optional tips in transactions to incentivize validators to include -> the transactions faster. Tips will include the chance of inclusion. More details are outside the -> scope of this writing. + [`WeightToFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) + type that makes the conversion. based on the above, the final fee of a dispatchable is: -`fee = base_fee + len(tx) * length_fee + WeightToFee(weight)` +``` +fee = + base_fee + + len(tx) * length_fee + + WeightToFee(weight) +``` ## Adjusting Multiplier -The above formula gives a fee which is constant through time. Except for transaction version updates -or minor changes in length due to different arguments, any dispatchable, given the same inputs, -_will always incur the same cost_. This might not always be desirable. Chains might need to increase -or decrease fees based on sime circumstance. To fulfill this requirement, Substrate provides: - - a multiplier stored in the system module that is applied to the outcome of the above formula by - default (needless to say, the default value of which is `One`, meaning that it has no effect) - - a configurable parameter for a runtime to describe how this multiplier can change. - - TODO more about it should be said + links +The above formula gives a fee which is __logically constant through time__. Of course, the weight +can, in practice, be dynamic and based on what `WeightToFee` is defined to be, the final fee can +have some degree of variability. As for the length fee, the inputs of the transaction could change +the length and hence affecting the length fee. Nonetheless, these changes are independent and a +_general update logic to the entire fee cannot be composed out of them trivially_. In other words, +for any dispatchable, given the same inputs, _it will always incur the same cost_. This might not +always be desirable. Chains might need to increase or decrease fees based on sime condition. To +fulfill this requirement, Substrate provides: + - a multiplier stored in the transaction-payment module that is applied to the outcome of the + above formula by default (needless to say, the default value of which is _multiplication + identity_, meaning that it has no effect). This is stored in + [`NextFeeMultiplier`](https://crates.parity.io/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) + storage. + - a configurable parameter for a runtime to describe how this multiplier can change. This is + expressed via + [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) ## More on Weights: Block Weight and Length Limit -Must cover: -- dispatch class and how it must be implemented (briefly, similar to above just link to a - tutorial or srml/example) -- MaximumBlockWeight + Maximum Block Length + how a ratio of it is for operational ones. -- glue it together by linking to CheckWeight -> Here we've presented the Normal variants. There is also `FixedOperational` and its relatives. -> Explain the difference and decide whether this article should be about dispatch info more -> generally. +Aside from affecting the fees, the main purpose of the weight system is to prevent a block from +being too filled with transactions. The system module, while processing blocks≠transactions within a +block, accumulates both the total length of the block (sum of encoded transactions in number of +bytes) and the total weight of the blocks. At any points, if these numbers surpass the limits, no +further transactions are accepted to the block. These limits are defined in +[`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +and +[`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +respectively. + +One important note about these limits is that a ratio of them are reserved for the `Operational` +dispatch class. This rule applies to both of the limits and the ratio can be found in +[`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). +For example, if the block length limit is 1 mega bytes and the ratio is set the 80%, all +transactions can fill the first 800 kilo bytes of the block while the last 200 can only be filled by +the operation class. + ## Gluing it Together: How Things Work in Substrate -explain: -- the implementation of our targetedMultiplierAdjustment -- our weightToFee -- our SimpleDispatchInfo - -Basically until this point all links should be to traits, now we link them to implementations. -Again, a good place to link to an example of how you can ditch our SimpleDispatchInfo and make your -own. - -## Examples -O(n) in an integer parameter -link to recipes - -## Limitations -Some operations get more expensive as storage grows and shrinks. - - Yes and a static weight will not help with that. As mentioned, weight should be known rapidly - ahead of dispatch. In this case the implementation of the dispatch should for example charge - manually more money etc. -Is the weighting function open to change through governance short of a - runtime upgrade? - - Yes, it is part of a runtime upgrade. A upgrade can just update weight functions. - -## What is the default weight if left undefined? -I think it's `FreeNormal` -Kian: can remove this and cover in the section `## Gluing it Together: How Things Work in Substrate` - - -## How should I measure the "weight" of a function? -@kianenigma Can you explain the fundamentals of what you did for -https://docs.google.com/spreadsheets/d/1h0RqncdqiWI4KgxO0z9JIpZEJESXjX_ZCK6LFX6veDo/edit?ts=5d8b02b7#gid=765851977 -Kian: I will fill this in later in the section `## Gluing it Together: How Things Work in -Substrate`. But I don't like to imply that everyone should do that, this is merely one example. - -Material to link -https://en.wikipedia.org/wiki/Analysis_of_algorithms -https://en.wikipedia.org/wiki/Computational_complexity + +All of the above parameters are configurable and having a look at the runtime crate in substrate +should demonstrate the concrete values. + +The configurable values (such as `MaximumBlockWeight`) are created using the `parameter_types!` +macro and passed to the associated module (`system` and `transaction-payment`). + +The default and minimum `Weight` is defined as `FixedNormal(10_000)` and the `WeightToFee` is +configured to map this value to `10^7` as the minimum weight fee of substrate. + +The update function is inspired by the Polkadot network and implements a targeted adjustment in +which a target saturation level of block weight is defined. If the previous block is more saturated, +then the fees are slightly increases. Similarly, if less transaction than the target are in the +previous block, fees are decreased by a small amount. More information about this can be found in +the [web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). + +> Note that this section demonstrates the configuration of the fee/weight system at the time of this +> writing in `substrate-node`. This can change over time and any other substrate chain is indeed +> encouraged to parameterize its fee system according to its own logic and economic incentives. + +## Frequent Questions + +> _Some operations get more expensive as storage grows and shrinks. How can the weight system +> represent that?_ + +As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking the storage and +analyzing it just to determine how much weight might be consumed does not fit this definition quite +well. In this case the implementation of the dispatch should take the state of the change into +account and manually take extra fees, bonds or take any other measures to make sure that the +transaction is safe. + +> _Is the weighting function open to change through governance short of a runtime upgrade?_ + +Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as +well. A upgrade can just update weight functions if needed. From b23e753eb7dc551e2f309e34eafb8000bbdc3f67 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 24 Oct 2019 13:57:18 +0200 Subject: [PATCH 06/26] Format line length, add next steps --- docs/development/module/fees.md | 235 +++++++++++++++++++------------- 1 file changed, 137 insertions(+), 98 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 8ec2d9d4d1..74acb98f1e 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -2,56 +2,66 @@ title: Transaction Weight and Fees --- -When transactions are submitted to a blockchain, they are executed by the nodes in the network. To -be economically sustainable, the cost nodes incur to execute a transaction must be covered by the -submitter of the transaction. The cost to execute transactions could vary over orders of magnitude, -and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, +When transactions are submitted to a blockchain, they are executed by the nodes +in the network. To be economically sustainable, the cost nodes incur to execute +a transaction must be covered by the submitter of the transaction. The cost to +execute transactions could vary over orders of magnitude, and thus Substrate +provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -Furthermore, a number of resources in any chain can be limited by definition such as time, memory -and computation. A mechanism should exist to prevent individual resource-consuming components of the -chain, including but not limited to dispatchables, to consume too much of any of the mentioned -resources. This concept is represented in substrate by weights. +Furthermore, a number of resources in any chain can be limited by definition +such as time, memory and computation. A mechanism should exist to prevent +individual resource-consuming components of the chain, including but not limited +to dispatchables, to consume too much of any of the mentioned resources. This +concept is represented in substrate by weights. ## Weights Basics The fee system is dependent on the -[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. Hence, we will first -briefly describe how weights work. - -As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom -implementation may use complex structures to demonstrate this. At the time of this writing, -substrate weights are simply a [numeric -value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Ea1ch dispatchable function -is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of -determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. -Nonetheless, a weight calculation should always: -- Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine - the weight of a dispatchable before actually deciding to accept it or not. -- While not enforced, calculating the weight should not consume much resource itself. It does not - make much sense to spend almost the same amount of time as dispatching a transaction to estimate - how much weight it will consume. Thus, weight computation should typically be very fast, much - faster than dispatching. This is analogous to a `poll()` function of the `Future` trait. It really - doesn't make sense to poll a future if it is going to stall us until the future itself is - resolved; A sensible `poll` must return fast. -- Finally, if a transaction is very variable in the amount of resources it consumes, it should - manually prevent restrict certain dispatches through any means that makes sense to that particular - chain. An example of such dispatches in substrate is the call to a smart contract (todo: link when - this is implemented). - -Given the above details, one can write their own weight computation function, `$x` in the above -listing, to determine the weight of dispatchable. The programming details of doing this is beyond -the scope of this document. For more information on that, refer to: -- the [weight module's](https://crates.parity.io/sr_primitives/weights/index.html) documentation in - substrate-primitives. -- the [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) - module and the inline documentation. Skim the code for `#[weight]` attributes and all the related - documentations. +[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. +Hence, we will first briefly describe how weights work. + +As mentioned, weights can technically refer to, or represent, numerous _limited_ +resources. A custom implementation may use complex structures to demonstrate +this. At the time of this writing, substrate weights are simply a [numeric +value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Ea1ch +dispatchable function is given a weight using the `#[weight = $x]` annotation, +where `$x` is some function capable of determining the weight of the dispatch. +`$x` can access and examine the arguments of the dispatch. Nonetheless, a weight +calculation should always: +- Be computable __ahead of dispatch__. A block producer, or a validator, should + be able to examine the weight of a dispatchable before actually deciding to + accept it or not. +- While not enforced, calculating the weight should not consume much resource + itself. It does not make much sense to spend almost the same amount of time as + dispatching a transaction to estimate how much weight it will consume. Thus, + weight computation should typically be very fast, much faster than + dispatching. This is analogous to a `poll()` function of the `Future` trait. + It really doesn't make sense to poll a future if it is going to stall us until + the future itself is resolved; A sensible `poll` must return fast. +- Finally, if a transaction is very variable in the amount of resources it + consumes, it should manually prevent restrict certain dispatches through any + means that makes sense to that particular chain. An example of such dispatches + in substrate is the call to a smart contract (todo: link when this is + implemented). + +Given the above details, one can write their own weight computation function, +`$x` in the above listing, to determine the weight of dispatchable. The +programming details of doing this is beyond the scope of this document. For more +information on that, refer to: +- the [weight + module's](https://crates.parity.io/sr_primitives/weights/index.html) + documentation in substrate-primitives. +- the + [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) + module and the inline documentation. Skim the code for `#[weight]` attributes + and all the related documentations. - the section on weights in Substrate recipes. TODO -The system module is responsible for accumulating the weight of each block as it gets executed and -making sure that it does not exceed the limit. The transaction-payment module is responsible for -interpreting these weights and deducting fees based upon them. +The system module is responsible for accumulating the weight of each block as it +gets executed and making sure that it does not exceed the limit. The +transaction-payment module is responsible for interpreting these weights and +deducting fees based upon them. ## Fees Calculation @@ -59,13 +69,15 @@ The fee to include a transaction consists of three parts: * `base_fee` a fixed fee that is applied to every single transaction. See [`TransactionBaseFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) -* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded - transaction. See +* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the + encoded transaction. See [`TransactionByteFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) -* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As - mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the - weight, it must be converted to a deductible `balance` type (typically denoted by a module that - implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a +* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the + transaction. As mentioned, weight of each dispatch is denoted via the flexible + `#[weight]` annotation. Knowing the weight, it must be converted to a + deductible `balance` type (typically denoted by a module that implements + `Currency`, `srml-balances` in substrate node). For this, each runtime must + define a [`WeightToFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) type that makes the conversion. @@ -73,82 +85,109 @@ based on the above, the final fee of a dispatchable is: ``` fee = - base_fee + - len(tx) * length_fee + - WeightToFee(weight) + base_fee + + len(tx) * length_fee + + WeightToFee(weight) ``` ## Adjusting Multiplier -The above formula gives a fee which is __logically constant through time__. Of course, the weight -can, in practice, be dynamic and based on what `WeightToFee` is defined to be, the final fee can -have some degree of variability. As for the length fee, the inputs of the transaction could change -the length and hence affecting the length fee. Nonetheless, these changes are independent and a -_general update logic to the entire fee cannot be composed out of them trivially_. In other words, -for any dispatchable, given the same inputs, _it will always incur the same cost_. This might not -always be desirable. Chains might need to increase or decrease fees based on sime condition. To -fulfill this requirement, Substrate provides: - - a multiplier stored in the transaction-payment module that is applied to the outcome of the - above formula by default (needless to say, the default value of which is _multiplication - identity_, meaning that it has no effect). This is stored in +The above formula gives a fee which is __logically constant through time__. Of +course, the weight can, in practice, be dynamic and based on what `WeightToFee` +is defined to be, the final fee can have some degree of variability. As for the +length fee, the inputs of the transaction could change the length and hence +affecting the length fee. Nonetheless, these changes are independent and a +_general update logic to the entire fee cannot be composed out of them +trivially_. In other words, for any dispatchable, given the same inputs, _it +will always incur the same cost_. This might not always be desirable. Chains +might need to increase or decrease fees based on sime condition. To fulfill +this requirement, Substrate provides: + - a multiplier stored in the transaction-payment module that is applied to the + outcome of the above formula by default (needless to say, the default value + of which is _multiplication identity_, meaning that it has no effect). This + is stored in [`NextFeeMultiplier`](https://crates.parity.io/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) storage. - - a configurable parameter for a runtime to describe how this multiplier can change. This is - expressed via + - a configurable parameter for a runtime to describe how this multiplier can + change. This is expressed via [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) ## More on Weights: Block Weight and Length Limit -Aside from affecting the fees, the main purpose of the weight system is to prevent a block from -being too filled with transactions. The system module, while processing blocks≠transactions within a -block, accumulates both the total length of the block (sum of encoded transactions in number of -bytes) and the total weight of the blocks. At any points, if these numbers surpass the limits, no -further transactions are accepted to the block. These limits are defined in +Aside from affecting the fees, the main purpose of the weight system is to +prevent a block from being too filled with transactions. The system module, +while processing blocks≠transactions within a block, accumulates both the total +length of the block (sum of encoded transactions in number of bytes) and the +total weight of the blocks. At any points, if these numbers surpass the limits, +no further transactions are accepted to the block. These limits are defined in [`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and [`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) respectively. -One important note about these limits is that a ratio of them are reserved for the `Operational` -dispatch class. This rule applies to both of the limits and the ratio can be found in +One important note about these limits is that a ratio of them are reserved for +the `Operational` dispatch class. This rule applies to both of the limits and +the ratio can be found in [`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). -For example, if the block length limit is 1 mega bytes and the ratio is set the 80%, all -transactions can fill the first 800 kilo bytes of the block while the last 200 can only be filled by -the operation class. +For example, if the block length limit is 1 mega bytes and the ratio is set the +80%, all transactions can fill the first 800 kilo bytes of the block while the +last 200 can only be filled by the operation class. ## Gluing it Together: How Things Work in Substrate -All of the above parameters are configurable and having a look at the runtime crate in substrate -should demonstrate the concrete values. +All of the above parameters are configurable and having a look at the runtime +crate in substrate should demonstrate the concrete values. -The configurable values (such as `MaximumBlockWeight`) are created using the `parameter_types!` -macro and passed to the associated module (`system` and `transaction-payment`). +The configurable values (such as `MaximumBlockWeight`) are created using the +`parameter_types!` macro and passed to the associated module (`system` and +`transaction-payment`). -The default and minimum `Weight` is defined as `FixedNormal(10_000)` and the `WeightToFee` is -configured to map this value to `10^7` as the minimum weight fee of substrate. +The default and minimum `Weight` is defined as `FixedNormal(10_000)` and the +`WeightToFee` is configured to map this value to `10^7` as the minimum weight +fee of substrate. -The update function is inspired by the Polkadot network and implements a targeted adjustment in -which a target saturation level of block weight is defined. If the previous block is more saturated, -then the fees are slightly increases. Similarly, if less transaction than the target are in the -previous block, fees are decreased by a small amount. More information about this can be found in -the [web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). +The update function is inspired by the Polkadot network and implements a +targeted adjustment in which a target saturation level of block weight is +defined. If the previous block is more saturated, then the fees are slightly +increases. Similarly, if less transaction than the target are in the previous +block, fees are decreased by a small amount. More information about this can be +found in the [web3 research +page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). -> Note that this section demonstrates the configuration of the fee/weight system at the time of this -> writing in `substrate-node`. This can change over time and any other substrate chain is indeed -> encouraged to parameterize its fee system according to its own logic and economic incentives. +> Note that this section demonstrates the configuration of the fee/weight system +> at the time of this writing in `substrate-node`. This can change over time and +> any other substrate chain is indeed encouraged to parameterize its fee system +> according to its own logic and economic incentives. ## Frequent Questions -> _Some operations get more expensive as storage grows and shrinks. How can the weight system -> represent that?_ +> _Some operations get more expensive as storage grows and shrinks. How can the +> weight system represent that?_ -As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking the storage and -analyzing it just to determine how much weight might be consumed does not fit this definition quite -well. In this case the implementation of the dispatch should take the state of the change into -account and manually take extra fees, bonds or take any other measures to make sure that the -transaction is safe. +As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking +the storage and analyzing it just to determine how much weight might be consumed +does not fit this definition quite well. In this case the implementation of the +dispatch should take the state of the change into account and manually take +extra fees, bonds or take any other measures to make sure that the transaction +is safe. -> _Is the weighting function open to change through governance short of a runtime upgrade?_ +> _Is the weighting function open to change through governance short of a +> runtime upgrade?_ + +Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to +the weight system as well. A upgrade can just update weight functions if needed. + +## Next Steps + +### Learn More + +TODO + +### Examples + +TODO + +### References + +TODO -Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as -well. A upgrade can just update weight functions if needed. From 7657db760557c678ce88a4e9c409117211d5cb66 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 24 Oct 2019 14:00:06 +0200 Subject: [PATCH 07/26] typo --- docs/development/module/fees.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 74acb98f1e..7850788c4f 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -24,7 +24,7 @@ Hence, we will first briefly describe how weights work. As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom implementation may use complex structures to demonstrate this. At the time of this writing, substrate weights are simply a [numeric -value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Ea1ch +value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each dispatchable function is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. Nonetheless, a weight From b0a967fb0adb7c3f53047b2c040e874cb7478581 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 24 Oct 2019 15:04:48 +0200 Subject: [PATCH 08/26] Suggested refactor --- docs/conceptual/runtime/weight.md | 120 ++++++++++++++++++++++++++++++ docs/development/module/fees.md | 120 ++++++------------------------ 2 files changed, 143 insertions(+), 97 deletions(-) create mode 100644 docs/conceptual/runtime/weight.md diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md new file mode 100644 index 0000000000..2a8229ae2e --- /dev/null +++ b/docs/conceptual/runtime/weight.md @@ -0,0 +1,120 @@ +--- +title: Transaction Weight and Fees +--- + +When transactions are submitted to a blockchain, they are executed by the nodes +in the network. To be economically sustainable, nodes charge a fee to execute a +transaction that must be covered by the submitter of the transaction. The cost +to execute transactions could vary over orders of magnitude, and thus Substrate +provides a flexible mechanism for characterizing the total, presumably minimum, +cost of a transaction in order to be included into a block. + +Furthermore, a number of resources in any chain can be limited by definition +such as time, memory and computation. A mechanism should exist to prevent +individual resource-consuming components of the chain, including but not limited +to dispatchables, to consume too much of any of the mentioned resources. This +concept is represented in substrate by weights. + +## Transaction Weight + +The fee system is dependent on the +[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. +Hence, we will first briefly describe how weights work. + +As mentioned, weights can technically refer to, or represent, numerous _limited_ +resources. A custom implementation may use complex structures to demonstrate +this. At the time of this writing, substrate weights are simply a [numeric +value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each +dispatchable function is given a weight using the `#[weight = $x]` annotation, +where `$x` is some function capable of determining the weight of the dispatch. +`$x` can access and examine the arguments of the dispatch. Nonetheless, a weight +calculation should always: +- Be computable __ahead of dispatch__. A block producer, or a validator, should + be able to examine the weight of a dispatchable before actually deciding to + accept it or not. +- While not enforced, calculating the weight should not consume much resource + itself. It does not make much sense to spend almost the same amount of time as + dispatching a transaction to estimate how much weight it will consume. Thus, + weight computation should typically be very fast, much faster than + dispatching. This is analogous to a `poll()` function of the `Future` trait. + It really doesn't make sense to poll a future if it is going to stall us until + the future itself is resolved; A sensible `poll` must return fast. +- Finally, if a transaction is very variable in the amount of resources it + consumes, it should manually prevent restrict certain dispatches through any + means that makes sense to that particular chain. An example of such dispatches + in substrate is the call to a smart contract (todo: link when this is + implemented). + +Given the above details, one can write their own weight computation function, +`$x` in the above listing, to determine the weight of dispatchable. The +programming details of doing this is beyond the scope of this document. For more +information on that, refer to: +- the [weight + module's](https://crates.parity.io/sr_primitives/weights/index.html) + documentation in substrate-primitives. +- the + [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) + module and the inline documentation. Skim the code for `#[weight]` attributes + and all the related documentations. +- the section on weights in Substrate recipes. TODO + +The system module is responsible for accumulating the weight of each block as it +gets executed and making sure that it does not exceed the limit. The +transaction-payment module is responsible for interpreting these weights and +deducting fees based upon them. + + +## More on Weights: Block Weight and Length Limit + +Aside from affecting the fees, the main purpose of the weight system is to +prevent a block from being too filled with transactions. The system module, +while processing blocks≠transactions within a block, accumulates both the total +length of the block (sum of encoded transactions in number of bytes) and the +total weight of the blocks. At any points, if these numbers surpass the limits, +no further transactions are accepted to the block. These limits are defined in +[`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +and +[`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +respectively. + +One important note about these limits is that a ratio of them are reserved for +the `Operational` dispatch class. This rule applies to both of the limits and +the ratio can be found in +[`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). +For example, if the block length limit is 1 mega bytes and the ratio is set the +80%, all transactions can fill the first 800 kilo bytes of the block while the +last 200 can only be filled by the operation class. + + +## Frequent Questions + +> _Some operations get more expensive as storage grows and shrinks. How can the +> weight system represent that?_ + +As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking +the storage and analyzing it just to determine how much weight might be consumed +does not fit this definition quite well. In this case the implementation of the +dispatch should take the state of the change into account and manually take +extra fees, bonds or take any other measures to make sure that the transaction +is safe. + +> _Is the weighting function open to change through governance short of a +> runtime upgrade?_ + +Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to +the weight system as well. A upgrade can just update weight functions if needed. + +## Next Steps + +### Learn More + +TODO + +### Examples + +TODO + +### References + +TODO + diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 7850788c4f..817c6f269b 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -3,67 +3,15 @@ title: Transaction Weight and Fees --- When transactions are submitted to a blockchain, they are executed by the nodes -in the network. To be economically sustainable, the cost nodes incur to execute -a transaction must be covered by the submitter of the transaction. The cost to -execute transactions could vary over orders of magnitude, and thus Substrate +in the network. To be economically sustainable, nodes charge a fee to execute a +transaction that must be covered by the submitter of the transaction. The cost +to execute transactions could vary over orders of magnitude, and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -Furthermore, a number of resources in any chain can be limited by definition -such as time, memory and computation. A mechanism should exist to prevent -individual resource-consuming components of the chain, including but not limited -to dispatchables, to consume too much of any of the mentioned resources. This -concept is represented in substrate by weights. - -## Weights Basics - -The fee system is dependent on the -[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. -Hence, we will first briefly describe how weights work. - -As mentioned, weights can technically refer to, or represent, numerous _limited_ -resources. A custom implementation may use complex structures to demonstrate -this. At the time of this writing, substrate weights are simply a [numeric -value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each -dispatchable function is given a weight using the `#[weight = $x]` annotation, -where `$x` is some function capable of determining the weight of the dispatch. -`$x` can access and examine the arguments of the dispatch. Nonetheless, a weight -calculation should always: -- Be computable __ahead of dispatch__. A block producer, or a validator, should - be able to examine the weight of a dispatchable before actually deciding to - accept it or not. -- While not enforced, calculating the weight should not consume much resource - itself. It does not make much sense to spend almost the same amount of time as - dispatching a transaction to estimate how much weight it will consume. Thus, - weight computation should typically be very fast, much faster than - dispatching. This is analogous to a `poll()` function of the `Future` trait. - It really doesn't make sense to poll a future if it is going to stall us until - the future itself is resolved; A sensible `poll` must return fast. -- Finally, if a transaction is very variable in the amount of resources it - consumes, it should manually prevent restrict certain dispatches through any - means that makes sense to that particular chain. An example of such dispatches - in substrate is the call to a smart contract (todo: link when this is - implemented). - -Given the above details, one can write their own weight computation function, -`$x` in the above listing, to determine the weight of dispatchable. The -programming details of doing this is beyond the scope of this document. For more -information on that, refer to: -- the [weight - module's](https://crates.parity.io/sr_primitives/weights/index.html) - documentation in substrate-primitives. -- the - [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) - module and the inline documentation. Skim the code for `#[weight]` attributes - and all the related documentations. -- the section on weights in Substrate recipes. TODO - -The system module is responsible for accumulating the weight of each block as it -gets executed and making sure that it does not exceed the limit. The -transaction-payment module is responsible for interpreting these weights and -deducting fees based upon them. - -## Fees Calculation +Make sure to read [Weights] before implementing fees. + +## Transaction Fee The fee to include a transaction consists of three parts: @@ -90,7 +38,21 @@ fee = WeightToFee(weight) ``` -## Adjusting Multiplier +TODO: Examples + +## Substrate Default Fee System + +How to use the base fee system + +### Declaring a Trasnaction Weight + +### Changing the Default Weight + +### Changing the fee calculation + +### Using Operational Transactions + +### Adjusting Multiplier The above formula gives a fee which is __logically constant through time__. Of course, the weight can, in practice, be dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree of variability. As for the @@ -111,26 +73,8 @@ this requirement, Substrate provides: change. This is expressed via [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) -## More on Weights: Block Weight and Length Limit +## Building a Custom Fee System -Aside from affecting the fees, the main purpose of the weight system is to -prevent a block from being too filled with transactions. The system module, -while processing blocks≠transactions within a block, accumulates both the total -length of the block (sum of encoded transactions in number of bytes) and the -total weight of the blocks. At any points, if these numbers surpass the limits, -no further transactions are accepted to the block. These limits are defined in -[`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) -and -[`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) -respectively. - -One important note about these limits is that a ratio of them are reserved for -the `Operational` dispatch class. This rule applies to both of the limits and -the ratio can be found in -[`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). -For example, if the block length limit is 1 mega bytes and the ratio is set the -80%, all transactions can fill the first 800 kilo bytes of the block while the -last 200 can only be filled by the operation class. ## Gluing it Together: How Things Work in Substrate @@ -159,29 +103,11 @@ page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#rel > any other substrate chain is indeed encouraged to parameterize its fee system > according to its own logic and economic incentives. -## Frequent Questions - -> _Some operations get more expensive as storage grows and shrinks. How can the -> weight system represent that?_ - -As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking -the storage and analyzing it just to determine how much weight might be consumed -does not fit this definition quite well. In this case the implementation of the -dispatch should take the state of the change into account and manually take -extra fees, bonds or take any other measures to make sure that the transaction -is safe. - -> _Is the weighting function open to change through governance short of a -> runtime upgrade?_ - -Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to -the weight system as well. A upgrade can just update weight functions if needed. - ## Next Steps ### Learn More -TODO +Read about weights ### Examples From 35be2aa534c6b0a4fd655bbfe72de85f74f68a32 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 24 Oct 2019 15:08:28 +0200 Subject: [PATCH 09/26] Update sidebar and title --- docs/conceptual/runtime/weight.md | 2 +- docs/development/module/fees.md | 2 +- website/sidebars.json | 1 + 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 2a8229ae2e..eeee865895 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -1,5 +1,5 @@ --- -title: Transaction Weight and Fees +title: Transaction Weight --- When transactions are submitted to a blockchain, they are executed by the nodes diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 817c6f269b..634bd7f626 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -1,5 +1,5 @@ --- -title: Transaction Weight and Fees +title: Transaction Fees --- When transactions are submitted to a blockchain, they are executed by the nodes diff --git a/website/sidebars.json b/website/sidebars.json index 2aa4ce683c..bfa28c9626 100644 --- a/website/sidebars.json +++ b/website/sidebars.json @@ -31,6 +31,7 @@ "ids": [ "conceptual/runtime/index", "conceptual/runtime/srml", + "conceptual/runtime/weight", "conceptual/runtime/contracts", "conceptual/runtime/primitives" ] From aa487f5d410a193c5ba9c207b11431efd1e9ca93 Mon Sep 17 00:00:00 2001 From: kianenigma Date: Fri, 25 Oct 2019 11:56:57 +0200 Subject: [PATCH 10/26] Revamped --- docs/development/module/fees.md | 181 ++++++++++++++++++++++++-------- 1 file changed, 140 insertions(+), 41 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 634bd7f626..d46c81471b 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -4,14 +4,15 @@ title: Transaction Fees When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, nodes charge a fee to execute a -transaction that must be covered by the submitter of the transaction. The cost -to execute transactions could vary over orders of magnitude, and thus Substrate -provides a flexible mechanism for characterizing the total, presumably minimum, -cost of a transaction in order to be included into a block. +transaction. This fee must be covered by the submitter of the transaction. The +cost to execute transactions could vary over orders of magnitude, and thus +Substrate provides a flexible mechanism for characterizing the total, presumably +minimum, cost of a transaction in order to be included into a block. -Make sure to read [Weights] before implementing fees. +The fee system is heavily linked to the [weight system](). Make sure to read and +understand weights section before continuing this document. -## Transaction Fee +## Transaction Fees The fee to include a transaction consists of three parts: @@ -27,7 +28,9 @@ The fee to include a transaction consists of three parts: `Currency`, `srml-balances` in substrate node). For this, each runtime must define a [`WeightToFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) - type that makes the conversion. + type that makes the conversion. `WeightToFee` must be a struct that implements + a [`Convert`](https://crates.parity.io/sr_primitives/traits/trait.Convert.html). based on the above, the final fee of a dispatchable is: @@ -38,59 +41,90 @@ fee = WeightToFee(weight) ``` -TODO: Examples +Customizing the above would be as simple as configuring the appropriate +associated type in the respective module. + +```rust +use sr_primitives::{traits::Convert, weights::Weight} +// Assume this is the balance type +type Balance = u64; + +// Assume we want all the weights to have a `100 + 2 * w` conversion to fees +struct CustomWeightToFee; +impl Convert for CustomWeightToFee { + fn convert(w: Weight) -> Balance { + let a = Balance::from(100); + let b = Balance::from(2); + let w = Balance::from(w); + a + b * w + } +} + +parameter_types! { + TransactionBaseFee: Balance = 10; + TransactionByteFee: Balance = 10; +} + +impl transaction_payment::Trait { + TransactionBaseFee = TransactionBaseFee; + TransactionByteFee = TransactionByteFee; + WeightToFee = CustomWeightToFee; + // we will get to this one soon enough + FeeMultiplierUpdate = (); +} -## Substrate Default Fee System - -How to use the base fee system - -### Declaring a Trasnaction Weight +``` -### Changing the Default Weight +Two further questions need to be answered, given the above snippet. -### Changing the fee calculation +1. What is the purpose of `FeeMultiplierUpdate`? +2. Knowing how we are capable of defining the conversion of weights to fees (via + `WeightToFee`), the question remains, how can we define and customize the + weights in the first place? -### Using Operational Transactions +We will answer each question in the following sections, respectively. -### Adjusting Multiplier +## Adjusting Multiplier The above formula gives a fee which is __logically constant through time__. Of -course, the weight can, in practice, be dynamic and based on what `WeightToFee` +course, the weight can be dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree of variability. As for the length fee, the inputs of the transaction could change the length and hence affecting the length fee. Nonetheless, these changes are independent and a -_general update logic to the entire fee cannot be composed out of them +_general update logic to the **entire fee** cannot be composed out of them trivially_. In other words, for any dispatchable, given the same inputs, _it will always incur the same cost_. This might not always be desirable. Chains -might need to increase or decrease fees based on sime condition. To fulfill +might need to increase or decrease fees based on some condition. To fulfill this requirement, Substrate provides: - a multiplier stored in the transaction-payment module that is applied to the outcome of the above formula by default (needless to say, the default value of which is _multiplication identity_, meaning that it has no effect). This is stored in [`NextFeeMultiplier`](https://crates.parity.io/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) - storage. + storage and can be configured through the genesis spec of the module. - a configurable parameter for a runtime to describe how this multiplier can change. This is expressed via - [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) - -## Building a Custom Fee System + [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). +`NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point +number with a billion points accuracy. So, given the final fee formula above, +the final version would be: +``` +fee = + base_fee + + len(tx) * length_fee + + WeightToFee(weight) -## Gluing it Together: How Things Work in Substrate - -All of the above parameters are configurable and having a look at the runtime -crate in substrate should demonstrate the concrete values. - -The configurable values (such as `MaximumBlockWeight`) are created using the -`parameter_types!` macro and passed to the associated module (`system` and -`transaction-payment`). +final_fee = fee * NextFeeMultiplier +``` -The default and minimum `Weight` is defined as `FixedNormal(10_000)` and the -`WeightToFee` is configured to map this value to `10^7` as the minimum weight -fee of substrate. +Updating the `NextFeeMultiplier` has a similar manner as `WeightToFee`. The +[`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) +associated type in `transaction-payment` is defined as a `Convert`, which should be read as: it receives the previous multiplier and +spits out the next one. -The update function is inspired by the Polkadot network and implements a +The default update function is inspired by the Polkadot network and implements a targeted adjustment in which a target saturation level of block weight is defined. If the previous block is more saturated, then the fees are slightly increases. Similarly, if less transaction than the target are in the previous @@ -98,16 +132,81 @@ block, fees are decreased by a small amount. More information about this can be found in the [web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). -> Note that this section demonstrates the configuration of the fee/weight system -> at the time of this writing in `substrate-node`. This can change over time and -> any other substrate chain is indeed encouraged to parameterize its fee system -> according to its own logic and economic incentives. +## Substrate Weight System + +> This section assumes that you have already read the `Weight` section + +The entire substrate runtime module library is already annotated with a simple +and fixed weight system. A user can decide to use the same system or implement a +new one from scratch. The latter is outside the scope of this document and is +explained int he dedicated [`Weight`]() conceptual document. + +### Using the Default Weight + +The default weight annotation is beyond _simple_. Substrate, by default, uses _fixed_ weights and the struct representing them is as follows: + +```rust +pub enum SimpleDispatchInfo { + /// A normal dispatch with fixed weight. + FixedNormal(Weight), + /// A normal dispatch with the maximum weight. + MaxNormal, + /// A normal dispatch with no weight. + FreeNormal, + /// An operational dispatch with fixed weight. + FixedOperational(Weight), + /// An operational dispatch with the maximum weight. + MaxOperational, + /// An operational dispatch with no weight. + FreeOperational, +} +``` + +This struct simple groups all dispatches into _normal_ and _operational_ (which makes the implementation of `ClassifyDispatch` pretty trivial) and gives them a fixed Weight. Fixed in this context means that the arguments of the dispatch do not play any role in the weight; the weight of a dispatch is always fixed. + +A simple example of using this simple struct in your runtime would be: + +```rust +use sr_primitives::weights::{SimpleDispatchInfo}; + +decl_module! { + // This means that this function has no weight. It will not contribute to block fullness at all, + // and no weight-fee is applied. + #[weight = SimpleDispatchInfo::FreeNormal] + pub fn some_normal_function_light() { noop(); } + + // This function will always have a weight `10`. + #[weight = SimpleDispatchInfo::FixedNormal(10)] + pub fn some_normal_function_heavy() { some_computation(); } + + // This function will have a fixed weight but can consume the reserved operational portion as well. + #[weight = SimpleDispatchInfo::FixedOperational(20)] + pub fn mission_critical_function() { some_sudo_op(); } + + // This will automatically get `#[weight = SimpleDispatchInfo::default()]` + pub fn something_else +} +``` + +**Be careful!** The default implementation of `SimpleDispatchInfo` resolves to +`FixedNormal(10_000)`. This is entire due to how things work in `substrate-node` +and the desired granularity of substrate. Even if you want to use the +`SimpleDispatchInfo`, it is very likely that you would want it to have a +different `Default`. + + ## Next Steps +One important note is that the entire logic of fees is encapsulated in +`srml-transaction-payment` via a `SignedExtension`. While this module provides a +high degree of flexibility, a user can opt to build their custom payment module +while inspiring from transaction-payment. + ### Learn More -Read about weights +- Dedicated weight document +- SignedExtensions ### Examples From e3502fa14b6eac7d4d2932247190acaa6a97478b Mon Sep 17 00:00:00 2001 From: kianenigma Date: Fri, 25 Oct 2019 14:39:11 +0200 Subject: [PATCH 11/26] A lot more docs --- docs/conceptual/runtime/weight.md | 223 ++++++++++++++++++------------ docs/development/module/fees.md | 192 ++++++++++++------------- 2 files changed, 225 insertions(+), 190 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index eeee865895..6f6632449f 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -2,119 +2,162 @@ title: Transaction Weight --- -When transactions are submitted to a blockchain, they are executed by the nodes -in the network. To be economically sustainable, nodes charge a fee to execute a -transaction that must be covered by the submitter of the transaction. The cost -to execute transactions could vary over orders of magnitude, and thus Substrate -provides a flexible mechanism for characterizing the total, presumably minimum, -cost of a transaction in order to be included into a block. - -Furthermore, a number of resources in any chain can be limited by definition -such as time, memory and computation. A mechanism should exist to prevent -individual resource-consuming components of the chain, including but not limited -to dispatchables, to consume too much of any of the mentioned resources. This -concept is represented in substrate by weights. +A number of resources in any chain can be limited by definition, such as time, memory and +computation. A mechanism should exist to prevent individual resource-consuming components of the +chain, including but not limited to dispatchable functions, from consume too much of any of the +mentioned resources. This concept is represented in substrate by weights. Furthermore, consuming +some weights could optionally incur some fee. + +The fee implications of the weight system is covered in the [Fee Developer document](). This +document mostly covers the concept of weights. ## Transaction Weight -The fee system is dependent on the -[weight](https://crates.parity.io/sr_primitives/weights/index.html) system. -Hence, we will first briefly describe how weights work. - -As mentioned, weights can technically refer to, or represent, numerous _limited_ -resources. A custom implementation may use complex structures to demonstrate -this. At the time of this writing, substrate weights are simply a [numeric -value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each -dispatchable function is given a weight using the `#[weight = $x]` annotation, -where `$x` is some function capable of determining the weight of the dispatch. -`$x` can access and examine the arguments of the dispatch. Nonetheless, a weight -calculation should always: -- Be computable __ahead of dispatch__. A block producer, or a validator, should - be able to examine the weight of a dispatchable before actually deciding to - accept it or not. -- While not enforced, calculating the weight should not consume much resource - itself. It does not make much sense to spend almost the same amount of time as - dispatching a transaction to estimate how much weight it will consume. Thus, - weight computation should typically be very fast, much faster than - dispatching. This is analogous to a `poll()` function of the `Future` trait. - It really doesn't make sense to poll a future if it is going to stall us until - the future itself is resolved; A sensible `poll` must return fast. -- Finally, if a transaction is very variable in the amount of resources it - consumes, it should manually prevent restrict certain dispatches through any - means that makes sense to that particular chain. An example of such dispatches - in substrate is the call to a smart contract (todo: link when this is - implemented). - -Given the above details, one can write their own weight computation function, -`$x` in the above listing, to determine the weight of dispatchable. The -programming details of doing this is beyond the scope of this document. For more -information on that, refer to: -- the [weight - module's](https://crates.parity.io/sr_primitives/weights/index.html) - documentation in substrate-primitives. -- the - [`srml-example`](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) - module and the inline documentation. Skim the code for `#[weight]` attributes - and all the related documentations. -- the section on weights in Substrate recipes. TODO - -The system module is responsible for accumulating the weight of each block as it -gets executed and making sure that it does not exceed the limit. The -transaction-payment module is responsible for interpreting these weights and -deducting fees based upon them. - - -## More on Weights: Block Weight and Length Limit - -Aside from affecting the fees, the main purpose of the weight system is to -prevent a block from being too filled with transactions. The system module, -while processing blocks≠transactions within a block, accumulates both the total -length of the block (sum of encoded transactions in number of bytes) and the -total weight of the blocks. At any points, if these numbers surpass the limits, -no further transactions are accepted to the block. These limits are defined in +As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom +implementation may use complex structures to demonstrate this. At the time of this writing, +substrate weights are simply a [numeric +value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each dispatchable function +is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of +determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. +Nonetheless, a weight calculation should always: +- Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine + the weight of a dispatchable before actually deciding to accept it or not. +- While not enforced, calculating the weight should not consume much resource itself. It does not + make sense to consume similar resources computing a transaction's weight as would be spent while + executing it. Thus, weight computation should typically be very fast, much faster than + dispatching. +- Finally, if a transaction's resource consumption varies over a wide range, it should manually + prevent restrict certain dispatches through any means that makes sense to that particular chain. + An example of such dispatches in substrate is the call to a smart contract (todo: link when this + is implemented). + +The system module is responsible for accumulating the weight of each block as it gets executed and +making sure that it does not exceed the limit. The transaction-payment module is responsible for +interpreting these weights and deducting fees based upon them. + + +## Block Weight and Length Limit + +Aside from affecting the fees, the main purpose of the weight system is to prevent a block from +being too filled with transactions. The system module, while processing transactions within a block, +accumulates both the total length of the block (sum of encoded transactions in number of bytes) and +the total weight of the blocks. At any points, if these numbers surpass the limits, no further +transactions are accepted to the block. These limits are defined in [`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and [`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) respectively. -One important note about these limits is that a ratio of them are reserved for -the `Operational` dispatch class. This rule applies to both of the limits and -the ratio can be found in +One important note about these limits is that a ratio of them are reserved for the `Operational` +dispatch class. This rule applies to both of the limits and the ratio can be found in [`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). -For example, if the block length limit is 1 mega bytes and the ratio is set the -80%, all transactions can fill the first 800 kilo bytes of the block while the -last 200 can only be filled by the operation class. +For example, if the block length limit is 1 mega bytes and the ratio is set the 80%, all +transactions can fill the first 800 kilo bytes of the block while the last 200 can only be filled by +the operation class. + +## Custom Weight Implementation + +Implementation a custom weight calculation function can vary from _very easy_ to _super +complicated_. The `SimpleDispatchInfo` struct provided by substrate is fairly one of the easiest +appraoches. Anything more sophisticated would require a bit more work. + +Any weight calculation function must provide two trait implementations: + - [`WeightData`]: To determine the weight of the dispatch. + - [`ClassifyDispatch`]: To determine the class of the dispatch. See the enum definition for + more information on dispatch classes. This struct can then be used with `#[weight]` annotation. + +Substrate then bundles then output information of the two traits into [`DispatchInfo`] struct and +provides it by implementing the [`GetDispatchInfo`] for all `Call` variants, and opaque extrinsic +types. This is used internally by the system and executive module; you won't use it most likely. + +Both `ClassifyDispatch` and `WeightData` are generic over `T` which gets resolved into the tuple of +all dispatch arguments except for the origin. To demonstrate, we will craft a struct that calculates +the weight as `m * len(args)` where `m` is a given multiplier and `args` is the concatenated tuple +of all dispatch arguments. Furthermore, the dispatch class is Operational if the transaction has +more than 100 bytes of length in arguments. + +```rust +use coded::Encode; +use sr_primitives::weights::{DispatchClass, ClassifyDispatch, WeightData} + +// self.0 is the multiplier, `m` +struct LenWeight(u32); + +// We don't quite know what T is. After all, different dispatches have different arguments, hence +// `T` will be different. All that we care about is that `T` is encodable. That is always true by +// definition. All dispatch arguments are encodable. +impl WeighData for LenWeight { + fn weigh_data(&self, target: T) -> Weight { + let multiplier = self.0; + let encoded_len = target.encode().len() as u32; + multiplier * encoded_len + } +} + +impl ClassifyDispatch for LenWeight { + fn classify_dispatch(&self, target: T) -> DispatchClass { + let encoded_len = target.encode().len() as u32; + if encoded_len > 100 { + DispatchClass::Operational + } else { + DispatchClass::Normal + } + } +} +``` + +A weight calculator function can also be entirely coerced to the final type of the argument, instead +of defining it as a vague type that is encodable. This is also possible and `srml-example` contains +an example of how to do this. Just note that in that case your code would roughly look like: + +```rust + +struct CustomWeight; +impl WeighData<(u32, u64)> for CustomWeight { + fn weigh_data(&self, target: (u32, u64)) -> Weight { + ... + } +} + +// given dispatch: +decl_module! { + fn foo(a: u32, b: u64) { ... } +} +``` + +which means **`CustomWeight` can only be used in conjunction with a dispatch with a particular +signature (u32, u64)**, as opposed to `LenWeight` which can be used with literally anything because +they don't make any strict assumptions about ``. ## Frequent Questions -> _Some operations get more expensive as storage grows and shrinks. How can the -> weight system represent that?_ +> _Some operations get more expensive as storage grows and shrinks. How can the weight system +> represent that?_ -As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking -the storage and analyzing it just to determine how much weight might be consumed -does not fit this definition quite well. In this case the implementation of the -dispatch should take the state of the change into account and manually take -extra fees, bonds or take any other measures to make sure that the transaction -is safe. +As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking the storage and +analyzing it just to determine how much weight might be consumed does not fit this definition quite +well. In this case the implementation of the dispatch should take the state of the change into +account and manually take extra fees, bonds or take any other measures to make sure that the +transaction is safe. -> _Is the weighting function open to change through governance short of a -> runtime upgrade?_ +> _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ -Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to -the weight system as well. A upgrade can just update weight functions if needed. +Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as +well. A upgrade can just update weight functions if needed. ## Next Steps ### Learn More -TODO - -### Examples +- srml-examples +- recipes on weights +- fee module +TODO: links -TODO ### References -TODO +- weights.rs +TODO: links diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index d46c81471b..d3fca98434 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -2,15 +2,14 @@ title: Transaction Fees --- -When transactions are submitted to a blockchain, they are executed by the nodes -in the network. To be economically sustainable, nodes charge a fee to execute a -transaction. This fee must be covered by the submitter of the transaction. The -cost to execute transactions could vary over orders of magnitude, and thus -Substrate provides a flexible mechanism for characterizing the total, presumably +When transactions are submitted to a blockchain, they are executed by the nodes in the network. To +be economically sustainable, nodes charge a fee to execute a transaction. This fee must be covered +by the submitter of the transaction. The cost to execute transactions could vary over orders of +magnitude, and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -The fee system is heavily linked to the [weight system](). Make sure to read and -understand weights section before continuing this document. +The fee system is heavily linked to the [weight system](). Make sure to read and understand weights +section before continuing this document. ## Transaction Fees @@ -18,18 +17,15 @@ The fee to include a transaction consists of three parts: * `base_fee` a fixed fee that is applied to every single transaction. See [`TransactionBaseFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) -* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the - encoded transaction. See +* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded + transaction. See [`TransactionByteFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) -* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the - transaction. As mentioned, weight of each dispatch is denoted via the flexible - `#[weight]` annotation. Knowing the weight, it must be converted to a - deductible `balance` type (typically denoted by a module that implements - `Currency`, `srml-balances` in substrate node). For this, each runtime must - define a +* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As + mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the + weight, it must be converted to a deductible `balance` type (typically denoted by a module that + implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a [`WeightToFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) - type that makes the conversion. `WeightToFee` must be a struct that implements - a [`Convert`](https://crates.parity.io/sr_primitives/traits/trait.Convert.html). based on the above, the final fee of a dispatchable is: @@ -41,8 +37,8 @@ fee = WeightToFee(weight) ``` -Customizing the above would be as simple as configuring the appropriate -associated type in the respective module. +Customizing the above would be as simple as configuring the appropriate associated type in the +respective module. ```rust use sr_primitives::{traits::Convert, weights::Weight} @@ -52,25 +48,25 @@ type Balance = u64; // Assume we want all the weights to have a `100 + 2 * w` conversion to fees struct CustomWeightToFee; impl Convert for CustomWeightToFee { - fn convert(w: Weight) -> Balance { - let a = Balance::from(100); - let b = Balance::from(2); - let w = Balance::from(w); - a + b * w - } + fn convert(w: Weight) -> Balance { + let a = Balance::from(100); + let b = Balance::from(2); + let w = Balance::from(w); + a + b * w + } } parameter_types! { - TransactionBaseFee: Balance = 10; - TransactionByteFee: Balance = 10; + TransactionBaseFee: Balance = 10; + TransactionByteFee: Balance = 10; } impl transaction_payment::Trait { - TransactionBaseFee = TransactionBaseFee; - TransactionByteFee = TransactionByteFee; - WeightToFee = CustomWeightToFee; - // we will get to this one soon enough - FeeMultiplierUpdate = (); + TransactionBaseFee = TransactionBaseFee; + TransactionByteFee = TransactionByteFee; + WeightToFee = CustomWeightToFee; + // we will get to this one soon enough + FeeMultiplierUpdate = (); } ``` @@ -78,36 +74,31 @@ impl transaction_payment::Trait { Two further questions need to be answered, given the above snippet. 1. What is the purpose of `FeeMultiplierUpdate`? -2. Knowing how we are capable of defining the conversion of weights to fees (via - `WeightToFee`), the question remains, how can we define and customize the - weights in the first place? +2. Knowing how we are capable of defining the conversion of weights to fees (via `WeightToFee`), the + question remains, how can we define and customize the weights in the first place? We will answer each question in the following sections, respectively. ## Adjusting Multiplier -The above formula gives a fee which is __logically constant through time__. Of -course, the weight can be dynamic and based on what `WeightToFee` -is defined to be, the final fee can have some degree of variability. As for the -length fee, the inputs of the transaction could change the length and hence -affecting the length fee. Nonetheless, these changes are independent and a -_general update logic to the **entire fee** cannot be composed out of them -trivially_. In other words, for any dispatchable, given the same inputs, _it -will always incur the same cost_. This might not always be desirable. Chains -might need to increase or decrease fees based on some condition. To fulfill -this requirement, Substrate provides: - - a multiplier stored in the transaction-payment module that is applied to the - outcome of the above formula by default (needless to say, the default value - of which is _multiplication identity_, meaning that it has no effect). This - is stored in +The above formula gives a fee which is __logically constant through time__. Of course, the weight +can be dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree +of variability. As for the length fee, the inputs of the transaction could change the length and +hence affecting the length fee. Nonetheless, these changes are independent and a _general update +logic to the **entire fee** cannot be composed out of them trivially_. In other words, for any +dispatchable, given the same inputs, _it will always incur the same cost_. This might not always be +desirable. Chains might need to increase or decrease fees based on some condition. To fulfill this +requirement, Substrate provides: + - a multiplier stored in the transaction-payment module that is applied to the outcome of the + above formula by default (needless to say, the default value of which is _multiplication + identity_, meaning that it has no effect). This is stored in [`NextFeeMultiplier`](https://crates.parity.io/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) storage and can be configured through the genesis spec of the module. - - a configurable parameter for a runtime to describe how this multiplier can - change. This is expressed via + - a configurable parameter for a runtime to describe how this multiplier can change. This is + expressed via [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). -`NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point -number with a billion points accuracy. So, given the final fee formula above, -the final version would be: +`NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number with a billion +points accuracy. So, given the final fee formula above, the final version would be: ``` fee = @@ -120,15 +111,13 @@ final_fee = fee * NextFeeMultiplier Updating the `NextFeeMultiplier` has a similar manner as `WeightToFee`. The [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) -associated type in `transaction-payment` is defined as a `Convert`, which should be read as: it receives the previous multiplier and -spits out the next one. - -The default update function is inspired by the Polkadot network and implements a -targeted adjustment in which a target saturation level of block weight is -defined. If the previous block is more saturated, then the fees are slightly -increases. Similarly, if less transaction than the target are in the previous -block, fees are decreased by a small amount. More information about this can be +associated type in `transaction-payment` is defined as a `Convert`, which should +be read as: it receives the previous multiplier and spits out the next one. + +The default update function is inspired by the Polkadot network and implements a targeted adjustment +in which a target saturation level of block weight is defined. If the previous block is more +saturated, then the fees are slightly increases. Similarly, if less transaction than the target are +in the previous block, fees are decreased by a small amount. More information about this can be found in the [web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). @@ -136,33 +125,37 @@ page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#rel > This section assumes that you have already read the `Weight` section -The entire substrate runtime module library is already annotated with a simple -and fixed weight system. A user can decide to use the same system or implement a -new one from scratch. The latter is outside the scope of this document and is -explained int he dedicated [`Weight`]() conceptual document. +The entire substrate runtime module library is already annotated with a simple and fixed weight +system. A user can decide to use the same system or implement a new one from scratch. The latter is +outside the scope of this document and is explained int he dedicated [`Weight`]() conceptual +document. ### Using the Default Weight -The default weight annotation is beyond _simple_. Substrate, by default, uses _fixed_ weights and the struct representing them is as follows: +The default weight annotation is beyond _simple_. Substrate, by default, uses _fixed_ weights and +the struct representing them is as follows: ```rust pub enum SimpleDispatchInfo { - /// A normal dispatch with fixed weight. - FixedNormal(Weight), - /// A normal dispatch with the maximum weight. - MaxNormal, - /// A normal dispatch with no weight. - FreeNormal, - /// An operational dispatch with fixed weight. - FixedOperational(Weight), - /// An operational dispatch with the maximum weight. - MaxOperational, - /// An operational dispatch with no weight. - FreeOperational, + /// A normal dispatch with fixed weight. + FixedNormal(Weight), + /// A normal dispatch with the maximum weight. + MaxNormal, + /// A normal dispatch with no weight. + FreeNormal, + /// An operational dispatch with fixed weight. + FixedOperational(Weight), + /// An operational dispatch with the maximum weight. + MaxOperational, + /// An operational dispatch with no weight. + FreeOperational, } ``` -This struct simple groups all dispatches into _normal_ and _operational_ (which makes the implementation of `ClassifyDispatch` pretty trivial) and gives them a fixed Weight. Fixed in this context means that the arguments of the dispatch do not play any role in the weight; the weight of a dispatch is always fixed. +This struct simple groups all dispatches into _normal_ and _operational_ (which makes the +implementation of `ClassifyDispatch` pretty trivial) and gives them a fixed Weight. Fixed in this +context means that the arguments of the dispatch do not play any role in the weight; the weight of a +dispatch is always fixed. A simple example of using this simple struct in your runtime would be: @@ -170,42 +163,41 @@ A simple example of using this simple struct in your runtime would be: use sr_primitives::weights::{SimpleDispatchInfo}; decl_module! { - // This means that this function has no weight. It will not contribute to block fullness at all, - // and no weight-fee is applied. - #[weight = SimpleDispatchInfo::FreeNormal] - pub fn some_normal_function_light() { noop(); } + // This means that this function has no weight. It will not contribute to block fullness at all, + // and no weight-fee is applied. + #[weight = SimpleDispatchInfo::FreeNormal] + pub fn some_normal_function_light() { noop(); } - // This function will always have a weight `10`. - #[weight = SimpleDispatchInfo::FixedNormal(10)] - pub fn some_normal_function_heavy() { some_computation(); } + // This function will always have a weight `10`. + #[weight = SimpleDispatchInfo::FixedNormal(10)] + pub fn some_normal_function_heavy() { some_computation(); } // This function will have a fixed weight but can consume the reserved operational portion as well. - #[weight = SimpleDispatchInfo::FixedOperational(20)] - pub fn mission_critical_function() { some_sudo_op(); } + #[weight = SimpleDispatchInfo::FixedOperational(20)] + pub fn mission_critical_function() { some_sudo_op(); } - // This will automatically get `#[weight = SimpleDispatchInfo::default()]` - pub fn something_else + // This will automatically get `#[weight = SimpleDispatchInfo::default()]` + pub fn something_else } ``` **Be careful!** The default implementation of `SimpleDispatchInfo` resolves to -`FixedNormal(10_000)`. This is entire due to how things work in `substrate-node` -and the desired granularity of substrate. Even if you want to use the -`SimpleDispatchInfo`, it is very likely that you would want it to have a -different `Default`. +`FixedNormal(10_000)`. This is entire due to how things work in `substrate-node` and the desired +granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that +you would want it to have a different `Default`. ## Next Steps -One important note is that the entire logic of fees is encapsulated in -`srml-transaction-payment` via a `SignedExtension`. While this module provides a -high degree of flexibility, a user can opt to build their custom payment module -while inspiring from transaction-payment. +One important note is that the entire logic of fees is encapsulated in `srml-transaction-payment` +via a `SignedExtension`. While this module provides a high degree of flexibility, a user can opt to +build their custom payment module while inspiring from transaction-payment. ### Learn More - Dedicated weight document +- srml-example - SignedExtensions ### Examples From 7e8a5c66d14a715a22cea877619979e3dcf4dbb3 Mon Sep 17 00:00:00 2001 From: kianenigma Date: Fri, 25 Oct 2019 14:44:54 +0200 Subject: [PATCH 12/26] small fixes --- docs/conceptual/runtime/weight.md | 12 ++++++------ docs/development/module/fees.md | 16 ++++++++-------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 6f6632449f..7423195b24 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -16,7 +16,7 @@ document mostly covers the concept of weights. As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom implementation may use complex structures to demonstrate this. At the time of this writing, substrate weights are simply a [numeric -value](https://crates.parity.io/sr_primitives/weights/type.Weight.html). Each dispatchable function +value](/rustdocs/master/sr_primitives/weights/type.Weight.html). Each dispatchable function is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. Nonetheless, a weight calculation should always: @@ -43,14 +43,14 @@ being too filled with transactions. The system module, while processing transact accumulates both the total length of the block (sum of encoded transactions in number of bytes) and the total weight of the blocks. At any points, if these numbers surpass the limits, no further transactions are accepted to the block. These limits are defined in -[`MaximumBlockLength`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +[`MaximumBlockLength`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and -[`MaximumBlockWeight`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) +[`MaximumBlockWeight`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) respectively. One important note about these limits is that a ratio of them are reserved for the `Operational` dispatch class. This rule applies to both of the limits and the ratio can be found in -[`AvailableBlockRatio`](https://crates.parity.io/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). +[`AvailableBlockRatio`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). For example, if the block length limit is 1 mega bytes and the ratio is set the 80%, all transactions can fill the first 800 kilo bytes of the block while the last 200 can only be filled by the operation class. @@ -113,8 +113,8 @@ an example of how to do this. Just note that in that case your code would roughl ```rust struct CustomWeight; -impl WeighData<(u32, u64)> for CustomWeight { - fn weigh_data(&self, target: (u32, u64)) -> Weight { +impl WeighData<(&u32, &u64)> for CustomWeight { + fn weigh_data(&self, target: (&u32, &u64)) -> Weight { ... } } diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index d3fca98434..ed6735a7d2 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -8,7 +8,7 @@ by the submitter of the transaction. The cost to execute transactions could vary magnitude, and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -The fee system is heavily linked to the [weight system](). Make sure to read and understand weights +The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to read and understand weights section before continuing this document. ## Transaction Fees @@ -16,17 +16,17 @@ section before continuing this document. The fee to include a transaction consists of three parts: * `base_fee` a fixed fee that is applied to every single transaction. See - [`TransactionBaseFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) + [`TransactionBaseFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) * `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded transaction. See - [`TransactionByteFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) + [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) * `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it must be converted to a deductible `balance` type (typically denoted by a module that implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a - [`WeightToFee`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) + [`WeightToFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) type that makes the conversion. `WeightToFee` must be a struct that implements a [`Convert`](https://crates.parity.io/sr_primitives/traits/trait.Convert.html). + Balance>`](/rustdocs/master/sr_primitives/traits/trait.Convert.html). based on the above, the final fee of a dispatchable is: @@ -91,11 +91,11 @@ requirement, Substrate provides: - a multiplier stored in the transaction-payment module that is applied to the outcome of the above formula by default (needless to say, the default value of which is _multiplication identity_, meaning that it has no effect). This is stored in - [`NextFeeMultiplier`](https://crates.parity.io/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) + [`NextFeeMultiplier`](/rustdocs/master/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) storage and can be configured through the genesis spec of the module. - a configurable parameter for a runtime to describe how this multiplier can change. This is expressed via - [`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). + [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). `NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number with a billion points accuracy. So, given the final fee formula above, the final version would be: @@ -110,7 +110,7 @@ final_fee = fee * NextFeeMultiplier ``` Updating the `NextFeeMultiplier` has a similar manner as `WeightToFee`. The -[`FeeMultiplierUpdate`](https://crates.parity.io/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) +[`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) associated type in `transaction-payment` is defined as a `Convert`, which should be read as: it receives the previous multiplier and spits out the next one. From cf98f5406716c34a566ab58089768f4bd3e26ffa Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Fri, 25 Oct 2019 14:57:16 -0400 Subject: [PATCH 13/26] struct -> enaum, and operational doesn't pay length fee. --- docs/development/module/fees.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index ed6735a7d2..bc98be189c 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -133,7 +133,7 @@ document. ### Using the Default Weight The default weight annotation is beyond _simple_. Substrate, by default, uses _fixed_ weights and -the struct representing them is as follows: +the enum representing them is as follows: ```rust pub enum SimpleDispatchInfo { @@ -152,12 +152,12 @@ pub enum SimpleDispatchInfo { } ``` -This struct simple groups all dispatches into _normal_ and _operational_ (which makes the +This enum groups all dispatches into _normal_ and _operational_ (which makes the implementation of `ClassifyDispatch` pretty trivial) and gives them a fixed Weight. Fixed in this -context means that the arguments of the dispatch do not play any role in the weight; the weight of a -dispatch is always fixed. +context means that the arguments of the dispatch do not play any role in the weight. Dispatches +classified as _operational_ are exempt from paying both the `base_fee`, and `length_fee`. -A simple example of using this simple struct in your runtime would be: +A simple example of using this enum in your runtime is: ```rust use sr_primitives::weights::{SimpleDispatchInfo}; @@ -207,4 +207,3 @@ TODO ### References TODO - From cd83715501e7b1649328007dce0909c57ba4ef6e Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Fri, 25 Oct 2019 15:09:52 -0400 Subject: [PATCH 14/26] Add some links. --- docs/conceptual/runtime/weight.md | 12 ++++-------- docs/development/module/fees.md | 12 ++++++------ 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 7423195b24..12258d0ef9 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -150,14 +150,10 @@ well. A upgrade can just update weight functions if needed. ### Learn More -- srml-examples -- recipes on weights -- fee module -TODO: links - +- Substrate Recipes contains examples of both [custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). +- The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module ### References -- weights.rs -TODO: links - +- SRML's [transaction-payment module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs) +- Much about weights including the `SimpleDispatchInfo` enum is defined in [weights.rs](https://github.com/paritytech/substrate/blob/master/core/sr-primitives/src/weights.rs). diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index bc98be189c..b14e34579b 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -190,19 +190,19 @@ you would want it to have a different `Default`. ## Next Steps -One important note is that the entire logic of fees is encapsulated in `srml-transaction-payment` -via a `SignedExtension`. While this module provides a high degree of flexibility, a user can opt to -build their custom payment module while inspiring from transaction-payment. +The entire logic of fees is encapsulated in `srml-transaction-payment` via a `SignedExtension`. +While this module provides a high degree of flexibility, a user can opt to build their custom +payment module drawing inspiration from transaction-payment. ### Learn More -- Dedicated weight document -- srml-example +- Dedicated [weight documentation](/docs/conceptual/runtime/weight) +- [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module - SignedExtensions ### Examples -TODO +Substrate Recipes contains examples of both [custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). ### References From e1e882b91289883fad20d4696183f42da2236761 Mon Sep 17 00:00:00 2001 From: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Fri, 25 Oct 2019 22:12:40 +0200 Subject: [PATCH 15/26] Update docs/conceptual/runtime/weight.md --- docs/conceptual/runtime/weight.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 12258d0ef9..4ca774f44c 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -5,7 +5,7 @@ title: Transaction Weight A number of resources in any chain can be limited by definition, such as time, memory and computation. A mechanism should exist to prevent individual resource-consuming components of the chain, including but not limited to dispatchable functions, from consume too much of any of the -mentioned resources. This concept is represented in substrate by weights. Furthermore, consuming +mentioned resources. This concept is represented in substrate runtime by weights. Furthermore, consuming some weights could optionally incur some fee. The fee implications of the weight system is covered in the [Fee Developer document](). This From 0b0025b531919bccc2e0e0f2cff453e3978fd618 Mon Sep 17 00:00:00 2001 From: joepetrowski Date: Mon, 28 Oct 2019 10:19:18 +0100 Subject: [PATCH 16/26] first pass at cleaning up --- docs/conceptual/runtime/weight.md | 111 +++++++++++++++--------------- docs/development/module/fees.md | 94 ++++++++++++------------- 2 files changed, 103 insertions(+), 102 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 4ca774f44c..1d56fcb1c2 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -2,79 +2,80 @@ title: Transaction Weight --- -A number of resources in any chain can be limited by definition, such as time, memory and -computation. A mechanism should exist to prevent individual resource-consuming components of the -chain, including but not limited to dispatchable functions, from consume too much of any of the -mentioned resources. This concept is represented in substrate runtime by weights. Furthermore, consuming -some weights could optionally incur some fee. +A number of resources in any chain can be limited, such as storage or computation. A mechanism +should exist to prevent individual resource-consuming components of the chain, including but not +limited to dispatchable functions, from consuming too much of any resource. This concept is +represented in the Substrate runtime by weights. Further, consuming some weights could incur a fee. -The fee implications of the weight system is covered in the [Fee Developer document](). This -document mostly covers the concept of weights. +The fee implications of the weight system are covered in the +[Fee Developer document](development/module/fees.md). This document mostly covers the concept of +weights. ## Transaction Weight -As mentioned, weights can technically refer to, or represent, numerous _limited_ resources. A custom -implementation may use complex structures to demonstrate this. At the time of this writing, -substrate weights are simply a [numeric -value](/rustdocs/master/sr_primitives/weights/type.Weight.html). Each dispatchable function +Weights represent the _limited_ resources of your blockchain, for example computational cycles, +memory, storage, etc. A custom implementation may use complex structures to express this. At the +time of this writing, Substrate weights are simply a +[numeric value](/rustdocs/master/sr_primitives/weights/type.Weight.html). Each dispatchable function is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. + Nonetheless, a weight calculation should always: + - Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine the weight of a dispatchable before actually deciding to accept it or not. -- While not enforced, calculating the weight should not consume much resource itself. It does not - make sense to consume similar resources computing a transaction's weight as would be spent while - executing it. Thus, weight computation should typically be very fast, much faster than - dispatching. +- While not enforced, calculating the weight should not consume many resources itself. It does not + make sense to consume similar resources computing a transaction's weight as would be spent to + execute it. Thus, weight computation should typically be much faster than dispatch. - Finally, if a transaction's resource consumption varies over a wide range, it should manually - prevent restrict certain dispatches through any means that makes sense to that particular chain. - An example of such dispatches in substrate is the call to a smart contract (todo: link when this - is implemented). + restrict certain dispatches through any means that make sense to that particular chain. An + example of such a dispatche in Substrate is a call to a smart contract. -The system module is responsible for accumulating the weight of each block as it gets executed and -making sure that it does not exceed the limit. The transaction-payment module is responsible for +The System module is responsible for accumulating the weight of each block as it gets executed and +making sure that it does not exceed the limit. The Transaction Payment module is responsible for interpreting these weights and deducting fees based upon them. - ## Block Weight and Length Limit -Aside from affecting the fees, the main purpose of the weight system is to prevent a block from -being too filled with transactions. The system module, while processing transactions within a block, -accumulates both the total length of the block (sum of encoded transactions in number of bytes) and -the total weight of the blocks. At any points, if these numbers surpass the limits, no further -transactions are accepted to the block. These limits are defined in +Aside from affecting fees, the main purpose of the weight system is to prevent a block from being +filled with too many transactions. The System module, while processing transactions within a block, +accumulates both the total length of the block (sum of encoded transactions in bytes) and the total +weight of the block. If these numbers surpass the limits, no further transactions are accepted in +that block. These limits are defined in [`MaximumBlockLength`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and -[`MaximumBlockWeight`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) -respectively. +[`MaximumBlockWeight`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength). -One important note about these limits is that a ratio of them are reserved for the `Operational` +One important note about these limits is that a portion of them are reserved for the `Operational` dispatch class. This rule applies to both of the limits and the ratio can be found in [`AvailableBlockRatio`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). -For example, if the block length limit is 1 mega bytes and the ratio is set the 80%, all -transactions can fill the first 800 kilo bytes of the block while the last 200 can only be filled by -the operation class. + +For example, if the block length limit is 1 megabyte and the ratio is set to 80%, all +transactions can fill the first 800 kilobytes of the block while the last 200 can only be filled by +the operational class. ## Custom Weight Implementation -Implementation a custom weight calculation function can vary from _very easy_ to _super -complicated_. The `SimpleDispatchInfo` struct provided by substrate is fairly one of the easiest -appraoches. Anything more sophisticated would require a bit more work. +Implementing a custom weight calculation function can vary in complexity. Using the +`SimpleDispatchInfo` struct provided by Substrate is one of the easiest appraoches. Anything more +sophisticated would require more work. Any weight calculation function must provide two trait implementations: + - [`WeightData`]: To determine the weight of the dispatch. - [`ClassifyDispatch`]: To determine the class of the dispatch. See the enum definition for - more information on dispatch classes. This struct can then be used with `#[weight]` annotation. + more information on dispatch classes. This struct can then be used with the `#[weight]` + annotation. -Substrate then bundles then output information of the two traits into [`DispatchInfo`] struct and -provides it by implementing the [`GetDispatchInfo`] for all `Call` variants, and opaque extrinsic -types. This is used internally by the system and executive module; you won't use it most likely. +Substrate then bundles the output information of the two traits into the [`DispatchInfo`] struct and +provides it by implementing the [`GetDispatchInfo`] for all `Call` variants and opaque extrinsic +types. This is used internally by the System and Executive modules; you probably won't use it. -Both `ClassifyDispatch` and `WeightData` are generic over `T` which gets resolved into the tuple of +Both `ClassifyDispatch` and `WeightData` are generic over `T`, which gets resolved into the tuple of all dispatch arguments except for the origin. To demonstrate, we will craft a struct that calculates the weight as `m * len(args)` where `m` is a given multiplier and `args` is the concatenated tuple -of all dispatch arguments. Furthermore, the dispatch class is Operational if the transaction has -more than 100 bytes of length in arguments. +of all dispatch arguments. Further, the dispatch class is `Operational` if the transaction has more +than 100 bytes of length in arguments. ```rust use coded::Encode; @@ -106,12 +107,11 @@ impl ClassifyDispatch for LenWeight { } ``` -A weight calculator function can also be entirely coerced to the final type of the argument, instead -of defining it as a vague type that is encodable. This is also possible and `srml-example` contains -an example of how to do this. Just note that in that case your code would roughly look like: +A weight calculator function can also be coerced to the final type of the argument, instead of +defining it as a vague type that is encodable. `srml-example` contains an example of how to do +this. Just note that, in that case, your code would roughly look like: ```rust - struct CustomWeight; impl WeighData<(&u32, &u64)> for CustomWeight { fn weigh_data(&self, target: (&u32, &u64)) -> Weight { @@ -125,35 +125,34 @@ decl_module! { } ``` -which means **`CustomWeight` can only be used in conjunction with a dispatch with a particular -signature (u32, u64)**, as opposed to `LenWeight` which can be used with literally anything because -they don't make any strict assumptions about ``. - +This means that `CustomWeight` can only be used in conjunction with a dispatch with a particular +signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they +don't make any strict assumptions about ``. -## Frequent Questions +## Frequently Asked Questions > _Some operations get more expensive as storage grows and shrinks. How can the weight system > represent that?_ As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking the storage and analyzing it just to determine how much weight might be consumed does not fit this definition quite -well. In this case the implementation of the dispatch should take the state of the change into -account and manually take extra fees, bonds or take any other measures to make sure that the +well. In this case, the implementation of the dispatch should take the state of the change into +account and manually take extra fees or bonds or take any other measures to make sure that the transaction is safe. > _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as -well. A upgrade can just update weight functions if needed. +well. An upgrade can update weight functions if needed. ## Next Steps ### Learn More - Substrate Recipes contains examples of both [custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). -- The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module +- The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module. ### References -- SRML's [transaction-payment module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs) +- SRML's [transaction-payment module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs). - Much about weights including the `SimpleDispatchInfo` enum is defined in [weights.rs](https://github.com/paritytech/substrate/blob/master/core/sr-primitives/src/weights.rs). diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index b14e34579b..1388f61dfd 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -8,27 +8,27 @@ by the submitter of the transaction. The cost to execute transactions could vary magnitude, and thus Substrate provides a flexible mechanism for characterizing the total, presumably minimum, cost of a transaction in order to be included into a block. -The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to read and understand weights -section before continuing this document. +The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to +understand weights before continuing this document. ## Transaction Fees The fee to include a transaction consists of three parts: -* `base_fee` a fixed fee that is applied to every single transaction. See - [`TransactionBaseFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee) -* `length_fee` a per-byte fee that is multiplied by the length, in bytes, of the encoded +* `base_fee`: a fixed fee that is applied to every transaction. See + [`TransactionBaseFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee). +* `length_fee`: a per-byte fee that is multiplied by the length, in bytes, of the encoded transaction. See - [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee) -* `weight_fee` a per-weight-unit fee that is multiplied by the weight of the transaction. As + [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee). +* `weight_fee`: a per-weight-unit fee that is multiplied by the weight of the transaction. As mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it must be converted to a deductible `balance` type (typically denoted by a module that - implements `Currency`, `srml-balances` in substrate node). For this, each runtime must define a + implements `Currency`, `srml-balances` in Substrate node). For this, each runtime must define a [`WeightToFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) type that makes the conversion. `WeightToFee` must be a struct that implements a [`Convert`](/rustdocs/master/sr_primitives/traits/trait.Convert.html). -based on the above, the final fee of a dispatchable is: +Based on the above, the final fee of a dispatchable is: ``` fee = @@ -68,37 +68,38 @@ impl transaction_payment::Trait { // we will get to this one soon enough FeeMultiplierUpdate = (); } - ``` Two further questions need to be answered, given the above snippet. 1. What is the purpose of `FeeMultiplierUpdate`? -2. Knowing how we are capable of defining the conversion of weights to fees (via `WeightToFee`), the - question remains, how can we define and customize the weights in the first place? +2. Knowing that we are capable of defining the conversion of weights to fees (via `WeightToFee`), + the question remains, how can we define and customize the weights in the first place? We will answer each question in the following sections, respectively. ## Adjusting Multiplier -The above formula gives a fee which is __logically constant through time__. Of course, the weight -can be dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree + +The above formula gives a fee that is logically constant through time. Because the weight can be +dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree of variability. As for the length fee, the inputs of the transaction could change the length and -hence affecting the length fee. Nonetheless, these changes are independent and a _general update -logic to the **entire fee** cannot be composed out of them trivially_. In other words, for any -dispatchable, given the same inputs, _it will always incur the same cost_. This might not always be -desirable. Chains might need to increase or decrease fees based on some condition. To fulfill this -requirement, Substrate provides: - - a multiplier stored in the transaction-payment module that is applied to the outcome of the - above formula by default (needless to say, the default value of which is _multiplication - identity_, meaning that it has no effect). This is stored in +affect the length fee. Nonetheless, these changes are independent and _general update logic to the +entire fee cannot be composed out of them trivially_. In other words, for any dispatchable, given +the same inputs, _it will always incur the same cost_. This might not always be desirable. Chains +might need to increase or decrease fees based on some condition. To fulfill this requirement, +Substrate provides: + + - a multiplier stored in the Transaction Payment module that is applied to the outcome of the + above formula by default (the default value of which is _multiplication identity_, meaning that + it has no effect). This is stored in [`NextFeeMultiplier`](/rustdocs/master/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) - storage and can be configured through the genesis spec of the module. + and can be configured through the genesis spec of the module. - a configurable parameter for a runtime to describe how this multiplier can change. This is expressed via [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). `NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number with a billion -points accuracy. So, given the final fee formula above, the final version would be: +points of accuracy. So, given the final fee formula above, the final version would be: ``` fee = @@ -109,31 +110,30 @@ fee = final_fee = fee * NextFeeMultiplier ``` -Updating the `NextFeeMultiplier` has a similar manner as `WeightToFee`. The +Updating the `NextFeeMultiplier` has a similar effect as updating `WeightToFee`. The [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) associated type in `transaction-payment` is defined as a `Convert`, which should be read as: it receives the previous multiplier and spits out the next one. The default update function is inspired by the Polkadot network and implements a targeted adjustment in which a target saturation level of block weight is defined. If the previous block is more -saturated, then the fees are slightly increases. Similarly, if less transaction than the target are -in the previous block, fees are decreased by a small amount. More information about this can be -found in the [web3 research +saturated, then the fees are slightly increased. Similarly, if the previous block has fewer +transactions than the target, fees are decreased by a small amount. More information about this can +be found in the [Web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). ## Substrate Weight System -> This section assumes that you have already read the `Weight` section +> This section assumes that you have already read the `Weight` section. -The entire substrate runtime module library is already annotated with a simple and fixed weight -system. A user can decide to use the same system or implement a new one from scratch. The latter is -outside the scope of this document and is explained int he dedicated [`Weight`]() conceptual -document. +The entire SRML is already annotated with a simple and fixed weight system. A user can decide to +use the same system or implement a new one from scratch. The latter is outside the scope of this +document and is explained in the dedicated [`Weight`]() conceptual document. ### Using the Default Weight -The default weight annotation is beyond _simple_. Substrate, by default, uses _fixed_ weights and -the enum representing them is as follows: +The default weight annotation is simple. Substrate, by default, uses _fixed_ weights and the enum +representing them is as follows: ```rust pub enum SimpleDispatchInfo { @@ -152,10 +152,10 @@ pub enum SimpleDispatchInfo { } ``` -This enum groups all dispatches into _normal_ and _operational_ (which makes the -implementation of `ClassifyDispatch` pretty trivial) and gives them a fixed Weight. Fixed in this -context means that the arguments of the dispatch do not play any role in the weight. Dispatches -classified as _operational_ are exempt from paying both the `base_fee`, and `length_fee`. +This enum groups all dispatches into _normal_ and _operational_ (which makes the implementation of +`ClassifyDispatch` trivial) and gives them a fixed weight. Fixed in this context means that the +arguments of the dispatch do not play any role in the weight. Dispatches classified as +_operational_ are exempt from paying both the `base_fee` and the `length_fee`. A simple example of using this enum in your runtime is: @@ -172,27 +172,26 @@ decl_module! { #[weight = SimpleDispatchInfo::FixedNormal(10)] pub fn some_normal_function_heavy() { some_computation(); } - // This function will have a fixed weight but can consume the reserved operational portion as well. + // This function will have a fixed weight but can consume the reserved operational portion as + // well. #[weight = SimpleDispatchInfo::FixedOperational(20)] pub fn mission_critical_function() { some_sudo_op(); } - // This will automatically get `#[weight = SimpleDispatchInfo::default()]` + // This will automatically get `#[weight = SimpleDispatchInfo::default()]`. pub fn something_else } ``` **Be careful!** The default implementation of `SimpleDispatchInfo` resolves to -`FixedNormal(10_000)`. This is entire due to how things work in `substrate-node` and the desired +`FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that you would want it to have a different `Default`. - - ## Next Steps The entire logic of fees is encapsulated in `srml-transaction-payment` via a `SignedExtension`. While this module provides a high degree of flexibility, a user can opt to build their custom -payment module drawing inspiration from transaction-payment. +payment module drawing inspiration from Transaction Payment. ### Learn More @@ -202,7 +201,10 @@ payment module drawing inspiration from transaction-payment. ### Examples -Substrate Recipes contains examples of both [custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). +Substrate Recipes contains examples of both +[custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) +and custom +[WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). ### References From c21b5d569703a531c8a4cd7993accb5bfa2ffdb2 Mon Sep 17 00:00:00 2001 From: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Thu, 31 Oct 2019 10:05:33 +0100 Subject: [PATCH 17/26] Apply suggestions from code review Co-Authored-By: Shawn Tabrizi --- docs/conceptual/runtime/weight.md | 2 +- docs/development/module/fees.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 1d56fcb1c2..9eae50e26d 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -140,7 +140,7 @@ well. In this case, the implementation of the dispatch should take the state of account and manually take extra fees or bonds or take any other measures to make sure that the transaction is safe. -> _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ +> _Is the weighing function open to change through governance or any sort of a runtime upgrade?_ Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as well. An upgrade can update weight functions if needed. diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 1388f61dfd..9d66bf1298 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -113,7 +113,7 @@ final_fee = fee * NextFeeMultiplier Updating the `NextFeeMultiplier` has a similar effect as updating `WeightToFee`. The [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) associated type in `transaction-payment` is defined as a `Convert`, which should -be read as: it receives the previous multiplier and spits out the next one. +be read: "it receives the previous multiplier and spits out the next one". The default update function is inspired by the Polkadot network and implements a targeted adjustment in which a target saturation level of block weight is defined. If the previous block is more @@ -182,7 +182,7 @@ decl_module! { } ``` -**Be careful!** The default implementation of `SimpleDispatchInfo` resolves to +> **Note:** Be careful! The default implementation of `SimpleDispatchInfo` resolves to `FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that you would want it to have a different `Default`. From bd470be6e7a06b2245e4f7d0f9bf5e02a88d5cf3 Mon Sep 17 00:00:00 2001 From: kianenigma Date: Thu, 31 Oct 2019 10:05:40 +0100 Subject: [PATCH 18/26] minor update --- docs/development/module/fees.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index ed6735a7d2..02068642d8 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -97,8 +97,8 @@ requirement, Substrate provides: expressed via [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). -`NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number with a billion -points accuracy. So, given the final fee formula above, the final version would be: +`NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number. So, given the +final fee formula above, the final version would be: ``` fee = From e0757cb12a3b1fac28b103c8d1501c36105765c6 Mon Sep 17 00:00:00 2001 From: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Thu, 31 Oct 2019 13:46:24 +0100 Subject: [PATCH 19/26] Update docs/conceptual/runtime/weight.md Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> --- docs/conceptual/runtime/weight.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 9eae50e26d..1d56fcb1c2 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -140,7 +140,7 @@ well. In this case, the implementation of the dispatch should take the state of account and manually take extra fees or bonds or take any other measures to make sure that the transaction is safe. -> _Is the weighing function open to change through governance or any sort of a runtime upgrade?_ +> _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as well. An upgrade can update weight functions if needed. From 57701a0737c84d94ddc92fc811cd01d9cdb71e40 Mon Sep 17 00:00:00 2001 From: joepetrowski Date: Thu, 31 Oct 2019 14:57:26 +0100 Subject: [PATCH 20/26] minor restructure --- docs/conceptual/runtime/weight.md | 23 +++--- docs/development/module/fees.md | 132 ++++++++++++++++-------------- 2 files changed, 82 insertions(+), 73 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 1d56fcb1c2..8f0a1364bb 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -16,11 +16,9 @@ weights. Weights represent the _limited_ resources of your blockchain, for example computational cycles, memory, storage, etc. A custom implementation may use complex structures to express this. At the time of this writing, Substrate weights are simply a -[numeric value](/rustdocs/master/sr_primitives/weights/type.Weight.html). Each dispatchable function -is given a weight using the `#[weight = $x]` annotation, where `$x` is some function capable of -determining the weight of the dispatch. `$x` can access and examine the arguments of the dispatch. +[numeric value](/rustdocs/master/sr_primitives/weights/type.Weight.html). -Nonetheless, a weight calculation should always: +A weight calculation should always: - Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine the weight of a dispatchable before actually deciding to accept it or not. @@ -29,7 +27,8 @@ Nonetheless, a weight calculation should always: execute it. Thus, weight computation should typically be much faster than dispatch. - Finally, if a transaction's resource consumption varies over a wide range, it should manually restrict certain dispatches through any means that make sense to that particular chain. An - example of such a dispatche in Substrate is a call to a smart contract. + example of such a dispatch in Substrate is a call to a smart contract. + The System module is responsible for accumulating the weight of each block as it gets executed and making sure that it does not exceed the limit. The Transaction Payment module is responsible for @@ -40,8 +39,8 @@ interpreting these weights and deducting fees based upon them. Aside from affecting fees, the main purpose of the weight system is to prevent a block from being filled with too many transactions. The System module, while processing transactions within a block, accumulates both the total length of the block (sum of encoded transactions in bytes) and the total -weight of the block. If these numbers surpass the limits, no further transactions are accepted in -that block. These limits are defined in +weight of the block. If either of these numbers surpass the limits, no further transactions are +accepted in that block. These limits are defined in [`MaximumBlockLength`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and [`MaximumBlockWeight`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength). @@ -57,15 +56,13 @@ the operational class. ## Custom Weight Implementation Implementing a custom weight calculation function can vary in complexity. Using the -`SimpleDispatchInfo` struct provided by Substrate is one of the easiest appraoches. Anything more +`SimpleDispatchInfo` struct provided by Substrate is one of the easiest approaches. Anything more sophisticated would require more work. Any weight calculation function must provide two trait implementations: - [`WeightData`]: To determine the weight of the dispatch. - - [`ClassifyDispatch`]: To determine the class of the dispatch. See the enum definition for - more information on dispatch classes. This struct can then be used with the `#[weight]` - annotation. + - [`ClassifyDispatch`]: To determine the class of the dispatch. Substrate then bundles the output information of the two traits into the [`DispatchInfo`] struct and provides it by implementing the [`GetDispatchInfo`] for all `Call` variants and opaque extrinsic @@ -142,8 +139,8 @@ transaction is safe. > _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ -Yes, it is part of a runtime itself, hence any runtime upgrade is an upgrade to the weight system as -well. An upgrade can update weight functions if needed. +Yes, it is part of the runtime itself, hence any runtime upgrade is an upgrade to the weight system +as well. An upgrade can update weight functions if needed. ## Next Steps diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 4be3727990..73020a28b3 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -4,9 +4,8 @@ title: Transaction Fees When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, nodes charge a fee to execute a transaction. This fee must be covered -by the submitter of the transaction. The cost to execute transactions could vary over orders of -magnitude, and thus Substrate provides a flexible mechanism for characterizing the total, presumably -minimum, cost of a transaction in order to be included into a block. +by the submitter of the transaction. The cost to execute transactions can vary, so Substrate +provides a flexible mechanism to characterize the minimum cost to include a transaction in a block. The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to understand weights before continuing this document. @@ -20,12 +19,12 @@ The fee to include a transaction consists of three parts: * `length_fee`: a per-byte fee that is multiplied by the length, in bytes, of the encoded transaction. See [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee). -* `weight_fee`: a per-weight-unit fee that is multiplied by the weight of the transaction. As - mentioned, weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the +* `weight_fee`: a per-weight-unit fee that is multiplied by the weight of the transaction. The + weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it must be converted to a deductible `balance` type (typically denoted by a module that implements `Currency`, `srml-balances` in Substrate node). For this, each runtime must define a [`WeightToFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) - type that makes the conversion. `WeightToFee` must be a struct that implements a [`Convert`](/rustdocs/master/sr_primitives/traits/trait.Convert.html). Based on the above, the final fee of a dispatchable is: @@ -37,69 +36,33 @@ fee = WeightToFee(weight) ``` -Customizing the above would be as simple as configuring the appropriate associated type in the -respective module. - -```rust -use sr_primitives::{traits::Convert, weights::Weight} -// Assume this is the balance type -type Balance = u64; - -// Assume we want all the weights to have a `100 + 2 * w` conversion to fees -struct CustomWeightToFee; -impl Convert for CustomWeightToFee { - fn convert(w: Weight) -> Balance { - let a = Balance::from(100); - let b = Balance::from(2); - let w = Balance::from(w); - a + b * w - } -} - -parameter_types! { - TransactionBaseFee: Balance = 10; - TransactionByteFee: Balance = 10; -} - -impl transaction_payment::Trait { - TransactionBaseFee = TransactionBaseFee; - TransactionByteFee = TransactionByteFee; - WeightToFee = CustomWeightToFee; - // we will get to this one soon enough - FeeMultiplierUpdate = (); -} -``` - -Two further questions need to be answered, given the above snippet. - -1. What is the purpose of `FeeMultiplierUpdate`? -2. Knowing that we are capable of defining the conversion of weights to fees (via `WeightToFee`), - the question remains, how can we define and customize the weights in the first place? - -We will answer each question in the following sections, respectively. +This `fee` is known as the "inclusion fee." Even if the extrinsic fails, the sender must pay this +inclusion fee. ## Adjusting Multiplier The above formula gives a fee that is logically constant through time. Because the weight can be -dynamic and based on what `WeightToFee` is defined to be, the final fee can have some degree -of variability. As for the length fee, the inputs of the transaction could change the length and -affect the length fee. Nonetheless, these changes are independent and _general update logic to the +dynamic and based on how `WeightToFee` is defined, the final fee can have some degree of +variability. As for the length fee, the inputs of the transaction could change the length and +affect the length fee. + +Nonetheless, these changes are independent and _general update logic to the entire fee cannot be composed out of them trivially_. In other words, for any dispatchable, given the same inputs, _it will always incur the same cost_. This might not always be desirable. Chains might need to increase or decrease fees based on some condition. To fulfill this requirement, Substrate provides: - - a multiplier stored in the Transaction Payment module that is applied to the outcome of the + - A multiplier stored in the Transaction Payment module that is applied to the outcome of the above formula by default (the default value of which is _multiplication identity_, meaning that it has no effect). This is stored in [`NextFeeMultiplier`](/rustdocs/master/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) and can be configured through the genesis spec of the module. - - a configurable parameter for a runtime to describe how this multiplier can change. This is + - A configurable parameter for a runtime to describe how this multiplier can change. This is expressed via [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). `NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number. So, given the -final fee formula above, the final version would be: +inclusion fee formula above, the final version would be: ``` fee = @@ -113,7 +76,7 @@ final_fee = fee * NextFeeMultiplier Updating the `NextFeeMultiplier` has a similar effect as updating `WeightToFee`. The [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) associated type in `transaction-payment` is defined as a `Convert`, which should -be read: "it receives the previous multiplier and spits out the next one". +be read: "it receives the previous multiplier and returns the next one". The default update function is inspired by the Polkadot network and implements a targeted adjustment in which a target saturation level of block weight is defined. If the previous block is more @@ -128,7 +91,8 @@ page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#rel The entire SRML is already annotated with a simple and fixed weight system. A user can decide to use the same system or implement a new one from scratch. The latter is outside the scope of this -document and is explained in the dedicated [`Weight`]() conceptual document. +document and is explained in the dedicated [`Weight`](/docs/conceptual/runtime/weight) conceptual +document. ### Using the Default Weight @@ -183,9 +147,57 @@ decl_module! { ``` > **Note:** Be careful! The default implementation of `SimpleDispatchInfo` resolves to -`FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired -granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that -you would want it to have a different `Default`. +> `FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired +> granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that +> you would want it to have a different `Default`. + +## Other Fees + +Inclusion fees don't know anything about the logic of the transaction being executed. That is, +Substrate doesn't care what happens in the transaction, it only cares about the size and weight of +the transaction. The inclusion fee will always be paid by the sender. + +It's possible to add fees inside dispatchable functions that are only paid if certain logic paths +are executed. Most likely, this will be if the transaction succeeds. The `transfer` function in the +Balances module, for example, takes a fixed fee for transferring tokens. + +It is important to note that if you query the chain for a transaction fee, it will only return the +inclusion fee. If you want to query internal function fees, you should emit Events for them. + +## Custom Inclusion Fee Example + +This is an example of how to customize your inclusion fee. You must configure the appropriate +associated types in the respective module. + +```rust +use sr_primitives::{traits::Convert, weights::Weight} +// Assume this is the balance type +type Balance = u64; + +// Assume we want all the weights to have a `100 + 2 * w` conversion to fees +struct CustomWeightToFee; +impl Convert for CustomWeightToFee { + fn convert(w: Weight) -> Balance { + let a = Balance::from(100); + let b = Balance::from(2); + let w = Balance::from(w); + a + b * w + } +} + +parameter_types! { + TransactionBaseFee: Balance = 10; + TransactionByteFee: Balance = 10; +} + +impl transaction_payment::Trait { + TransactionBaseFee = TransactionBaseFee; + TransactionByteFee = TransactionByteFee; + WeightToFee = CustomWeightToFee; + // we will get to this one soon enough + FeeMultiplierUpdate = (); +} +``` ## Next Steps @@ -196,8 +208,8 @@ payment module drawing inspiration from Transaction Payment. ### Learn More - Dedicated [weight documentation](/docs/conceptual/runtime/weight) -- [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module -- SignedExtensions +- [Example module](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) +- [SignedExtension](/rustdocs/master/sr_primitives/traits/trait.SignedExtension.html) ### Examples @@ -208,4 +220,4 @@ and custom ### References -TODO +- [Web3 Foundation Research](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees-and-per-block-transaction-limits) From 6eed65320f7030274429fe45efdafe442672bf7e Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Wed, 6 Nov 2019 10:57:10 +0100 Subject: [PATCH 21/26] Auto-spacing --- docs/conceptual/runtime/weight.md | 65 +++++++++++++++++-------------- 1 file changed, 35 insertions(+), 30 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 8f0a1364bb..a63e77be66 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -2,21 +2,20 @@ title: Transaction Weight --- -A number of resources in any chain can be limited, such as storage or computation. A mechanism -should exist to prevent individual resource-consuming components of the chain, including but not -limited to dispatchable functions, from consuming too much of any resource. This concept is +A number of resources in any chain can be limited, such as storage or computation. A mechanism +should exist to prevent individual resource-consuming components of the chain, including but not +limited to dispatchable functions, from consuming too much of any resource. This concept is represented in the Substrate runtime by weights. Further, consuming some weights could incur a fee. -The fee implications of the weight system are covered in the -[Fee Developer document](development/module/fees.md). This document mostly covers the concept of -weights. +The fee implications of the weight system are covered in the [Fee Developer +document](development/module/fees.md). This document mostly covers the concept of weights. ## Transaction Weight -Weights represent the _limited_ resources of your blockchain, for example computational cycles, -memory, storage, etc. A custom implementation may use complex structures to express this. At the -time of this writing, Substrate weights are simply a -[numeric value](/rustdocs/master/sr_primitives/weights/type.Weight.html). +Weights represent the _limited_ resources of your blockchain, for example computational cycles, +memory, storage, etc. A custom implementation may use complex structures to express this. At the +time of this writing, Substrate weights are simply a [numeric +value](/rustdocs/master/sr_primitives/weights/type.Weight.html). A weight calculation should always: @@ -26,8 +25,8 @@ A weight calculation should always: make sense to consume similar resources computing a transaction's weight as would be spent to execute it. Thus, weight computation should typically be much faster than dispatch. - Finally, if a transaction's resource consumption varies over a wide range, it should manually - restrict certain dispatches through any means that make sense to that particular chain. An - example of such a dispatch in Substrate is a call to a smart contract. + restrict certain dispatches through any means that make sense to that particular chain. An example + of such a dispatch in Substrate is a call to a smart contract. The System module is responsible for accumulating the weight of each block as it gets executed and @@ -36,10 +35,10 @@ interpreting these weights and deducting fees based upon them. ## Block Weight and Length Limit -Aside from affecting fees, the main purpose of the weight system is to prevent a block from being +Aside from affecting fees, the main purpose of the weight system is to prevent a block from being filled with too many transactions. The System module, while processing transactions within a block, -accumulates both the total length of the block (sum of encoded transactions in bytes) and the total -weight of the block. If either of these numbers surpass the limits, no further transactions are +accumulates both the total length of the block (sum of encoded transactions in bytes) and the total +weight of the block. If either of these numbers surpass the limits, no further transactions are accepted in that block. These limits are defined in [`MaximumBlockLength`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.MaximumBlockLength) and @@ -49,14 +48,14 @@ One important note about these limits is that a portion of them are reserved for dispatch class. This rule applies to both of the limits and the ratio can be found in [`AvailableBlockRatio`](/rustdocs/master/srml_system/trait.Trait.html#associatedtype.AvailableBlockRatio). -For example, if the block length limit is 1 megabyte and the ratio is set to 80%, all -transactions can fill the first 800 kilobytes of the block while the last 200 can only be filled by -the operational class. +For example, if the block length limit is 1 megabyte and the ratio is set to 80%, all transactions +can fill the first 800 kilobytes of the block while the last 200 can only be filled by the +operational class. ## Custom Weight Implementation -Implementing a custom weight calculation function can vary in complexity. Using the -`SimpleDispatchInfo` struct provided by Substrate is one of the easiest approaches. Anything more +Implementing a custom weight calculation function can vary in complexity. Using the +`SimpleDispatchInfo` struct provided by Substrate is one of the easiest approaches. Anything more sophisticated would require more work. Any weight calculation function must provide two trait implementations: @@ -71,7 +70,7 @@ types. This is used internally by the System and Executive modules; you probably Both `ClassifyDispatch` and `WeightData` are generic over `T`, which gets resolved into the tuple of all dispatch arguments except for the origin. To demonstrate, we will craft a struct that calculates the weight as `m * len(args)` where `m` is a given multiplier and `args` is the concatenated tuple -of all dispatch arguments. Further, the dispatch class is `Operational` if the transaction has more +of all dispatch arguments. Further, the dispatch class is `Operational` if the transaction has more than 100 bytes of length in arguments. ```rust @@ -104,9 +103,9 @@ impl ClassifyDispatch for LenWeight { } ``` -A weight calculator function can also be coerced to the final type of the argument, instead of -defining it as a vague type that is encodable. `srml-example` contains an example of how to do -this. Just note that, in that case, your code would roughly look like: +A weight calculator function can also be coerced to the final type of the argument, instead of +defining it as a vague type that is encodable. `srml-example` contains an example of how to do this. +Just note that, in that case, your code would roughly look like: ```rust struct CustomWeight; @@ -123,7 +122,7 @@ decl_module! { ``` This means that `CustomWeight` can only be used in conjunction with a dispatch with a particular -signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they +signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they don't make any strict assumptions about ``. ## Frequently Asked Questions @@ -139,17 +138,23 @@ transaction is safe. > _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ -Yes, it is part of the runtime itself, hence any runtime upgrade is an upgrade to the weight system +Yes, it is part of the runtime itself, hence any runtime upgrade is an upgrade to the weight system as well. An upgrade can update weight functions if needed. ## Next Steps ### Learn More -- Substrate Recipes contains examples of both [custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). -- The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module. +- Substrate Recipes contains examples of both [custom + weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) + and custom + [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). +- The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) + module. ### References -- SRML's [transaction-payment module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs). -- Much about weights including the `SimpleDispatchInfo` enum is defined in [weights.rs](https://github.com/paritytech/substrate/blob/master/core/sr-primitives/src/weights.rs). +- SRML's [transaction-payment + module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs). +- Much about weights including the `SimpleDispatchInfo` enum is defined in + [weights.rs](https://github.com/paritytech/substrate/blob/master/core/sr-primitives/src/weights.rs). From e93d14938a6fea034ddff488869a43ddf1b18138 Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Wed, 6 Nov 2019 13:54:24 +0100 Subject: [PATCH 22/26] Remove outdated TODO. (link is already added at the end) --- docs/conceptual/runtime/weight.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index a63e77be66..207eba9e72 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -27,7 +27,6 @@ A weight calculation should always: - Finally, if a transaction's resource consumption varies over a wide range, it should manually restrict certain dispatches through any means that make sense to that particular chain. An example of such a dispatch in Substrate is a call to a smart contract. - The System module is responsible for accumulating the weight of each block as it gets executed and making sure that it does not exceed the limit. The Transaction Payment module is responsible for From e2ce9f87ff0e8f2936f0eb2825e45ce827806938 Mon Sep 17 00:00:00 2001 From: Joshy Orndorff Date: Wed, 6 Nov 2019 14:02:21 +0100 Subject: [PATCH 23/26] Revise (slightly) repeated section heading. --- docs/conceptual/runtime/weight.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 207eba9e72..2e1dd6edf5 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -10,7 +10,7 @@ represented in the Substrate runtime by weights. Further, consuming some weights The fee implications of the weight system are covered in the [Fee Developer document](development/module/fees.md). This document mostly covers the concept of weights. -## Transaction Weight +## Weight Fundamentals Weights represent the _limited_ resources of your blockchain, for example computational cycles, memory, storage, etc. A custom implementation may use complex structures to express this. At the From 06ee0dbc7c4df2c96369ae4d354702671a96eae8 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Wed, 6 Nov 2019 21:05:57 +0100 Subject: [PATCH 24/26] some updates --- docs/conceptual/runtime/weight.md | 32 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index a63e77be66..a36fb38773 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -2,32 +2,28 @@ title: Transaction Weight --- -A number of resources in any chain can be limited, such as storage or computation. A mechanism -should exist to prevent individual resource-consuming components of the chain, including but not -limited to dispatchable functions, from consuming too much of any resource. This concept is -represented in the Substrate runtime by weights. Further, consuming some weights could incur a fee. +A number of resources in any chain can be limited, such as storage or computation. Weights exist to +prevent individual components of the chain from consuming too much of any resource. -The fee implications of the weight system are covered in the [Fee Developer -document](development/module/fees.md). This document mostly covers the concept of weights. +Consuming some weights should generally incur a fee. The fee implications of the weight system are +covered in the [Fee Developer document](development/module/fees.md). ## Transaction Weight Weights represent the _limited_ resources of your blockchain, for example computational cycles, -memory, storage, etc. A custom implementation may use complex structures to express this. At the -time of this writing, Substrate weights are simply a [numeric -value](/rustdocs/master/sr_primitives/weights/type.Weight.html). +memory, storage, etc. A custom implementation may use complex structures to express this. Substrate +weights are simply a [numeric value](/rustdocs/master/sr_primitives/weights/type.Weight.html). A weight calculation should always: -- Be computable __ahead of dispatch__. A block producer, or a validator, should be able to examine - the weight of a dispatchable before actually deciding to accept it or not. -- While not enforced, calculating the weight should not consume many resources itself. It does not - make sense to consume similar resources computing a transaction's weight as would be spent to - execute it. Thus, weight computation should typically be much faster than dispatch. -- Finally, if a transaction's resource consumption varies over a wide range, it should manually - restrict certain dispatches through any means that make sense to that particular chain. An example - of such a dispatch in Substrate is a call to a smart contract. - +- Be computable __ahead of dispatch__. A block producer should be able to examine the weight of a + dispatchable before actually deciding to accept it or not. +- Consume few resources itself. It does not make sense to consume similar resources computing a + transaction's weight as would be spent to execute it. Thus, weight computation should be much + lighter than dispatch. +- Delegate _variable_ resource consumption costs and limitations to the dispatched logic. Weights are + good at representing _fixed_ measurements, whereas logic may not be consistently heavy. In these + cases, additional logic will be needed to limit resource consumption. The System module is responsible for accumulating the weight of each block as it gets executed and making sure that it does not exceed the limit. The Transaction Payment module is responsible for From 352db7db4283710a5af28d8742b7280f700154eb Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 7 Nov 2019 00:13:21 +0100 Subject: [PATCH 25/26] Finish fix weight --- docs/conceptual/runtime/weight.md | 53 +++++++-------- docs/development/front-end/polkadot-js.md | 2 +- docs/development/module/fees.md | 66 +++++++++---------- docs/overview/session-keys.md | 2 +- .../version-1.0/overview/session-keys.md | 2 +- 5 files changed, 60 insertions(+), 65 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index a36fb38773..df53e825dc 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -21,18 +21,22 @@ A weight calculation should always: - Consume few resources itself. It does not make sense to consume similar resources computing a transaction's weight as would be spent to execute it. Thus, weight computation should be much lighter than dispatch. -- Delegate _variable_ resource consumption costs and limitations to the dispatched logic. Weights are - good at representing _fixed_ measurements, whereas logic may not be consistently heavy. In these - cases, additional logic will be needed to limit resource consumption. - -The System module is responsible for accumulating the weight of each block as it gets executed and -making sure that it does not exceed the limit. The Transaction Payment module is responsible for -interpreting these weights and deducting fees based upon them. +- Delegate _variable_ resource consumption costs and limitations to the dispatched logic. Weights + are good at representing _fixed_ measurements, whereas logic may not be consistently heavy. The + implementation of the dispatch should take the state of the change into account and manually take + extra fees or bonds or take any other measures to make sure that the transaction is safe. + +The [System module](https://substrate.dev/rustdocs/master/srml_system/struct.Module.html) is +responsible for accumulating the weight of each block as it gets executed and making sure that it +does not exceed the limit. The [Transaction Payment +module](https://substrate.dev/rustdocs/master/srml_transaction_payment/index.html) is responsible +for interpreting these weights and deducting fees based upon them. The weighing function is part of +the runtime so it can be upgraded if needed. ## Block Weight and Length Limit Aside from affecting fees, the main purpose of the weight system is to prevent a block from being -filled with too many transactions. The System module, while processing transactions within a block, +filled with too many transactions. While processing transactions within a block, the System module accumulates both the total length of the block (sum of encoded transactions in bytes) and the total weight of the block. If either of these numbers surpass the limits, no further transactions are accepted in that block. These limits are defined in @@ -50,11 +54,12 @@ operational class. ## Custom Weight Implementation + + Implementing a custom weight calculation function can vary in complexity. Using the -`SimpleDispatchInfo` struct provided by Substrate is one of the easiest approaches. Anything more -sophisticated would require more work. +`SimpleDispatchInfo` struct provided by Substrate is the easiest approach. -Any weight calculation function must provide two trait implementations: +A weight calculation function must provide two trait implementations: - [`WeightData`]: To determine the weight of the dispatch. - [`ClassifyDispatch`]: To determine the class of the dispatch. @@ -121,22 +126,6 @@ This means that `CustomWeight` can only be used in conjunction with a dispatch w signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they don't make any strict assumptions about ``. -## Frequently Asked Questions - -> _Some operations get more expensive as storage grows and shrinks. How can the weight system -> represent that?_ - -As mentioned, weight should be known _readily_ and _ahead of dispatch_. Peeking the storage and -analyzing it just to determine how much weight might be consumed does not fit this definition quite -well. In this case, the implementation of the dispatch should take the state of the change into -account and manually take extra fees or bonds or take any other measures to make sure that the -transaction is safe. - -> _Is the weighting function open to change through governance or any sort of a runtime upgrade?_ - -Yes, it is part of the runtime itself, hence any runtime upgrade is an upgrade to the weight system -as well. An upgrade can update weight functions if needed. - ## Next Steps ### Learn More @@ -148,9 +137,15 @@ as well. An upgrade can update weight functions if needed. - The [srml-example](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) module. +### Examples + +- See an example of [adding a transaction + weight](https://substrate.dev/recipes/design/econsecurity.html?highlight=weight#assigning-transaction-weights) + to a custom runtime function. + ### References -- SRML's [transaction-payment +- Take a look at the [SRML Transaction Payment module](https://github.com/paritytech/substrate/blob/master/srml/transaction-payment/src/lib.rs). -- Much about weights including the `SimpleDispatchInfo` enum is defined in +- Find info about weights including the `SimpleDispatchInfo` enum in [weights.rs](https://github.com/paritytech/substrate/blob/master/core/sr-primitives/src/weights.rs). diff --git a/docs/development/front-end/polkadot-js.md b/docs/development/front-end/polkadot-js.md index 9bd71c1ab0..f31c075343 100644 --- a/docs/development/front-end/polkadot-js.md +++ b/docs/development/front-end/polkadot-js.md @@ -4,7 +4,7 @@ title: Polkadot-JS The Polkadot-JS project is a collection of tools, interfaces, and libraries around Polkadot and Substrate. -> **Note**: While the project is named after "Polkadot", know that these tools, interfaces, and libraries are fully compatible with any Substrate based chain. +> **Note:**: While the project is named after "Polkadot", know that these tools, interfaces, and libraries are fully compatible with any Substrate based chain. ## Polkadot-JS API diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index 73020a28b3..f5beef1b39 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -4,25 +4,25 @@ title: Transaction Fees When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, nodes charge a fee to execute a transaction. This fee must be covered -by the submitter of the transaction. The cost to execute transactions can vary, so Substrate -provides a flexible mechanism to characterize the minimum cost to include a transaction in a block. +by the signer of the transaction. The cost to execute transactions can vary, so Substrate provides a +flexible mechanism to characterize the minimum cost to include a transaction in a block. The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to -understand weights before continuing this document. +understand weights before reading this document. ## Transaction Fees -The fee to include a transaction consists of three parts: +A transaction fee consists of three parts: -* `base_fee`: a fixed fee that is applied to every transaction. See +* `base_fee`: A fixed fee that is applied to every transaction. See [`TransactionBaseFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionBaseFee). -* `length_fee`: a per-byte fee that is multiplied by the length, in bytes, of the encoded +* `length_fee`: A per-byte fee that is multiplied by the length, in bytes, of the encoded transaction. See [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee). -* `weight_fee`: a per-weight-unit fee that is multiplied by the weight of the transaction. The - weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the - weight, it must be converted to a deductible `balance` type (typically denoted by a module that - implements `Currency`, `srml-balances` in Substrate node). For this, each runtime must define a +* `weight_fee`: A per-weight-unit fee that is multiplied by the weight of the transaction. The + weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it + must be converted to a deductible `balance` type (typically denoted by a module that implements + `Currency`, `srml-balances` in Substrate node). For this, each runtime must define a [`WeightToFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) type that makes the conversion. `WeightToFee` must be a struct that implements [`Convert`](/rustdocs/master/sr_primitives/traits/trait.Convert.html). @@ -42,15 +42,14 @@ inclusion fee. ## Adjusting Multiplier The above formula gives a fee that is logically constant through time. Because the weight can be -dynamic and based on how `WeightToFee` is defined, the final fee can have some degree of -variability. As for the length fee, the inputs of the transaction could change the length and -affect the length fee. +dynamic and based on how `WeightToFee` is defined, the final fee can have some degree of +variability. As for the length fee, the inputs of the transaction could change the length and affect +the length fee. -Nonetheless, these changes are independent and _general update logic to the -entire fee cannot be composed out of them trivially_. In other words, for any dispatchable, given -the same inputs, _it will always incur the same cost_. This might not always be desirable. Chains -might need to increase or decrease fees based on some condition. To fulfill this requirement, -Substrate provides: +Nonetheless, these changes are independent and _general update logic to the entire fee cannot be +composed out of them trivially_. In other words, for any dispatchable, given the same inputs, _it +will always incur the same cost_. This might not always be desirable. Chains might need to increase +or decrease fees based on some condition. To fulfill this requirement, Substrate provides: - A multiplier stored in the Transaction Payment module that is applied to the outcome of the above formula by default (the default value of which is _multiplication identity_, meaning that @@ -89,9 +88,9 @@ page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#rel > This section assumes that you have already read the `Weight` section. -The entire SRML is already annotated with a simple and fixed weight system. A user can decide to -use the same system or implement a new one from scratch. The latter is outside the scope of this -document and is explained in the dedicated [`Weight`](/docs/conceptual/runtime/weight) conceptual +The entire SRML is already annotated with a simple and fixed weight system. A user can decide to use +the same system or implement a new one from scratch. The latter is outside the scope of this +document and is explained in the dedicated [`Weight`](/docs/conceptual/runtime/weight) conceptual document. ### Using the Default Weight @@ -118,8 +117,8 @@ pub enum SimpleDispatchInfo { This enum groups all dispatches into _normal_ and _operational_ (which makes the implementation of `ClassifyDispatch` trivial) and gives them a fixed weight. Fixed in this context means that the -arguments of the dispatch do not play any role in the weight. Dispatches classified as -_operational_ are exempt from paying both the `base_fee` and the `length_fee`. +arguments of the dispatch do not play any role in the weight. Dispatches classified as _operational_ +are exempt from paying both the `base_fee` and the `length_fee`. A simple example of using this enum in your runtime is: @@ -153,20 +152,20 @@ decl_module! { ## Other Fees -Inclusion fees don't know anything about the logic of the transaction being executed. That is, -Substrate doesn't care what happens in the transaction, it only cares about the size and weight of +Inclusion fees don't know anything about the logic of the transaction being executed. That is, +Substrate doesn't care what happens in the transaction, it only cares about the size and weight of the transaction. The inclusion fee will always be paid by the sender. -It's possible to add fees inside dispatchable functions that are only paid if certain logic paths -are executed. Most likely, this will be if the transaction succeeds. The `transfer` function in the +It's possible to add fees inside dispatchable functions that are only paid if certain logic paths +are executed. Most likely, this will be if the transaction succeeds. The `transfer` function in the Balances module, for example, takes a fixed fee for transferring tokens. -It is important to note that if you query the chain for a transaction fee, it will only return the +It is important to note that if you query the chain for a transaction fee, it will only return the inclusion fee. If you want to query internal function fees, you should emit Events for them. ## Custom Inclusion Fee Example -This is an example of how to customize your inclusion fee. You must configure the appropriate +This is an example of how to customize your inclusion fee. You must configure the appropriate associated types in the respective module. ```rust @@ -213,11 +212,12 @@ payment module drawing inspiration from Transaction Payment. ### Examples -Substrate Recipes contains examples of both -[custom weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) -and custom +Substrate Recipes contains examples of both [custom +weights](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/modules/weights) and +custom [WeightToFee](https://github.com/substrate-developer-hub/recipes/tree/master/kitchen/runtimes/weight-fee-runtime). ### References -- [Web3 Foundation Research](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees-and-per-block-transaction-limits) +- [Web3 Foundation + Research](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees-and-per-block-transaction-limits) diff --git a/docs/overview/session-keys.md b/docs/overview/session-keys.md index 27d3e95b62..fc27339526 100644 --- a/docs/overview/session-keys.md +++ b/docs/overview/session-keys.md @@ -19,7 +19,7 @@ pub struct SessionKeys { } ``` -> **NOTE:** This code is just an example of the Substrate node at the time of writing. Refer to the runtime for the most up-to-date implementation. +> **Note:** This code is just an example of the Substrate node at the time of writing. Refer to the runtime for the most up-to-date implementation. The default Substrate node implements Session keys in the [Session module](/rustdocs/v1.0/srml_session/index.html). diff --git a/website/versioned_docs/version-1.0/overview/session-keys.md b/website/versioned_docs/version-1.0/overview/session-keys.md index 6ffbf2efb8..f2b027b13c 100644 --- a/website/versioned_docs/version-1.0/overview/session-keys.md +++ b/website/versioned_docs/version-1.0/overview/session-keys.md @@ -21,7 +21,7 @@ pub struct SessionKeys { } ``` -> **NOTE:** This code is just an example of the Substrate node at the time of writing. Refer to the runtime for the most up-to-date implementation. +> **Note:** This code is just an example of the Substrate node at the time of writing. Refer to the runtime for the most up-to-date implementation. The default Substrate node implements Session keys in the [Session module](/rustdocs/v1.0/srml_session/index.html). From b4aac7ef4edc835ac2f047c3330c85e24d012377 Mon Sep 17 00:00:00 2001 From: Shawn Tabrizi Date: Thu, 7 Nov 2019 01:39:26 +0100 Subject: [PATCH 26/26] Refactor custom weight --- docs/conceptual/runtime/weight.md | 74 --------- docs/development/module/fees.md | 251 +++++++++++++++++++----------- 2 files changed, 161 insertions(+), 164 deletions(-) diff --git a/docs/conceptual/runtime/weight.md b/docs/conceptual/runtime/weight.md index 0e4c574ea8..8e90d8542a 100644 --- a/docs/conceptual/runtime/weight.md +++ b/docs/conceptual/runtime/weight.md @@ -52,80 +52,6 @@ For example, if the block length limit is 1 megabyte and the ratio is set to 80% can fill the first 800 kilobytes of the block while the last 200 can only be filled by the operational class. -## Custom Weight Implementation - - - -Implementing a custom weight calculation function can vary in complexity. Using the -`SimpleDispatchInfo` struct provided by Substrate is the easiest approach. - -A weight calculation function must provide two trait implementations: - - - [`WeightData`]: To determine the weight of the dispatch. - - [`ClassifyDispatch`]: To determine the class of the dispatch. - -Substrate then bundles the output information of the two traits into the [`DispatchInfo`] struct and -provides it by implementing the [`GetDispatchInfo`] for all `Call` variants and opaque extrinsic -types. This is used internally by the System and Executive modules; you probably won't use it. - -Both `ClassifyDispatch` and `WeightData` are generic over `T`, which gets resolved into the tuple of -all dispatch arguments except for the origin. To demonstrate, we will craft a struct that calculates -the weight as `m * len(args)` where `m` is a given multiplier and `args` is the concatenated tuple -of all dispatch arguments. Further, the dispatch class is `Operational` if the transaction has more -than 100 bytes of length in arguments. - -```rust -use coded::Encode; -use sr_primitives::weights::{DispatchClass, ClassifyDispatch, WeightData} - -// self.0 is the multiplier, `m` -struct LenWeight(u32); - -// We don't quite know what T is. After all, different dispatches have different arguments, hence -// `T` will be different. All that we care about is that `T` is encodable. That is always true by -// definition. All dispatch arguments are encodable. -impl WeighData for LenWeight { - fn weigh_data(&self, target: T) -> Weight { - let multiplier = self.0; - let encoded_len = target.encode().len() as u32; - multiplier * encoded_len - } -} - -impl ClassifyDispatch for LenWeight { - fn classify_dispatch(&self, target: T) -> DispatchClass { - let encoded_len = target.encode().len() as u32; - if encoded_len > 100 { - DispatchClass::Operational - } else { - DispatchClass::Normal - } - } -} -``` - -A weight calculator function can also be coerced to the final type of the argument, instead of -defining it as a vague type that is encodable. `srml-example` contains an example of how to do this. -Just note that, in that case, your code would roughly look like: - -```rust -struct CustomWeight; -impl WeighData<(&u32, &u64)> for CustomWeight { - fn weigh_data(&self, target: (&u32, &u64)) -> Weight { - ... - } -} - -// given dispatch: -decl_module! { - fn foo(a: u32, b: u64) { ... } -} -``` - -This means that `CustomWeight` can only be used in conjunction with a dispatch with a particular -signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they -don't make any strict assumptions about ``. - ## Next Steps ### Learn More diff --git a/docs/development/module/fees.md b/docs/development/module/fees.md index f5beef1b39..b0636d5aaa 100644 --- a/docs/development/module/fees.md +++ b/docs/development/module/fees.md @@ -4,13 +4,84 @@ title: Transaction Fees When transactions are submitted to a blockchain, they are executed by the nodes in the network. To be economically sustainable, nodes charge a fee to execute a transaction. This fee must be covered -by the signer of the transaction. The cost to execute transactions can vary, so Substrate provides a +by the sender of the transaction. The cost to execute transactions can vary, so Substrate provides a flexible mechanism to characterize the minimum cost to include a transaction in a block. The fee system is heavily linked to the [weight system](conceptual/runtime/weight.md). Make sure to understand weights before reading this document. -## Transaction Fees +## Default Fees + +The default fee system uses the uses _fixed_ weights represented by the following enum: + +```rust +pub enum SimpleDispatchInfo { + /// A normal dispatch with fixed weight. + FixedNormal(Weight), + /// A normal dispatch with the maximum weight. + MaxNormal, + /// A normal dispatch with no weight. + FreeNormal, + /// An operational dispatch with fixed weight. + FixedOperational(Weight), + /// An operational dispatch with the maximum weight. + MaxOperational, + /// An operational dispatch with no weight. + FreeOperational, +} +``` + +This enum groups all dispatches into _normal_ or _operational_ and gives them a fixed weight. This +means that the arguments of a dispatchable function do not affect the weight. A dispatch classified +as _operational_ is exempt from paying both the `base_fee` and the `length_fee`. + +### Free Normal + +This means that this function has no weight. It will not contribute to block fullness at all, and no +weight-fee is applied. + +```rust +#[weight = SimpleDispatchInfo::FreeNormal] +pub fn some_normal_function_light() { noop(); } +``` + +### Fixed Normal + +This function will always have a weight `10`. + +```rust +#[weight = SimpleDispatchInfo::FixedNormal(10)] +pub fn some_normal_function_heavy() { some_computation(); } +``` + +### Fixed Operational + +This function will have a fixed weight but can consume the reserved operational portion as well. + +```rust +#[weight = SimpleDispatchInfo::FixedOperational(20)] +pub fn mission_critical_function() { some_sudo_op(); } +``` + +### Default + +This function will automatically get `#[weight = SimpleDispatchInfo::default()]`. + +```rust +pub fn something_else() { noop(); } +``` + +> **Note:** Be careful! The default implementation of `SimpleDispatchInfo` resolves to +> `FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired +> granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that +> you would want it to have a different `Default`. + +## Fee Calculation + +The final fee of a dispatch is calculated using the weight of the function and a number of +configurable parameters. + +### Inclusion Fee A transaction fee consists of three parts: @@ -21,8 +92,7 @@ A transaction fee consists of three parts: [`TransactionByteFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.TransactionByteFee). * `weight_fee`: A per-weight-unit fee that is multiplied by the weight of the transaction. The weight of each dispatch is denoted via the flexible `#[weight]` annotation. Knowing the weight, it - must be converted to a deductible `balance` type (typically denoted by a module that implements - `Currency`, `srml-balances` in Substrate node). For this, each runtime must define a + must be converted to the `Currency` type. For this, each runtime must define a [`WeightToFee`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.WeightToFee) type that makes the conversion. `WeightToFee` must be a struct that implements [`Convert`](/rustdocs/master/sr_primitives/traits/trait.Convert.html). @@ -36,29 +106,19 @@ fee = WeightToFee(weight) ``` -This `fee` is known as the "inclusion fee." Even if the extrinsic fails, the sender must pay this +This `fee` is known as the "inclusion fee." Even if the extrinsic fails, the signer must pay this inclusion fee. -## Adjusting Multiplier +### Fee Multiplier -The above formula gives a fee that is logically constant through time. Because the weight can be -dynamic and based on how `WeightToFee` is defined, the final fee can have some degree of -variability. As for the length fee, the inputs of the transaction could change the length and affect -the length fee. +The above formula gives a fee that is always the same for the same input. However, weight can be +dynamic and based on how `WeightToFee` is defined, the final fee can include some degree of +variability. To fulfill this requirement, Substrate provides: -Nonetheless, these changes are independent and _general update logic to the entire fee cannot be -composed out of them trivially_. In other words, for any dispatchable, given the same inputs, _it -will always incur the same cost_. This might not always be desirable. Chains might need to increase -or decrease fees based on some condition. To fulfill this requirement, Substrate provides: - - - A multiplier stored in the Transaction Payment module that is applied to the outcome of the - above formula by default (the default value of which is _multiplication identity_, meaning that - it has no effect). This is stored in - [`NextFeeMultiplier`](/rustdocs/master/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier) - and can be configured through the genesis spec of the module. - - A configurable parameter for a runtime to describe how this multiplier can change. This is - expressed via - [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate). + - [`NextFeeMultiplier`](/rustdocs/master/srml_transaction_payment/struct.Module.html#method.next_fee_multiplier): + A multiplier stored in the Transaction Payment module and configurable. + - [`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate): + A configurable parameter for a runtime to describe how this multiplier can change. `NextFeeMultiplier` has the type `Fixed64`, which can represent a fixed point number. So, given the inclusion fee formula above, the final version would be: @@ -73,9 +133,10 @@ final_fee = fee * NextFeeMultiplier ``` Updating the `NextFeeMultiplier` has a similar effect as updating `WeightToFee`. The -[`FeeMultiplierUpdate`](/rustdocs/master/srml_transaction_payment/trait.Trait.html#associatedtype.FeeMultiplierUpdate) -associated type in `transaction-payment` is defined as a `Convert`, which should -be read: "it receives the previous multiplier and returns the next one". +`FeeMultiplierUpdate` associated type in Transaction Payment module is defined as a +`Convert`, which should be read: + +> "it receives the previous multiplier and returns the next one". The default update function is inspired by the Polkadot network and implements a targeted adjustment in which a target saturation level of block weight is defined. If the previous block is more @@ -84,86 +145,96 @@ transactions than the target, fees are decreased by a small amount. More informa be found in the [Web3 research page](https://research.web3.foundation/en/latest/polkadot/Token%20Economics/#relay-chain-transaction-fees). -## Substrate Weight System +### Additional Fees -> This section assumes that you have already read the `Weight` section. +Inclusion fees don't know anything about the logic of the transaction being executed. That is, +Substrate doesn't care what happens in the transaction, it only cares about the size and weight of +the transaction. The inclusion fee will always be paid by the sender. -The entire SRML is already annotated with a simple and fixed weight system. A user can decide to use -the same system or implement a new one from scratch. The latter is outside the scope of this -document and is explained in the dedicated [`Weight`](/docs/conceptual/runtime/weight) conceptual -document. +It's possible to add fees inside dispatchable functions that are only paid if certain logic paths +are executed. Most likely, this will be if the transaction succeeds. The `transfer` function in the +Balances module, for example, takes a fixed fee for transferring tokens. -### Using the Default Weight +It is important to note that if you query the chain for a transaction fee, it will only return the +inclusion fee. -The default weight annotation is simple. Substrate, by default, uses _fixed_ weights and the enum -representing them is as follows: +## Custom Fees -```rust -pub enum SimpleDispatchInfo { - /// A normal dispatch with fixed weight. - FixedNormal(Weight), - /// A normal dispatch with the maximum weight. - MaxNormal, - /// A normal dispatch with no weight. - FreeNormal, - /// An operational dispatch with fixed weight. - FixedOperational(Weight), - /// An operational dispatch with the maximum weight. - MaxOperational, - /// An operational dispatch with no weight. - FreeOperational, -} -``` +You can also define custom fee systems through custom weight functions or inclusion fee functions. + +### Custom Weights + +Implementing a custom weight calculation function can vary in complexity. Using the +`SimpleDispatchInfo` struct provided by Substrate is the easiest approach. -This enum groups all dispatches into _normal_ and _operational_ (which makes the implementation of -`ClassifyDispatch` trivial) and gives them a fixed weight. Fixed in this context means that the -arguments of the dispatch do not play any role in the weight. Dispatches classified as _operational_ -are exempt from paying both the `base_fee` and the `length_fee`. +A weight calculation function must provide two trait implementations: -A simple example of using this enum in your runtime is: + - [`WeightData`]: To determine the weight of the dispatch. + - [`ClassifyDispatch`]: To determine the class of the dispatch. + +Substrate then bundles the output information of the two traits into the [`DispatchInfo`] struct and +provides it by implementing the [`GetDispatchInfo`] for all `Call` variants and opaque extrinsic +types. This is used internally by the System and Executive modules; you probably won't use it. + +Both `ClassifyDispatch` and `WeightData` are generic over `T`, which gets resolved into the tuple of +all dispatch arguments except for the origin. To demonstrate, we will craft a struct that calculates +the weight as `m * len(args)` where `m` is a given multiplier and `args` is the concatenated tuple +of all dispatch arguments. Further, the dispatch class is `Operational` if the transaction has more +than 100 bytes of length in arguments. ```rust -use sr_primitives::weights::{SimpleDispatchInfo}; +use coded::Encode; +use sr_primitives::weights::{DispatchClass, ClassifyDispatch, WeightData} + +// self.0 is the multiplier, `m` +struct LenWeight(u32); + +// We don't quite know what T is. After all, different dispatches have different arguments, hence +// `T` will be different. All that we care about is that `T` is encodable. That is always true by +// definition. All dispatch arguments are encodable. +impl WeighData for LenWeight { + fn weigh_data(&self, target: T) -> Weight { + let multiplier = self.0; + let encoded_len = target.encode().len() as u32; + multiplier * encoded_len + } +} -decl_module! { - // This means that this function has no weight. It will not contribute to block fullness at all, - // and no weight-fee is applied. - #[weight = SimpleDispatchInfo::FreeNormal] - pub fn some_normal_function_light() { noop(); } - - // This function will always have a weight `10`. - #[weight = SimpleDispatchInfo::FixedNormal(10)] - pub fn some_normal_function_heavy() { some_computation(); } - - // This function will have a fixed weight but can consume the reserved operational portion as - // well. - #[weight = SimpleDispatchInfo::FixedOperational(20)] - pub fn mission_critical_function() { some_sudo_op(); } - - // This will automatically get `#[weight = SimpleDispatchInfo::default()]`. - pub fn something_else +impl ClassifyDispatch for LenWeight { + fn classify_dispatch(&self, target: T) -> DispatchClass { + let encoded_len = target.encode().len() as u32; + if encoded_len > 100 { + DispatchClass::Operational + } else { + DispatchClass::Normal + } + } } ``` -> **Note:** Be careful! The default implementation of `SimpleDispatchInfo` resolves to -> `FixedNormal(10_000)`. This is due to how things work in `substrate-node` and the desired -> granularity of substrate. Even if you want to use the `SimpleDispatchInfo`, it is very likely that -> you would want it to have a different `Default`. - -## Other Fees +A weight calculator function can also be coerced to the final type of the argument, instead of +defining it as a vague type that is encodable. `srml-example` contains an example of how to do this. +Just note that, in that case, your code would roughly look like: -Inclusion fees don't know anything about the logic of the transaction being executed. That is, -Substrate doesn't care what happens in the transaction, it only cares about the size and weight of -the transaction. The inclusion fee will always be paid by the sender. +```rust +struct CustomWeight; +impl WeighData<(&u32, &u64)> for CustomWeight { + fn weigh_data(&self, target: (&u32, &u64)) -> Weight { + ... + } +} -It's possible to add fees inside dispatchable functions that are only paid if certain logic paths -are executed. Most likely, this will be if the transaction succeeds. The `transfer` function in the -Balances module, for example, takes a fixed fee for transferring tokens. +// given dispatch: +decl_module! { + fn foo(a: u32, b: u64) { ... } +} +``` -It is important to note that if you query the chain for a transaction fee, it will only return the -inclusion fee. If you want to query internal function fees, you should emit Events for them. +This means that `CustomWeight` can only be used in conjunction with a dispatch with a particular +signature `(u32, u64)`, as opposed to `LenWeight`, which can be used with anything because they +don't make any strict assumptions about ``. -## Custom Inclusion Fee Example +### Custom Inclusion Fee This is an example of how to customize your inclusion fee. You must configure the appropriate associated types in the respective module. @@ -206,7 +277,7 @@ payment module drawing inspiration from Transaction Payment. ### Learn More -- Dedicated [weight documentation](/docs/conceptual/runtime/weight) +- Dedicated [weight documentation](conceptual/runtime/weight.md) - [Example module](https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs) - [SignedExtension](/rustdocs/master/sr_primitives/traits/trait.SignedExtension.html)