From ed54c822c682655a418538a734b40be9c2141f1c Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Thu, 7 Oct 2021 16:46:21 +0100 Subject: [PATCH 01/15] tx fees first version --- flips/20211007-transaction-fees.md | 233 +++++++++++++++++++++++++++++ 1 file changed, 233 insertions(+) create mode 100644 flips/20211007-transaction-fees.md diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md new file mode 100644 index 000000000..89a8ef2f6 --- /dev/null +++ b/flips/20211007-transaction-fees.md @@ -0,0 +1,233 @@ + + + + +# Transaction fees + +| Status | Proposed | +:-------------- |:---------------------------------------------------- | +| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/NNN) (update when you have PR #)| +| **Author(s)** | Janez Podhostnik (janez.podhostnik@dapperlabs.com) | +| **Updated** | 2021-10-07 | + +## Objective + +Change the calculation of transaction fees on the FLOW network to better secure and stabilise the network. + +## Motivation + +Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tends to the optimum throughput, they should also help deter malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. + +## Current design + +Currently transaction fees are the same for every transaction and don't change automatically over time. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). + +The fees are deducted from the transaction payers account automatically. If the fee deduction fails, no state change from the transaction is committed. + +The paid transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal. + +There is an existing concept of computation limit (a.k.a.: gas limit), but it is not tied to transaction fees. The computation limit currently only prevents transactions to run for a very long time. + +## Design requirements + +We can separate the requirements into two perspectives. + +From the protocols perspective the following requirements can be set: + +- Larger (byte size) transactions take more bandwidth and should be costlier than smaller transactions. +- Transactions that are computationally more demanding should be costlier than simpler transactions. +- If the protocol is experiencing a lot of traffic, all transactions should become more expensive. + +From the users perspective the transaction fees implementation should satisfy the following criteria: + +- Fees should be easy to understand: + - It should be made clear why they exist in the form they do, and what else was considered. + - It should be easy to explain how to calculate them, and how to view the actual fees that were deducted. +- It should be possible to calculate the transaction fees (within a reasonable margin), before actually sending a transaction. +- The price of sending a specific transaction should not fluctuate over time so quickly that the user cannot respond to it. +- It should be possible to see the details of the transaction fee calculation after the transaction is executed. + +## Proposed design + +The idea for this design is to break down the transaction fees into smaller parts that are easier to define control and implement. + +Lets first separate transaction fees () into inclusion fees () and computation fees (). The inclusion fees represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the operational cost of executing the transaction script. + +This separation allows for finer control over the type of transactions that are desired. For example if the volume of transactions is high and is causing congestion on the the collection nodes but the execution nodes have no problem executing all of the transactions, the inclusion fees can be increased without changing the computation fees. + +Each part of the fees can be further split into *effort*() and a *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when we send the transaction[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* change over time depending on network activity. + +[^1]: Assuming there weren't any changes to the state that would make the transaction script behave differently. + +If we could send a completely empty transaction (without any script, arguments, or signatures; *nil transaction*), it would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. Because of this the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depends on the transaction properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. + +The final result in equation form looks like this: + + + +- : Inclusion fees; Represent the cost of including the transaction in a collection. + - : Inclusion effort cost factor; The current cost of one unit of inclusion effort. + - : Inclusion effort; The time invariant effort required to include this transaction into a collection. + - : Constant inclusion effort; The effort of including a *nil transaction* into a collection + - : Dynamic inclusion effort; The increase in effort due to the particulars transaction properties (due to its size in bytes). +- : Computation fees; Represent the cost of executing the transaction. + - : Computation effort cost factor; The current cost of one unit of computation effort. + - : Computation effort; The time invariant effort required to execute this transactions script. + - : Constant computation effort; The effort of executing a *nil transaction*. + - : Dynamic computation effort; The effort of executing the transactions script. + + +### Fee effort cost factors + +The effort cost factors and are used to change the cost of transaction fees over time, and should be changed depending on how much load the network is under. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. + +Adjusting the effort cost factors should be done with an automatic process that responds to the network load. But this does not need to be done right away. + +They should be defined on a smart contract and adjustable via the service account. They should be accessible for everyone to read. + +Another reason to tweak the cost factors is that over time some parts of the code will get optimized so the same amount of effort will get cheaper to handle. An example would be that if execution of transactions became 10% faster because of optimizations, the computation effort cost factor could be reduced by 10%. + +A potential third use of the effort cost factors would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factors to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. + +### Dynamic inclusion effort + +The definition of a unit of (inclusion) effort is an abstract term, and can be defined as a byte of the rlp encoded transaction. + +The constant inclusion effort should be chosen by observing the impact of transactions with a different byte size and a similar computation effort and using linear regression to see how much impact a 0 bytes long transaction would make. + +Because there is a positive constant inclusion effort as well as a dynamic one it would not make sense for users to split their transaction into smaller pieces (considering only the inclusion part of the transaction fees). If this would be desired from the protocols perspective, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. + +### Dynamic computation effort + +During the calculation of fees there is one part that is unknown until the transaction is actually executed which it the dynamic computation effort. The reason for this is that the execution of the transaction script depends on the current state. + +The consequence of this is that the fees are not exactly known before the transaction is submitted and executed. + +The minimum bound for transaction fees () can still be calculated by assuming the dynamic computation effort is 0. + + + + +The maximum transaction fees should be decided by the payer. The way to specify the maximum transaction fees is by setting the maximum computation limit () (currently known as the gas limit). If the computation limit is hit during the execution of thr transaction the execution stops and the state changes made up to that point are dropped. The transaction fees are still collected. + +Currently the dynamic computation effort is calculated by adding 1 for every loop or function call. This can be sufficient for the first iteration, but in the future this would be changed so that function calls would have a different computational effort assigned to them according to the complexity of the method. + +### Failing transactions + +There can be multiple reasons for a transaction to fail. For the purpose of transaction fees failures can be split into four scenarios, according to who gets charged and how much. + +1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the dynamic computation effort set to the computation limit). +2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) +3. Transaction failed during transaction script execution, during fee deduction or during the storage used check. +4. Transaction failed because the computation limit was reached. + +As mentioned earlier in scenario 1. the transaction shouldn't have been included in a collection in the first place. Because it was and there is no way to charge the payer, the collection node is charged with the transaction fees where the dynamic computation effort is set to 0. + +In the second scenario the payer is charged for the transaction fees with the dynamic computation effort is set to 0. + +In the third case, the fees are charged normally with the dynamic computation effort set to the actual usage before the error was produced. + +Finally in the last case the payer is charge with the transaction fees where the dynamic computation effort is set to the computation limit. + +### Preventing malicious actors + +The purpose of fees is to prevent malicious actors sending transactions with the purpose of overloading the network. To achieve that, we have to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. + +Tho achieve this the following checks should happen on the collection node, before the transaction is included in a transaction: +- Is the payers signature correct. +- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. + +If this transaction still gets included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the collection nodes account instead of the payers account. + +### Calibrating the parameters + +The goal of the calibration illustrated below is to have transaction fees comparable to the current constant fees (`TC`) + +- Dynamic inclusion effort would be the byte size of a transaction +- : Constant inclusion effort would be picked by using linear regression on the impact of different size transactions (with the same computational effort) on the network, and extrapolating for a 0 size transaction. +- : Inclusion effort cost factor would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then `TC/2`. + +- : Dynamic computation effort can be kept as the computation effort that is currently in place. This can be later upgraded so the more expensive methods also cost more computation effort. +- : Constant computation effort would be picked by using a similar method as for the constant inclusion effort. +- : Computation effort cost factor would be picked so that the at the maximum computation limit the computation fees would be `TC/2`. + +### Viewing the deducted fees + +After a transaction was executed the fee breakdown should be visible in the fee deduction event. All of the components of the fee calculation should be inferable from the state of the network at the time when the transaction was executed and the event data. + +The effort cost factors and the constant inclusion/dynamic effort can be read from the state, so the remaining data needed on the transaction fee event are the dynamic inclusion effort an the dynamic calculation effort. For redundancy the total amount of fees should also be a field of the fee deduction event. + +### Transaction Fee estimation + +To estimate the minimum transaction fees () needed for a transaction some data needs to be retrieved from the state (access node): +- The effort cost factors. +- The constant inclusion effort. +- The constant dynamic effort. + +This could be done by the SDK with a script or a possibly a dedicated endpoint on the access nodes. + +Due to the effort cost factors slowly changing, this calculation will still be just an estimation, but if the effort cost factors change slowly enough the estimation will be fairly close. + +After the user has chosen a good value for the computation limit. The maximum transaction fees can be estimated as well. + +## Implementation + +### Step 1. + +Decoupling the concept of effort and effort fee factors. The fee calculation would still now have any dynamic parts they would be reported as 0. The parts that would change are: + +- The transaction fee deduction event emitted would also emit 0 as the dynamic inclusion effort and the dynamic computation effort. +- The constant efforts would be set to a value that would approximate the effort of a nil transaction. +- The fee effort cost factors would be set so that there is no change to the final fees. +- the fee calculation would be done inside the `FlowFees` smart contract + +Implementing this step would have no impact on users, but they could start reading the fee breakdown from the emitted fee event. + +This would also set up the stage nicely for the following steps + +### Step 2. + +Introduction of dynamic inclusion effort. + +- the dynamic inclusion effort would be set to the byte size of the transaction +- other inclusion parameters would be adjusted so that 95% of transactions would be below the current transaction prices. + +All of the code pieces would already be in place, so the main part of this task would be choosing good parameter values. + +### Step 3. + +Introduction of the basic dynamic computation effort. + +- use the current computation usage calculation as the dynamic computation effort +- other computation parameters would be adjusted so that 95% of transactions would be below the current transaction prices. +- add checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on collection nodes) + +This would mean that all the "happy path" and most of the failure paths are in place, but the computation metering is still very rudimentary. + +### Step 4. + +Make collection nodes ignore transactions that cannot be paid for. + +- Add payer eligibility checks to the collection nodes. +- Add mechanism that the collection node pays for transactions it included in collections where the payer could not have paid. + +### Next steps + +- Improve dynamic computation effort calculation. +- Add mechanism for changing the fee effort cost factors to make fees higher when the traffic is high. +- Parameter fine tuning. + + +## User impact + +Users are already aware of the computation limit (a.k.a.: gas limit), but they will now need to pay more attention to it as it has a few new implications: +- setting a high computation limit will mean that the payer needs to have more funds in their account before sending the transaction. +- using more computation will result in more expensive transactions + +Users will also need to pay attention to the transaction fees effort cost factors that will slowly change over time. It sometimes might be prudent to wait for a less busy (and cheaper) time to send a transaction. + + +## Questions and Discussion Topics + +Q: While there is an incentive for collection nodes not to include transactions from payers that potentially might not be able to pay for those transactions, what is the incentive in the opposite direction? If a payer was a bad payer and collection nodes stop including his transactions, what motivates the collection nodes to then start including transactions from that payer, if the payer has added funding to their account? + From c919cc5d20e22025a6e2a2555b03ddb66bac044f Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Fri, 8 Oct 2021 17:14:51 +0100 Subject: [PATCH 02/15] minor fixes --- flips/20211007-transaction-fees.md | 37 +++++++++++++++++------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 89a8ef2f6..c433b5b26 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -6,7 +6,7 @@ | Status | Proposed | :-------------- |:---------------------------------------------------- | -| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/NNN) (update when you have PR #)| +| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/660) (update when you have PR #)| | **Author(s)** | Janez Podhostnik (janez.podhostnik@dapperlabs.com) | | **Updated** | 2021-10-07 | @@ -20,46 +20,48 @@ Transaction fees should allow the Flow blockchain to self regulate transaction t ## Current design -Currently transaction fees are the same for every transaction and don't change automatically over time. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). +Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). -The fees are deducted from the transaction payers account automatically. If the fee deduction fails, no state change from the transaction is committed. +[^2]: Except when an explicit decision is made to change them. -The paid transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal. +The fees are deducted from the transaction payer automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. -There is an existing concept of computation limit (a.k.a.: gas limit), but it is not tied to transaction fees. The computation limit currently only prevents transactions to run for a very long time. +The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. + +There is an existing concept of computation limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The computation limit is currently only used prevent transactions from running for a very long time. ## Design requirements -We can separate the requirements into two perspectives. +We can separate the requirements into two groups depending on perspective. From the protocols perspective the following requirements can be set: - Larger (byte size) transactions take more bandwidth and should be costlier than smaller transactions. - Transactions that are computationally more demanding should be costlier than simpler transactions. -- If the protocol is experiencing a lot of traffic, all transactions should become more expensive. +- If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. From the users perspective the transaction fees implementation should satisfy the following criteria: - Fees should be easy to understand: - - It should be made clear why they exist in the form they do, and what else was considered. - - It should be easy to explain how to calculate them, and how to view the actual fees that were deducted. + - It should be made clear why they exist in the form they do. + - It should be easy to explain how the fees are calculated. - It should be possible to calculate the transaction fees (within a reasonable margin), before actually sending a transaction. -- The price of sending a specific transaction should not fluctuate over time so quickly that the user cannot respond to it. +- The price of sending a specific transaction should not fluctuate over time so quickly that the user cannot respond to the fluctuation. - It should be possible to see the details of the transaction fee calculation after the transaction is executed. ## Proposed design -The idea for this design is to break down the transaction fees into smaller parts that are easier to define control and implement. +The idea for this design is to break down the transaction fees into smaller parts that are easier to define, control, and implement. -Lets first separate transaction fees () into inclusion fees () and computation fees (). The inclusion fees represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the operational cost of executing the transaction script. +The first separation of the transaction fees () is into inclusion fees () and computation fees (). The inclusion fees represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the operational cost of executing the transaction script. This separation allows for finer control over the type of transactions that are desired. For example if the volume of transactions is high and is causing congestion on the the collection nodes but the execution nodes have no problem executing all of the transactions, the inclusion fees can be increased without changing the computation fees. -Each part of the fees can be further split into *effort*() and a *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when we send the transaction[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* change over time depending on network activity. +Each part of the fees can be further split into *effort*() and a *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* would change over time depending on network activity. [^1]: Assuming there weren't any changes to the state that would make the transaction script behave differently. -If we could send a completely empty transaction (without any script, arguments, or signatures; *nil transaction*), it would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. Because of this the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depends on the transaction properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. +If we could send a completely empty transaction (without any script, arguments, or signatures; a *nil transaction*), it would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. Because of this the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depend on the transactions properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. The final result in equation form looks like this: @@ -81,14 +83,17 @@ The final result in equation form looks like this: The effort cost factors and are used to change the cost of transaction fees over time, and should be changed depending on how much load the network is under. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. -Adjusting the effort cost factors should be done with an automatic process that responds to the network load. But this does not need to be done right away. - They should be defined on a smart contract and adjustable via the service account. They should be accessible for everyone to read. Another reason to tweak the cost factors is that over time some parts of the code will get optimized so the same amount of effort will get cheaper to handle. An example would be that if execution of transactions became 10% faster because of optimizations, the computation effort cost factor could be reduced by 10%. A potential third use of the effort cost factors would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factors to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. +### Fee effort cost factors automation (surge pricing) + +A mechanism would be in place to increase the price of transactions if the network is experiencing load. +This would be done by increasing one or both of the fee effort cost factors. This would incentivise users that can wait before sending their transaction to do so. + ### Dynamic inclusion effort The definition of a unit of (inclusion) effort is an abstract term, and can be defined as a byte of the rlp encoded transaction. From 6999d115683e7552bebc597e21a377144558f573 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Wed, 13 Oct 2021 17:22:22 +0100 Subject: [PATCH 03/15] second pass --- flips/20211007-transaction-fees.md | 59 ++++++++++++++++-------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index c433b5b26..2f4f7ed63 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -2,11 +2,11 @@ -# Transaction fees +# Variable Transaction fees | Status | Proposed | :-------------- |:---------------------------------------------------- | -| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/660) (update when you have PR #)| +| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/660)| | **Author(s)** | Janez Podhostnik (janez.podhostnik@dapperlabs.com) | | **Updated** | 2021-10-07 | @@ -16,11 +16,11 @@ Change the calculation of transaction fees on the FLOW network to better secure ## Motivation -Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tends to the optimum throughput, they should also help deter malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. +Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tends to the optimum throughput, they should also discourage malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. ## Current design -Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). +Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). [^2]: Except when an explicit decision is made to change them. @@ -28,7 +28,7 @@ The fees are deducted from the transaction payer automatically. If the transacti The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. -There is an existing concept of computation limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The computation limit is currently only used prevent transactions from running for a very long time. +There is an existing concept of a computation limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The computation limit is currently used only to prevent transactions from running for a very long time (or indefinitely). ## Design requirements @@ -40,6 +40,8 @@ From the protocols perspective the following requirements can be set: - Transactions that are computationally more demanding should be costlier than simpler transactions. - If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. +The ultimate goal from the protocols perspective would be that transactions cost exactly the amount of FLOW needed to pay for the cost of the time needed by the nodes to process the transaction. However this would include so many variables that it would not be feasible. Instead the goal is to make a reasonably good approximation, and improve that approximation over time. + From the users perspective the transaction fees implementation should satisfy the following criteria: - Fees should be easy to understand: @@ -51,17 +53,17 @@ From the users perspective the transaction fees implementation should satisfy th ## Proposed design -The idea for this design is to break down the transaction fees into smaller parts that are easier to define, control, and implement. +The idea for this design is to the transaction fees break down into smaller parts that are easier to define, control, and implement. -The first separation of the transaction fees () is into inclusion fees () and computation fees (). The inclusion fees represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the operational cost of executing the transaction script. +The first separation of the transaction fees () is into inclusion fees () and computation fees (). The inclusion fees roughly represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the approximate operational cost of executing the transaction script. This separation allows for finer control over the type of transactions that are desired. For example if the volume of transactions is high and is causing congestion on the the collection nodes but the execution nodes have no problem executing all of the transactions, the inclusion fees can be increased without changing the computation fees. -Each part of the fees can be further split into *effort*() and a *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* would change over time depending on network activity. +Each part of the fees can be further split into *effort*() and an *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* would change over time depending on network activity. -[^1]: Assuming there weren't any changes to the state that would make the transaction script behave differently. +[^1]: Assuming there weren't any changes to the state that would make the transactions script execute differently. -If we could send a completely empty transaction (without any script, arguments, or signatures; a *nil transaction*), it would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. Because of this the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depend on the transactions properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. +It is possible to drill down even further by imagining what work would be needed for a theoretical transaction that is completely empty (without any script, arguments, or signatures; a *nil transaction*). A transaction like this would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. This means that the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depend on the transactions properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. The final result in equation form looks like this: @@ -81,45 +83,46 @@ The final result in equation form looks like this: ### Fee effort cost factors -The effort cost factors and are used to change the cost of transaction fees over time, and should be changed depending on how much load the network is under. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. +The effort cost factors and should be defined on a smart contract and adjustable via the service account admin resource and they should be accessible for everyone to read. + +The effort cost factors could be used in three different ways: -They should be defined on a smart contract and adjustable via the service account. They should be accessible for everyone to read. +1. The main usage of the effort cost factors is adjusting them when the network is under load. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. -Another reason to tweak the cost factors is that over time some parts of the code will get optimized so the same amount of effort will get cheaper to handle. An example would be that if execution of transactions became 10% faster because of optimizations, the computation effort cost factor could be reduced by 10%. +2. The second reason to tweak the effort cost factors is that over time some parts of the code will get optimized. This could cause the same amount of effort to be easier to handle. An example would be that if execution of transactions became 10% faster because of optimizations, the computation effort cost factor could be reduced by 10%. -A potential third use of the effort cost factors would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factors to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. +3. A potential third use of the effort cost factors would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factors to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. -### Fee effort cost factors automation (surge pricing) +While the effort cost factors could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the effort cost factors automatically would be a service that monitored the networks performance and sent a transaction to update the fee factors when there was a change in the network load. -A mechanism would be in place to increase the price of transactions if the network is experiencing load. -This would be done by increasing one or both of the fee effort cost factors. This would incentivise users that can wait before sending their transaction to do so. +### Inclusion effort -### Dynamic inclusion effort +For the first iteration the dynamic inclusion effort can be defined to be equal to the byte length of the (rlp encoded) transaction. The maximum dynamic inclusion effort would be equal to the maximum allowed transaction size, which is currently 1,500,000 bytes. -The definition of a unit of (inclusion) effort is an abstract term, and can be defined as a byte of the rlp encoded transaction. +The constant inclusion effort could then be chosen by observing the impact of transactions with a different byte size and a similar computation effort. This data would then be used to predict how much impact a 0 bytes long transaction would make. -The constant inclusion effort should be chosen by observing the impact of transactions with a different byte size and a similar computation effort and using linear regression to see how much impact a 0 bytes long transaction would make. +Having a positive constant inclusion effort as well as a dynamic one means that splitting a transaction into smaller pieces does not make sense (considering only the inclusion part of the transaction fees). If this would be desired (from the protocols perspective) that the transactions are as small as possible or of a certain size, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. -Because there is a positive constant inclusion effort as well as a dynamic one it would not make sense for users to split their transaction into smaller pieces (considering only the inclusion part of the transaction fees). If this would be desired from the protocols perspective, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. -### Dynamic computation effort +### Computation effort -During the calculation of fees there is one part that is unknown until the transaction is actually executed which it the dynamic computation effort. The reason for this is that the execution of the transaction script depends on the current state. +The computation effort represents the effort needed to execute the transaction script and to do pre and post execution steps (like checking signatures and storage limits and deducting transaction fees). -The consequence of this is that the fees are not exactly known before the transaction is submitted and executed. +Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the computation effort before the actual on chain execution of the transaction. -The minimum bound for transaction fees () can still be calculated by assuming the dynamic computation effort is 0. +The lower bound for computation fees (and thus transaction fees) () can still be calculated by assuming the dynamic computation effort is 0. +The upper bound of the computation fees (and thus transaction fees) is indirectly set in the transaction by setting the maximum computation limit () (currently known as the gas limit). -The maximum transaction fees should be decided by the payer. The way to specify the maximum transaction fees is by setting the maximum computation limit () (currently known as the gas limit). If the computation limit is hit during the execution of thr transaction the execution stops and the state changes made up to that point are dropped. The transaction fees are still collected. +If the transaction computation limit is reached during the execution of the transaction the execution stops and the state changes made up to that point are dropped. The transaction fees for this transaction are still collected. -Currently the dynamic computation effort is calculated by adding 1 for every loop or function call. This can be sufficient for the first iteration, but in the future this would be changed so that function calls would have a different computational effort assigned to them according to the complexity of the method. +Currently the dynamic computation effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the dynamic computation effort would be changed so that function calls would have a different computational effort assigned to them according to the time complexity of the method. ### Failing transactions -There can be multiple reasons for a transaction to fail. For the purpose of transaction fees failures can be split into four scenarios, according to who gets charged and how much. +There are a few reasons why a transaction fails, but for the purpose of transaction fees, transaction failures can be split into four categories, according to who then gets charged for the transaction fees and how much do they get charged. 1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the dynamic computation effort set to the computation limit). 2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) From d176fdb5f482c41e4989b3324b787d0de5a1c072 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 18 Oct 2021 18:40:16 +0100 Subject: [PATCH 04/15] updates --- flips/20211007-transaction-fees.md | 71 +++++++++++-------- .../fees_diagram.svg | 1 + 2 files changed, 44 insertions(+), 28 deletions(-) create mode 100644 flips/20211007-transaction-fees/fees_diagram.svg diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 2f4f7ed63..eca242c99 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -69,6 +69,8 @@ The final result in equation form looks like this: +![Diagram](20211007-transaction-fees/fees_diagram.svg) + - : Inclusion fees; Represent the cost of including the transaction in a collection. - : Inclusion effort cost factor; The current cost of one unit of inclusion effort. - : Inclusion effort; The time invariant effort required to include this transaction into a collection. @@ -85,7 +87,7 @@ The final result in equation form looks like this: The effort cost factors and should be defined on a smart contract and adjustable via the service account admin resource and they should be accessible for everyone to read. -The effort cost factors could be used in three different ways: +The effort cost factors can be used in three different ways: 1. The main usage of the effort cost factors is adjusting them when the network is under load. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. @@ -99,7 +101,7 @@ While the effort cost factors could be adjusted manually from the start, that wo For the first iteration the dynamic inclusion effort can be defined to be equal to the byte length of the (rlp encoded) transaction. The maximum dynamic inclusion effort would be equal to the maximum allowed transaction size, which is currently 1,500,000 bytes. -The constant inclusion effort could then be chosen by observing the impact of transactions with a different byte size and a similar computation effort. This data would then be used to predict how much impact a 0 bytes long transaction would make. +The constant inclusion effort could be chosen by observing the impact of transactions with a different byte size and a similar computation effort. This data could be used to predict how much impact a transaction that is 0 bytes long would make. Having a positive constant inclusion effort as well as a dynamic one means that splitting a transaction into smaller pieces does not make sense (considering only the inclusion part of the transaction fees). If this would be desired (from the protocols perspective) that the transactions are as small as possible or of a certain size, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. @@ -122,34 +124,36 @@ Currently the dynamic computation effort is calculated by adding 1 for every loo ### Failing transactions -There are a few reasons why a transaction fails, but for the purpose of transaction fees, transaction failures can be split into four categories, according to who then gets charged for the transaction fees and how much do they get charged. +There are a few reasons why a transaction fails, which can be split into four categories, according to who finally gets charged for the transaction fees and how much do they get charged. 1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the dynamic computation effort set to the computation limit). 2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) 3. Transaction failed during transaction script execution, during fee deduction or during the storage used check. 4. Transaction failed because the computation limit was reached. -As mentioned earlier in scenario 1. the transaction shouldn't have been included in a collection in the first place. Because it was and there is no way to charge the payer, the collection node is charged with the transaction fees where the dynamic computation effort is set to 0. +As discussed below, in scenario 1. the transaction shouldn't have been included in a collection in the first place. However since it was included and there is no way to charge the payer, the collection node account is charged with the transaction fees. The transaction fees are computed with the dynamic computation effort set to 0. -In the second scenario the payer is charged for the transaction fees with the dynamic computation effort is set to 0. +In the second scenario the payer is charged for the transaction fees with the dynamic computation effort set to 0. In the third case, the fees are charged normally with the dynamic computation effort set to the actual usage before the error was produced. -Finally in the last case the payer is charge with the transaction fees where the dynamic computation effort is set to the computation limit. +Finally in the last case the payer is charged with the transaction fees where the dynamic computation effort is set to the computation limit. ### Preventing malicious actors The purpose of fees is to prevent malicious actors sending transactions with the purpose of overloading the network. To achieve that, we have to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. -Tho achieve this the following checks should happen on the collection node, before the transaction is included in a transaction: -- Is the payers signature correct. +Tho achieve this the following checks should happen on the collection node, before the transaction is even included in a collection: +- Is the payers signature valid. - Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. -If this transaction still gets included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the collection nodes account instead of the payers account. +It might still happen that this transaction will get included, either due to a bug or malicious behaviour from the collection node, or due to the collection node not having up to date information of the balance of the payer. In case does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the collection nodes account instead of the payers account. ### Calibrating the parameters -The goal of the calibration illustrated below is to have transaction fees comparable to the current constant fees (`TC`) +When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate that the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. + +To achieve this the 6 different parts of variable transaction fees need to be calibrated accordingly. - Dynamic inclusion effort would be the byte size of a transaction - : Constant inclusion effort would be picked by using linear regression on the impact of different size transactions (with the same computational effort) on the network, and extrapolating for a 0 size transaction. @@ -161,10 +165,22 @@ The goal of the calibration illustrated below is to have transaction fees compar ### Viewing the deducted fees -After a transaction was executed the fee breakdown should be visible in the fee deduction event. All of the components of the fee calculation should be inferable from the state of the network at the time when the transaction was executed and the event data. +The breakdown of the transaction fee calculation for a transaction should be somehow visible after the transaction is executed. + +The proposed solution is to allow anyone reconstruct the transaction fees breakdown from two sources of information. + +Some data can be retrieved from querying the blockchain at that height: +- fee effort cost factors +- inclusion static effort +- inclusion dynamic effort -The effort cost factors and the constant inclusion/dynamic effort can be read from the state, so the remaining data needed on the transaction fee event are the dynamic inclusion effort an the dynamic calculation effort. For redundancy the total amount of fees should also be a field of the fee deduction event. +Some data will be on the fee deduction event emitted when the fees are deducted: +- dynamic inclusion effort +- dynamic computation effort +From this data the entire breakdown and the final price can be deducted. + +The fee deduction event should also contain what the final fee price was. This is so the final fee value is more accessible and also for redundancy. ### Transaction Fee estimation To estimate the minimum transaction fees () needed for a transaction some data needs to be retrieved from the state (access node): @@ -172,26 +188,34 @@ To estimate the minimum transaction fees ( \ No newline at end of file From ced5dc2e5e0fd2cecd7ab4478fd2ce4ec03a11dc Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 18 Oct 2021 18:57:37 +0100 Subject: [PATCH 05/15] diagram fix --- flips/20211007-transaction-fees.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index eca242c99..52f84e62d 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -69,7 +69,8 @@ The final result in equation form looks like this: -![Diagram](20211007-transaction-fees/fees_diagram.svg) +![Alt text](./20211007-transaction-fees/fees_diagram.svg) + - : Inclusion fees; Represent the cost of including the transaction in a collection. - : Inclusion effort cost factor; The current cost of one unit of inclusion effort. From 97ff9d4fd132a3bf9ce07cf4285b311c3789afb3 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 18 Oct 2021 19:06:23 +0100 Subject: [PATCH 06/15] fix image --- flips/20211007-transaction-fees.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 52f84e62d..1d864e867 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -69,8 +69,7 @@ The final result in equation form looks like this: -![Alt text](./20211007-transaction-fees/fees_diagram.svg) - +![Alt text](20211007-transaction-fees/fees_diagram.svg) - : Inclusion fees; Represent the cost of including the transaction in a collection. - : Inclusion effort cost factor; The current cost of one unit of inclusion effort. From 4d02b362dd51fc922b7fc312e97ca1e932086fea Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 18 Oct 2021 19:27:58 +0100 Subject: [PATCH 07/15] update --- flips/20211007-transaction-fees.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 1d864e867..2e6b7d886 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -201,17 +201,25 @@ Users are already aware of the computation limit (a.k.a.: gas limit), the change - setting a high computation limit will mean that the payer needs to have more funds in their account before sending the transaction. - using more computation will result in more expensive transactions. -The second thing to keep in mind is that the transaction fees effort cost factors that will slowly change over time. If the price is currently high the user might want to to wait for a less busy (and cheaper) time to send a transaction. +The second thing for the user to keep in mind is that the transaction fees effort cost factors that will slowly change over time. If the price is currently high the user might want to to wait for a less busy (and cheaper) time to send a transaction. ## Implementation +The proposed implementation path is to get all the parts of variable transaction fees in place (step 1-4). This would mean that everyone can get accustomed to the variable transaction fees: + +- How to read the emitted fee event. +- How to present the transaction fee receipt. +- Where to check if the effort factors have changed. + +After all the parts are in place, we could then focus on fine tuning each part independently. + ### Step 1. Decoupling the concept of effort and effort fee factors. The fee calculation would still not have any dynamic parts. The dynamic effort would be reported as 0 in the transaction fee event. The parts that would change are: - The constant efforts would be set to a value that would approximate the effort of a nil transaction. - The fee effort cost factors would be set so that there is no change to the final fees. -- the fee calculation would be done inside the `FlowFees` smart contract. +- The fee calculation would be done inside the `FlowFees` smart contract. Implementing this step would have no impact on users, but they could start reading the fee breakdown from the emitted fee event. @@ -234,8 +242,6 @@ Introduction of the basic dynamic computation effort. - other computation parameters would be adjusted so that 95% of transactions would be below the current transaction prices. - add checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on collection nodes) -This would mean that all the "happy path" and most of the failure paths are in place, but the computation metering is still very rudimentary. - ### Step 4. Make collection nodes ignore transactions that cannot be paid for. @@ -243,12 +249,11 @@ Make collection nodes ignore transactions that cannot be paid for. - Add payer eligibility checks to the collection nodes. - Add mechanism that the collection node pays for transactions it included in collections where the payer could not have paid. -### Next steps +### Future steps - Improve dynamic computation effort calculation. -- Add mechanism for changing the fee effort cost factors to make fees higher when the traffic is high. -- Parameter fine tuning. - +- Add mechanism for changing the fee effort cost factors to make fees higher when the traffic is high. (a.k.a: surge pricing) +- Fee parameters fine tuning. ## Questions and Discussion Topics From 2fc4364746b6ccd1775779b520df8fe714a11c7c Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Tue, 23 Nov 2021 19:05:17 +0000 Subject: [PATCH 08/15] redesign fee breakdown --- flips/20211007-transaction-fees.md | 171 +++++++++--------- .../fees_diagram.svg | 2 +- 2 files changed, 88 insertions(+), 85 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 2e6b7d886..6e6c2afda 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -16,19 +16,22 @@ Change the calculation of transaction fees on the FLOW network to better secure ## Motivation -Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tends to the optimum throughput, they should also discourage malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. +Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tend to the optimum throughput, they should also discourage malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. +## Note + +In this document I used the term **effort**, **effort cost**, and **effort limit** instead of **gas**, **gas cost**, **gas limit**. As I feel these better describe the sentiment of what is going on. ## Current design -Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here]( https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). +Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). [^2]: Except when an explicit decision is made to change them. -The fees are deducted from the transaction payer automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. +The fees are deducted from the transaction payers default flow wallet automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. -There is an existing concept of a computation limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The computation limit is currently used only to prevent transactions from running for a very long time (or indefinitely). +There is an existing concept of a execution limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The execution limit is currently used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is very rudimentary and often doesn't reflect how long/how many the transaction will take to execute. ## Design requirements @@ -36,8 +39,7 @@ We can separate the requirements into two groups depending on perspective. From the protocols perspective the following requirements can be set: -- Larger (byte size) transactions take more bandwidth and should be costlier than smaller transactions. -- Transactions that are computationally more demanding should be costlier than simpler transactions. +- Transactions that require more resources from the network to be processed, should cost more. - If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. The ultimate goal from the protocols perspective would be that transactions cost exactly the amount of FLOW needed to pay for the cost of the time needed by the nodes to process the transaction. However this would include so many variables that it would not be feasible. Instead the goal is to make a reasonably good approximation, and improve that approximation over time. @@ -55,99 +57,104 @@ From the users perspective the transaction fees implementation should satisfy th The idea for this design is to the transaction fees break down into smaller parts that are easier to define, control, and implement. -The first separation of the transaction fees () is into inclusion fees () and computation fees (). The inclusion fees roughly represent the cost of getting the transaction included in a collection and the cost of the increase in network traffic created by including the transaction. The computation fees account for the approximate operational cost of executing the transaction script. -This separation allows for finer control over the type of transactions that are desired. For example if the volume of transactions is high and is causing congestion on the the collection nodes but the execution nodes have no problem executing all of the transactions, the inclusion fees can be increased without changing the computation fees. +The first separation of the transaction fees () is into inclusion fees () and execution fees (). The inclusion fees roughly represent the cost of including the transaction in a collection and passing it from node to node. Inclusion fees must be designed in a way so they can be calculated without executing the transaction. The execution fees account for the approximate operational cost of executing the transaction script passing the results to the verification node and verifying it. Execution fees cannot be know without actually executing the transaction, but each transaction should specify a stopping limit for the execution fees. + +This separation into a part that can be know before the transaction is executed, and a part that cannot be know before the transaction is executed, allows for better oversight from the user into how much the transaction will actually cost. Each part of the fees can be further split into *effort*() and an *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* would change over time depending on network activity. [^1]: Assuming there weren't any changes to the state that would make the transactions script execute differently. -It is possible to drill down even further by imagining what work would be needed for a theoretical transaction that is completely empty (without any script, arguments, or signatures; a *nil transaction*). A transaction like this would still require some effort from the nodes to include it in a collection and go through the pre-execution and post-execution steps. This means that the *effort* needs to be further split into a constant part() and a dynamic part (). The dynamic parts depend on the transactions properties (how large it is, how long it takes to execute, ...) while the constant parts are the same for all transactions. - The final result in equation form looks like this: - + + + ![Alt text](20211007-transaction-fees/fees_diagram.svg) - : Inclusion fees; Represent the cost of including the transaction in a collection. - - : Inclusion effort cost factor; The current cost of one unit of inclusion effort. - - : Inclusion effort; The time invariant effort required to include this transaction into a collection. - - : Constant inclusion effort; The effort of including a *nil transaction* into a collection - - : Dynamic inclusion effort; The increase in effort due to the particulars transaction properties (due to its size in bytes). -- : Computation fees; Represent the cost of executing the transaction. - - : Computation effort cost factor; The current cost of one unit of computation effort. - - : Computation effort; The time invariant effort required to execute this transactions script. - - : Constant computation effort; The effort of executing a *nil transaction*. - - : Dynamic computation effort; The effort of executing the transactions script. + - : The current cost of one unit of effort. + - : Inclusion effort; The time invariant effort required to handle this transaction (include it into a collection, check signatures are valid, check that the payer can pay, transfer the transaction over the wire,...). The inclusion effort is known before the transaction is executed. + - : Execution effort; The time invariant effort required to execute this transaction (run the script, process the results, send results to verification node, verify the results, ...). The Execution effort cannot be know without executing the transactions script. -### Fee effort cost factors +### Fee effort cost factor -The effort cost factors and should be defined on a smart contract and adjustable via the service account admin resource and they should be accessible for everyone to read. +The effort cost factor should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. -The effort cost factors can be used in three different ways: +The effort cost factor can be used in three different ways: -1. The main usage of the effort cost factors is adjusting them when the network is under load. If the execution or verification nodes are struggling to execute all of the transactions, the computation effort cost factor should be increased. If the network finds it difficult to create collections from all the transactions coming in, but the execution nodes are still fine, the computation effort cost factor should be increased. +1. The main usage of the effort cost factors is adjusting it when the network is under load. -2. The second reason to tweak the effort cost factors is that over time some parts of the code will get optimized. This could cause the same amount of effort to be easier to handle. An example would be that if execution of transactions became 10% faster because of optimizations, the computation effort cost factor could be reduced by 10%. +2. The second reason to tweak the effort cost factors is that over time as hardware improves, the same amount of effort will be easier to handle by the nodes. -3. A potential third use of the effort cost factors would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factors to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. +3. A potential third use of the effort cost factor would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factor to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. -While the effort cost factors could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the effort cost factors automatically would be a service that monitored the networks performance and sent a transaction to update the fee factors when there was a change in the network load. +While the effort cost factor could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the effort cost factor automatically would be a service that monitored the networks performance and sent a transaction to update the fee factors when there was a change in the network load. ### Inclusion effort -For the first iteration the dynamic inclusion effort can be defined to be equal to the byte length of the (rlp encoded) transaction. The maximum dynamic inclusion effort would be equal to the maximum allowed transaction size, which is currently 1,500,000 bytes. +Inclusion effort can represent many different parts of the system. Listed below are some potential factors that may play into the inclusion effort part of the fees listed by node type. Adding factors to this list (and actually implementing them) can happen gradually. One way to notice additional factors that might play into the inclusion effort is to compare transactions that have similar inclusion effort but cause a different load on the system. -The constant inclusion effort could be chosen by observing the impact of transactions with a different byte size and a similar computation effort. This data could be used to predict how much impact a transaction that is 0 bytes long would make. +1. Access node: + 1. Sending a transaction causes load on the access node (might depend on the byte size of the transaction). + 2. The access node needs to potentially do a check (at a constant effort). + 3. ... +2. Collection node: + 1. Load because of tx size (might depend on the byte size of the transaction). + 2. Include tx in a collection (at a constant effort). + 3. ... +3. Execution node: + 1. check signatures are valid (constant x number of signatures) + 2. ... +4. Verification node: -Having a positive constant inclusion effort as well as a dynamic one means that splitting a transaction into smaller pieces does not make sense (considering only the inclusion part of the transaction fees). If this would be desired (from the protocols perspective) that the transactions are as small as possible or of a certain size, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. +For the first iteration the inclusion effort could be defined to be relative to the byte length of the (rlp encoded) transaction (scaled to match average execution effort) plus a well chosen constant part. The constant inclusion effort part could be chosen by observing the impact of transactions with a different byte size and a similar execution effort. This data could be used to predict how much impact a transaction that is 0 bytes long would make. -### Computation effort +Having a positive constant inclusion effort as well as a dynamic one means that splitting a transaction into smaller pieces does not make sense (considering only the inclusion part of the transaction fees). If this would be desired (from the protocols perspective) that the transactions are as small as possible or of a certain size, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. -The computation effort represents the effort needed to execute the transaction script and to do pre and post execution steps (like checking signatures and storage limits and deducting transaction fees). -Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the computation effort before the actual on chain execution of the transaction. +### Execution effort -The lower bound for computation fees (and thus transaction fees) () can still be calculated by assuming the dynamic computation effort is 0. +The execution effort represents the effort needed to execute the transaction script and to handle the execution results. Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the execution effort before the actual on chain execution of the transaction. - +The way execution effort is defined, means that the lower bound for execution effort is 0. Any constant part would just get counted under the inclusion effort. -The upper bound of the computation fees (and thus transaction fees) is indirectly set in the transaction by setting the maximum computation limit () (currently known as the gas limit). +The upper bound of the execution fees (and thus also for transaction fees; assuming the inclusion fees are known) is indirectly set in the transaction by setting the maximum execution effort limit () (in the current implementation this is known as the gas limit). -If the transaction computation limit is reached during the execution of the transaction the execution stops and the state changes made up to that point are dropped. The transaction fees for this transaction are still collected. +If the transaction execution effort limit is reached during the execution of the transaction the execution stops and the state changes made up to that point are dropped. The transaction fees for this transaction are still collected at the maximum execution effort limit. -Currently the dynamic computation effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the dynamic computation effort would be changed so that function calls would have a different computational effort assigned to them according to the time complexity of the method. +Currently the execution effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the execution effort would be changed so that function calls would have a different execution effort cost assigned to them according to the time complexity of the method. ### Failing transactions There are a few reasons why a transaction fails, which can be split into four categories, according to who finally gets charged for the transaction fees and how much do they get charged. -1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the dynamic computation effort set to the computation limit). +1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the execution effort set to the execution effort limit). 2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) 3. Transaction failed during transaction script execution, during fee deduction or during the storage used check. -4. Transaction failed because the computation limit was reached. +4. Transaction failed because the execution effort limit was reached. -As discussed below, in scenario 1. the transaction shouldn't have been included in a collection in the first place. However since it was included and there is no way to charge the payer, the collection node account is charged with the transaction fees. The transaction fees are computed with the dynamic computation effort set to 0. +As discussed below, in scenario 1. the transaction shouldn't have passed the access node in the first place. However since it has and there is no way to charge the payer, the access node account is charged with the transaction fees. The transaction fees are computed with the execution effort set to 0. -In the second scenario the payer is charged for the transaction fees with the dynamic computation effort set to 0. +In the second scenario the payer is charged for the transaction fees with the execution effort set to 0. -In the third case, the fees are charged normally with the dynamic computation effort set to the actual usage before the error was produced. +In the third case, the fees are charged normally with the execution effort set to the actual usage before the error was produced. -Finally in the last case the payer is charged with the transaction fees where the dynamic computation effort is set to the computation limit. +Finally in the last case the payer is charged with the transaction fees where the execution effort is set to the execution effort limit. ### Preventing malicious actors The purpose of fees is to prevent malicious actors sending transactions with the purpose of overloading the network. To achieve that, we have to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. -Tho achieve this the following checks should happen on the collection node, before the transaction is even included in a collection: +Tho achieve this the following checks should happen on the access node, before the transaction is even included in a collection: - Is the payers signature valid. -- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. +- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. -It might still happen that this transaction will get included, either due to a bug or malicious behaviour from the collection node, or due to the collection node not having up to date information of the balance of the payer. In case does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the collection nodes account instead of the payers account. +It might still happen that this transaction will get included in a collection, either due to a bug or malicious behaviour from the access node, or due to the access node not having up to date information of the balance of the payer. In case does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the access nodes account instead of the payers account. ### Calibrating the parameters @@ -155,51 +162,47 @@ When turning on variable transaction fees the goal is to have a smooth transitio To achieve this the 6 different parts of variable transaction fees need to be calibrated accordingly. -- Dynamic inclusion effort would be the byte size of a transaction -- : Constant inclusion effort would be picked by using linear regression on the impact of different size transactions (with the same computational effort) on the network, and extrapolating for a 0 size transaction. -- : Inclusion effort cost factor would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then `TC/2`. +- Inclusion effort would be the byte size of a transaction plus a constant inclusion effort that would be picked by using linear regression on the impact of different size transactions (with the same execution effort) on the network, and extrapolating for a 0 size transaction. +- : Effort cost factor would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then `TC/2`. -- : Dynamic computation effort can be kept as the computation effort that is currently in place. This can be later upgraded so the more expensive methods also cost more computation effort. -- : Constant computation effort would be picked by using a similar method as for the constant inclusion effort. -- : Computation effort cost factor would be picked so that the at the maximum computation limit the computation fees would be `TC/2`. +- : Execution effort can be kept as the execution effort that is currently in place. This can be later upgraded so the more expensive methods also cost more execution effort. Execution effort would scaled so that the at the maximum execution effort limit the execution fees would be `TC/2`. ### Viewing the deducted fees -The breakdown of the transaction fee calculation for a transaction should be somehow visible after the transaction is executed. +The breakdown of the transaction fee calculation for a transaction should be visible after the transaction is executed. The proposed solution is to allow anyone reconstruct the transaction fees breakdown from two sources of information. Some data can be retrieved from querying the blockchain at that height: -- fee effort cost factors -- inclusion static effort -- inclusion dynamic effort +- fee effort cost factor Some data will be on the fee deduction event emitted when the fees are deducted: -- dynamic inclusion effort -- dynamic computation effort +- inclusion effort +- execution effort From this data the entire breakdown and the final price can be deducted. -The fee deduction event should also contain what the final fee price was. This is so the final fee value is more accessible and also for redundancy. +The fee deduction event should also contain what the final fee price was. This is so the final fee value is more accessible and for extra redundancy. ### Transaction Fee estimation To estimate the minimum transaction fees () needed for a transaction some data needs to be retrieved from the state (access node): -- The effort cost factors. -- The constant inclusion effort. -- The constant dynamic effort. +- The effort cost factor. +- The inclusion effort. This could be done by the SDK with a script or a possibly over a dedicated endpoint on the access nodes. -Due to the effort cost factors changing over time, this calculation will inherently be an estimation. To estimate the upper bound of the transaction fees the user has to choose a good value for the computation limit. +Due to the effort cost factor changing over time, this calculation will inherently be an estimation. + +To estimate the upper bound of the transaction fees the user has to choose a good value for the execution effort limit. -If the user is unsure of what computation limit is good enough for their transaction an option is that they just go with the maximum computation limit as the transaction fees will only be deducted per computation usage. +If the user is unsure of what execution effort limit is good enough for their transaction an option is that they just go with the maximum execution effort limit as the transaction fees will only be deducted per execution effort usage. ## User impact -Users are already aware of the computation limit (a.k.a.: gas limit), the change is only that they will now need to pay more attention to it as setting the computation limit has a few implications: -- setting a high computation limit will mean that the payer needs to have more funds in their account before sending the transaction. -- using more computation will result in more expensive transactions. +Users are already aware of the execution effort limit (a.k.a.: gas limit), the change is only that they will now need to pay more attention to it as setting the execution effort limit has a few implications: +- setting a high execution effort limit will mean that the payer needs to have more funds in their account before sending the transaction. +- using more execution effort will result in more expensive transactions. The second thing for the user to keep in mind is that the transaction fees effort cost factors that will slowly change over time. If the price is currently high the user might want to to wait for a less busy (and cheaper) time to send a transaction. @@ -215,11 +218,11 @@ After all the parts are in place, we could then focus on fine tuning each part i ### Step 1. -Decoupling the concept of effort and effort fee factors. The fee calculation would still not have any dynamic parts. The dynamic effort would be reported as 0 in the transaction fee event. The parts that would change are: +Decoupling the concept of effort and the effort fee factor. The fee calculation would still not have any variable parts. The execution effort would be reported as 0 in the transaction fee event. The parts that would change are: -- The constant efforts would be set to a value that would approximate the effort of a nil transaction. -- The fee effort cost factors would be set so that there is no change to the final fees. -- The fee calculation would be done inside the `FlowFees` smart contract. +- The inclusion effort would be set to a constant value that would approximate the effort of a nil transaction (or a randomly picked value). +- The fee effort cost factor would be set so that there is no change to the final fees as they are now. +- The fee calculation (multiplying effort with the factor) would be done inside the `FlowFees` smart contract. Implementing this step would have no impact on users, but they could start reading the fee breakdown from the emitted fee event. @@ -227,35 +230,35 @@ This would also set up the stage nicely for the upcoming steps. ### Step 2. -Introduction of dynamic inclusion effort. +Introduction of variable inclusion effort. -- the dynamic inclusion effort would be set to the byte size of the transaction +- the inclusion effort would be set to the byte size of the transaction plus a constant - other inclusion parameters would be adjusted so that 95% of transactions would be below the current transaction prices. All of the code pieces would already be in place, so the main part of this task would be choosing good parameter values. ### Step 3. -Introduction of the basic dynamic computation effort. +Introduction of the execution effort. -- use the current computation usage calculation as the dynamic computation effort -- other computation parameters would be adjusted so that 95% of transactions would be below the current transaction prices. -- add checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on collection nodes) +- use the current execution effort usage calculation but scale it with a constant factor, so that 95% of transactions would be below the current transaction prices. +- add checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on access nodes) ### Step 4. -Make collection nodes ignore transactions that cannot be paid for. +Make access nodes ignore transactions that cannot be paid for. -- Add payer eligibility checks to the collection nodes. -- Add mechanism that the collection node pays for transactions it included in collections where the payer could not have paid. +- Add payer eligibility checks to the access nodes. +- Add mechanism that the access node pays for transactions it included in collections where the payer could not have paid. ### Future steps -- Improve dynamic computation effort calculation. -- Add mechanism for changing the fee effort cost factors to make fees higher when the traffic is high. (a.k.a: surge pricing) +- Improve execution effort calculation so that transactions are priced more fairly according to the load they cause. +- Add mechanism for changing the fee effort cost factor to make fees higher when the traffic is high. (a.k.a: surge pricing) - Fee parameters fine tuning. +- adding more factors to both inclusion effort and execution effort ## Questions and Discussion Topics -Q: While there is an incentive for collection nodes not to include transactions from payers that potentially might not be able to pay for those transactions, what is the incentive in the opposite direction? If a payer was a bad payer and collection nodes stop including his transactions, what motivates the collection nodes to then start including transactions from that payer, if the payer has added funding to their account? +Q: While there is an incentive for access nodes not to include transactions from payers that potentially might not be able to pay for those transactions, what is the incentive in the opposite direction? If a payer was a bad payer and access nodes stop including his transactions, what motivates the access nodes to then start including transactions from that payer, if the payer has added funding to their account? diff --git a/flips/20211007-transaction-fees/fees_diagram.svg b/flips/20211007-transaction-fees/fees_diagram.svg index c288dfd75..25bb74c45 100644 --- a/flips/20211007-transaction-fees/fees_diagram.svg +++ b/flips/20211007-transaction-fees/fees_diagram.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file From 0f098514b36f1f3878534ec2ab1bb157ef62aa7d Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Thu, 2 Dec 2021 22:15:29 +0000 Subject: [PATCH 09/15] update fees flip --- flips/20211007-transaction-fees.md | 219 ++++++++++++++++++----------- 1 file changed, 139 insertions(+), 80 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 6e6c2afda..34dadc284 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -1,12 +1,8 @@ - - - - # Variable Transaction fees | Status | Proposed | :-------------- |:---------------------------------------------------- | -| **FLIP #** | [NNN](https://github.com/onflow/flow/pull/660)| +| **FLIP 660** | [660](https://github.com/onflow/flow/pull/660)| | **Author(s)** | Janez Podhostnik (janez.podhostnik@dapperlabs.com) | | **Updated** | 2021-10-07 | @@ -21,30 +17,31 @@ Transaction fees should allow the Flow blockchain to self regulate transaction t ## Note In this document I used the term **effort**, **effort cost**, and **effort limit** instead of **gas**, **gas cost**, **gas limit**. As I feel these better describe the sentiment of what is going on. + ## Current design Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). [^2]: Except when an explicit decision is made to change them. -The fees are deducted from the transaction payers default flow wallet automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. +Transaction fees are deducted from the transaction payer's default Flow wallet automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. -The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards the leftover FLOW gets burnt. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. +The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards, the leftover FLOW is kept in the pool for the next payout. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. -There is an existing concept of a execution limit (a.k.a.: gas limit), but it is currently not tied to transaction fees. The execution limit is currently used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is very rudimentary and often doesn't reflect how long/how many the transaction will take to execute. +There is an existing concept of a execution limit (a.k.a.: gas limit or computation limit), but it is currently not tied to transaction fees. The execution limit is used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is very rudimentary and often doesn't reflect how long/how many the transaction will take to execute. ## Design requirements -We can separate the requirements into two groups depending on perspective. +We can separate the requirements for transaction fees into two groups depending on perspective. -From the protocols perspective the following requirements can be set: +From the protocol's perspective the following requirements can be set: -- Transactions that require more resources from the network to be processed, should cost more. -- If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. +- Fees should be proportional to effort: Transactions that require more resources from the network, to be processed, should cost more. +- Surge pricing: If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. -The ultimate goal from the protocols perspective would be that transactions cost exactly the amount of FLOW needed to pay for the cost of the time needed by the nodes to process the transaction. However this would include so many variables that it would not be feasible. Instead the goal is to make a reasonably good approximation, and improve that approximation over time. +The ultimate goal from the protocols perspective would be that transactions cost exactly the amount of FLOW needed to pay for the cost of the time/resources needed by the nodes to process the transaction. However this would include so many variables that it would not be feasible. Instead the goal is to make a reasonably good approximation, and improve that approximation over time. -From the users perspective the transaction fees implementation should satisfy the following criteria: +From the users perspective the implementation of transaction fees should satisfy the following criteria: - Fees should be easy to understand: - It should be made clear why they exist in the form they do. @@ -55,53 +52,74 @@ From the users perspective the transaction fees implementation should satisfy th ## Proposed design -The idea for this design is to the transaction fees break down into smaller parts that are easier to define, control, and implement. +The idea behind this design is to break down the transaction fees into smaller parts that are easier to define, control, and implement. + +The motivation behind the first separation is to separate the fees into a part that can be known before executing the transaction script, inclusion fees, and a part that can only be know by executing the transaction script, execution fees. + +The inclusion part of fees accounts for the resources needed by the transaction because of the transactions properties (that can be known without execution the transaction). Transaction properties are things like the byte size of the transaction and the number of signatures. Some examples of resources that will be needed are the bandwidth to transfer the transaction from node to node, the verification of signatures and the verification of the sequence number. -The first separation of the transaction fees () is into inclusion fees () and execution fees (). The inclusion fees roughly represent the cost of including the transaction in a collection and passing it from node to node. Inclusion fees must be designed in a way so they can be calculated without executing the transaction. The execution fees account for the approximate operational cost of executing the transaction script passing the results to the verification node and verifying it. Execution fees cannot be know without actually executing the transaction, but each transaction should specify a stopping limit for the execution fees. +The execution part of fees account for the approximate operational cost of executing the transaction script, passing the results to the verification node and verifying the results. Execution fees cannot be know without actually executing the transaction. In order for the user to avoid having unexpectedly high execution fees each transaction should specify a stopping limit for the execution fees. -This separation into a part that can be know before the transaction is executed, and a part that cannot be know before the transaction is executed, allows for better oversight from the user into how much the transaction will actually cost. +The inclusion fees and the execution fees can be further split into effort and an the cost of that effort in units of FLOW. The effort required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. The cost of the effort required to execute a transaction can be a factor with the unit of FLOW per effort multiplied by the effort, however in general terms the price of effort would be a function of effort itself. This would allow increasing the price of effort if a lot of effort was used in a transaction. -Each part of the fees can be further split into *effort*() and an *effort cost factor*() of FLOW per unit of effort. The *effort* required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. On the other hand the *effort cost factor* would change over time depending on network activity. +The way to specify the stopping limit for the execution fees is to define a maximum execution effort. [^1]: Assuming there weren't any changes to the state that would make the transactions script execute differently. -The final result in equation form looks like this: +Finally we need to multiply the transaction fees by a (unit-less) surge factor that would be more than 1 in the case the network is experiencing heavy load. - +### Terminology - +- : Transaction fees. +- : Surge factor; unit-less factor that is higher if the network is experiencing heavy load. +- : Inclusion part of the transaction fees. +- : Execution part of the transaction fees. +- : Inclusion effort; The time invariant effort required to handle this transaction (include it into a collection, check signatures are valid, check that the payer can pay, transfer the transaction over the wire,...). The inclusion effort is known before the transaction is executed. +- : Execution effort; The time invariant effort required to execute this transaction (run the script, process the results, send results to verification node, verify the results, ...). The Execution effort cannot be know without executing the +- : Maximum execution effort (gas/computation limit). +- or : Flow cost of effort. In general execution and inclusion effort can be priced differently. -![Alt text](20211007-transaction-fees/fees_diagram.svg) +The transaction fees equation can then be written in the following form. -- : Inclusion fees; Represent the cost of including the transaction in a collection. - - : The current cost of one unit of effort. - - : Inclusion effort; The time invariant effort required to handle this transaction (include it into a collection, check signatures are valid, check that the payer can pay, transfer the transaction over the wire,...). The inclusion effort is known before the transaction is executed. - - : Execution effort; The time invariant effort required to execute this transaction (run the script, process the results, send results to verification node, verify the results, ...). The Execution effort cannot be know without executing the transactions script. + +The maximum fees a transaction can incur can be written with the equation below. -### Fee effort cost factor + -The effort cost factor should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. +### Surge factor -The effort cost factor can be used in three different ways: +The surge factor serves only one purpose, which is to give all transactions on the network a surcharge or potentially a discount when the network load is high or low respectively. It should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for anyone to read. -1. The main usage of the effort cost factors is adjusting it when the network is under load. +While the surge factor could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the surge factor automatically would be a service that would monitor the network load and would send a transaction to update the surge factor when there was a change in the network load. -2. The second reason to tweak the effort cost factors is that over time as hardware improves, the same amount of effort will be easier to handle by the nodes. +When automation will be in place for adjusting the surge factor it will be adjusted frequently. In the span it takes to run a few blocks and detect a surge. +### Effort cost -3. A potential third use of the effort cost factor would be to keep the cost of transaction stable according to the USD value of flow. This would require a periodic job to run, check the USD value of FLOW and update the cost factor to account for any changes. This would allow for the average transaction fees to stay relatively close to a set USD value. +In general there would be two different effort cost functions, one for inclusion fees and one for execution fees . This allows for fine tuning which transactions are the most cost optimal transactions. For example if the execution effort cost function is just a linear function, it would make sense to pack as much execution effort into one transaction as possible, but if the execution effort cost becomes quadratic at a certain effort threshold, it would make sense for users to break transactions into smaller transactions (if possible) that use effort up to that threshold. -While the effort cost factor could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the effort cost factor automatically would be a service that monitored the networks performance and sent a transaction to update the fee factors when there was a change in the network load. + + +In the fist iteration of variable transaction fees the effort cost functions can simply be coefficients: and . The coefficients are referred to as the inclusion effort cost parameter and the execution effort cost parameter, or together as effort cost parameters. + +The effort cost parameters should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. + +The effort cost parameters would need be adjusted infrequently. The reasons for adjusting them would be: + +1. After the calculation of variable transaction fees was changed. + +2. Over time, due to technological factors, the same amount of effort will be easier to handle by the nodes. ### Inclusion effort -Inclusion effort can represent many different parts of the system. Listed below are some potential factors that may play into the inclusion effort part of the fees listed by node type. Adding factors to this list (and actually implementing them) can happen gradually. One way to notice additional factors that might play into the inclusion effort is to compare transactions that have similar inclusion effort but cause a different load on the system. +Inclusion effort can represent many different parts of the system. Listed below are some potential factors that may play into the inclusion effort part of the fees listed by node type. Adding factors to this list (and actually implementing them) can happen gradually. Not all of these factors play equally into the total inclusion effort and it is possible that some factors are completely negligible. 1. Access node: 1. Sending a transaction causes load on the access node (might depend on the byte size of the transaction). - 2. The access node needs to potentially do a check (at a constant effort). - 3. ... + 2. The access node needs to potentially do a check, that the payer can pay (at a constant effort). + 3. The access node should check the transaction code for syntactical correctness (most likely correlated to the transactions size) + 4. ... 2. Collection node: 1. Load because of tx size (might depend on the byte size of the transaction). 2. Include tx in a collection (at a constant effort). @@ -110,35 +128,66 @@ Inclusion effort can represent many different parts of the system. Listed below 1. check signatures are valid (constant x number of signatures) 2. ... 4. Verification node: + 1. ... +A way to notice additional factors that might play into the inclusion effort is to compare transactions that have similar inclusion effort but cause a different load on the system. -For the first iteration the inclusion effort could be defined to be relative to the byte length of the (rlp encoded) transaction (scaled to match average execution effort) plus a well chosen constant part. The constant inclusion effort part could be chosen by observing the impact of transactions with a different byte size and a similar execution effort. This data could be used to predict how much impact a transaction that is 0 bytes long would make. - -Having a positive constant inclusion effort as well as a dynamic one means that splitting a transaction into smaller pieces does not make sense (considering only the inclusion part of the transaction fees). If this would be desired (from the protocols perspective) that the transactions are as small as possible or of a certain size, the dynamic inclusion effort could be defined to grow faster then linear with the byte size of the transaction. - +For the first iteration the inclusion effort could be defined to be a linear function of the byte length of the (rlp encoded) transaction (scaled to match average execution effort). ### Execution effort -The execution effort represents the effort needed to execute the transaction script and to handle the execution results. Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the execution effort before the actual on chain execution of the transaction. +The execution effort represents the effort needed to execute the transaction script (the CPU time) and the effort to handle the execution results (transferring the necessary data to the verification node for the transaction to be verified). Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the execution effort before the actual on chain execution of the transaction. The way execution effort is defined, means that the lower bound for execution effort is 0. Any constant part would just get counted under the inclusion effort. -The upper bound of the execution fees (and thus also for transaction fees; assuming the inclusion fees are known) is indirectly set in the transaction by setting the maximum execution effort limit () (in the current implementation this is known as the gas limit). +The upper bound of the execution fees (and thus also for transaction fees; assuming the inclusion fees are known) is indirectly set in the transaction by setting the maximum execution effort limit () (in the current implementation this is known as the gas limit or computation limit). If the transaction execution effort limit is reached during the execution of the transaction the execution stops and the state changes made up to that point are dropped. The transaction fees for this transaction are still collected at the maximum execution effort limit. Currently the execution effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the execution effort would be changed so that function calls would have a different execution effort cost assigned to them according to the time complexity of the method. +### Keeping the cost of transaction fees stable to a FIAT currency + +If desired the price of transaction fees could be kept relatively stable according to a FIAT currency. To achieved this the FIAT to FLOW ratio () would be added to the fee equation: + + + +This would require a periodic job to run and check the FIAT value of FLOW then update ratio () that would be stored in a smart contract on chain. + +### Preventing malicious actors + +The purpose of fees is to prevent malicious actors sending transactions with the intention of overloading the network. To achieve this, it is necessary to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. + +Tho achieve this the following checks should happen on the access node, before the transaction is even included in a collection: +- Is the payers signature valid. +- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. + +It might still happen that this transaction will get included in a collection, either due to a bug or malicious behaviour from the access node, or due to the access node not having up to date information of the balance of the payer. In case it does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and if the check fails on the execution node, the transaction fees should be deducted from the access nodes account instead of the payers account. + +#### Economic implications + +The access node will be charged, for any transactions where the payer cannot pay. These are by definition failing transactions, so no state change (besides fee deduction) will be made. + +In a scenario where a payer that is close to their minimum balance repeatedly sends transactions the payer would eventually hit the minimum account balance. At this point the transactions would start failing, but fees can still be deducted from the payer. If the payer continues to send transactions, they will eventually hit a balance that is lower then the current transaction fees. At this point if the payer sends more transactions, and the access node includes them, the access node will start paying for the transaction fees. + +Assuming an honest and aware payer, the payer would realize their transaction failed when their balance first dropped below minimum account balance. They would then top off their account before continuing to send more transactions. + +With an unaware payer (for example due to bad automation) or a malicious payer, that wants to drain the access node account or wants to cause excess traffic without paying transaction fees, it is in the access nodes best interest to not include these transactions. + +The access node can check the balance of the payer prior to including their transaction and if the balance is below the minimum account balance, it would not include the transaction. However this prevents the payer from topping up his own balance, after an hones mistake. Some thought is needed here to address this edge case. + +The problem of providing economic incentive to include transactions of not part of this FLIP. This FLIP only provides a way to discourage access nodes to include transactions that cannot be paid for by the payer. + ### Failing transactions There are a few reasons why a transaction fails, which can be split into four categories, according to who finally gets charged for the transaction fees and how much do they get charged. 1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the execution effort set to the execution effort limit). 2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) -3. Transaction failed during transaction script execution, during fee deduction or during the storage used check. +3. Transaction failed during transaction script parsing or during script execution, during fee deduction or during the storage used check. 4. Transaction failed because the execution effort limit was reached. -As discussed below, in scenario 1. the transaction shouldn't have passed the access node in the first place. However since it has and there is no way to charge the payer, the access node account is charged with the transaction fees. The transaction fees are computed with the execution effort set to 0. +In scenario 1. the transaction should have been rejected at the access node level. However since it was not and there is no way to charge the payer, the access node account is charged with the transaction fees. The transaction fees are computed with the execution effort set to 0. In the second scenario the payer is charged for the transaction fees with the execution effort set to 0. @@ -146,35 +195,37 @@ In the third case, the fees are charged normally with the execution effort set t Finally in the last case the payer is charged with the transaction fees where the execution effort is set to the execution effort limit. -### Preventing malicious actors +### Parameter calibration -The purpose of fees is to prevent malicious actors sending transactions with the purpose of overloading the network. To achieve that, we have to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. +When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate that the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. -Tho achieve this the following checks should happen on the access node, before the transaction is even included in a collection: -- Is the payers signature valid. -- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. +To achieve this the 2 different parts of variable transaction fees need to be calibrated accordingly. -It might still happen that this transaction will get included in a collection, either due to a bug or malicious behaviour from the access node, or due to the access node not having up to date information of the balance of the payer. In case does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and in case the check fails on the execution node, the transaction fees should be deducted from the access nodes account instead of the payers account. +#### Inclusion fees -### Calibrating the parameters +Inclusion fees would be defined as a linear function of the byte size () of a transaction (with coefficient and constant term ). -When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate that the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. + -To achieve this the 6 different parts of variable transaction fees need to be calibrated accordingly. +To get the terms this linear function we need a way to compare the impact of the transaction byte size on the network. This can be done by taking a reference transaction that is at the 95th percentile of transaction byte sizes currently seen on mainnet, and does very little computation. This transaction should have inclusion fees of half of the current static fees. We then see how many of those the network can handle per second before it runs into problems. We can define this as the saturation point. The transactions saturation point is inversely proportionate to the fees that should be charged for the transaction. If the network can handle half as many transactions as transactions than should be twice as expensive as . Using this relation and getting the saturation points for a few transactions of different sizes, we can calibrate the linear dependency. -- Inclusion effort would be the byte size of a transaction plus a constant inclusion effort that would be picked by using linear regression on the impact of different size transactions (with the same execution effort) on the network, and extrapolating for a 0 size transaction. -- : Effort cost factor would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then `TC/2`. +The inclusion effort cost parameter is a free variable in the inclusion fees equation, so we can define the inclusion effort cost parameter such that the coefficient is 1/1000. This means the Inclusion fees are some base fee, plus 1 effort cost for each kilobyte. -- : Execution effort can be kept as the execution effort that is currently in place. This can be later upgraded so the more expensive methods also cost more execution effort. Execution effort would scaled so that the at the maximum execution effort limit the execution fees would be `TC/2`. +#### Execution fees + +Execution effort can be kept as the execution effort measuring that is currently in place. This can be later upgraded so the execution effort, more closely matches the resource usage of the transaction. + +Effort cost parameter would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then half of the current static transaction fees. ### Viewing the deducted fees The breakdown of the transaction fee calculation for a transaction should be visible after the transaction is executed. -The proposed solution is to allow anyone reconstruct the transaction fees breakdown from two sources of information. +The proposed solution would allow anyone to reconstruct the transaction fees breakdown from two sources of information. Some data can be retrieved from querying the blockchain at that height: -- fee effort cost factor +- effort cost parameters +- surge factor Some data will be on the fee deduction event emitted when the fees are deducted: - inclusion effort @@ -183,28 +234,31 @@ Some data will be on the fee deduction event emitted when the fees are deducted: From this data the entire breakdown and the final price can be deducted. The fee deduction event should also contain what the final fee price was. This is so the final fee value is more accessible and for extra redundancy. + ### Transaction Fee estimation -To estimate the minimum transaction fees () needed for a transaction some data needs to be retrieved from the state (access node): -- The effort cost factor. -- The inclusion effort. +In order to estimate the minimum transaction fees needed for a transaction, some data needs to be retrieved from the state (access node): -This could be done by the SDK with a script or a possibly over a dedicated endpoint on the access nodes. +- The effort cost parameters. +- The surge factor. + +The calculation of inclusion effort can be done locally. With this if we assume execution effort t is 0 we get the minimum possible transaction fees. The maximum possible transaction fees can be computed with the execution effort set to the chosen execution effort limit. -Due to the effort cost factor changing over time, this calculation will inherently be an estimation. +This could be done by the SDK with a script or a possibly over a dedicated endpoint on the access nodes. -To estimate the upper bound of the transaction fees the user has to choose a good value for the execution effort limit. +Due to the surge factor changing over time, this calculation will inherently be an estimation. If the user is unsure of what execution effort limit is good enough for their transaction an option is that they just go with the maximum execution effort limit as the transaction fees will only be deducted per execution effort usage. ## User impact -Users are already aware of the execution effort limit (a.k.a.: gas limit), the change is only that they will now need to pay more attention to it as setting the execution effort limit has a few implications: +Users are already aware of the execution effort limit (a.k.a.: gas limit or computation limit), the change is only that they will now need to pay more attention to it as setting the execution effort limit has a few implications: + - setting a high execution effort limit will mean that the payer needs to have more funds in their account before sending the transaction. - using more execution effort will result in more expensive transactions. -The second thing for the user to keep in mind is that the transaction fees effort cost factors that will slowly change over time. If the price is currently high the user might want to to wait for a less busy (and cheaper) time to send a transaction. +The second thing for the user to keep in mind is that the surge factor will slowly change over time. If the price is currently high the user might want to to wait for a less busy (and cheaper) time to send a transaction if possible. ## Implementation @@ -212,17 +266,20 @@ The proposed implementation path is to get all the parts of variable transaction - How to read the emitted fee event. - How to present the transaction fee receipt. -- Where to check if the effort factors have changed. +- Where to check if the surge factor has changed. After all the parts are in place, we could then focus on fine tuning each part independently. ### Step 1. -Decoupling the concept of effort and the effort fee factor. The fee calculation would still not have any variable parts. The execution effort would be reported as 0 in the transaction fee event. The parts that would change are: +The first step would be to decouple the concept of effort and the effort cost. + +The fee calculation would still not have any variable parts, and the execution effort would be reported as 0 in the transaction fee event. The parts that would change are: -- The inclusion effort would be set to a constant value that would approximate the effort of a nil transaction (or a randomly picked value). -- The fee effort cost factor would be set so that there is no change to the final fees as they are now. -- The fee calculation (multiplying effort with the factor) would be done inside the `FlowFees` smart contract. +- The inclusion effort would be set to a constant value. +- The inclusion effort cost parameter would be set so that there is no change to the final fees as they are now. +- The surge factor and the effort cost parameters would be defined on the `FlowFees` smart contract. +- The fee calculation would be done inside the `FlowFees` smart contract where the FVM would send the inclusion and execution effort as parameters. Implementing this step would have no impact on users, but they could start reading the fee breakdown from the emitted fee event. @@ -232,33 +289,35 @@ This would also set up the stage nicely for the upcoming steps. Introduction of variable inclusion effort. -- the inclusion effort would be set to the byte size of the transaction plus a constant -- other inclusion parameters would be adjusted so that 95% of transactions would be below the current transaction prices. +- The inclusion effort would be defined as a linear function of transaction byte length and calibrated roughly. All of the code pieces would already be in place, so the main part of this task would be choosing good parameter values. ### Step 3. -Introduction of the execution effort. +Introduction of variable execution effort. -- use the current execution effort usage calculation but scale it with a constant factor, so that 95% of transactions would be below the current transaction prices. -- add checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on access nodes) +- The current implementation of execution effort usage metering would be used and the execution cost parameter would be set so that 95% of transactions would be below the current transaction prices. +- Checks listed in the [Failing transactions](#failing_transactions) chapter (excluding those needed on access nodes) would be added. ### Step 4. Make access nodes ignore transactions that cannot be paid for. - Add payer eligibility checks to the access nodes. -- Add mechanism that the access node pays for transactions it included in collections where the payer could not have paid. +- Add mechanism where the access node pays for transactions it includes in collections where the payer could not have paid. ### Future steps - Improve execution effort calculation so that transactions are priced more fairly according to the load they cause. -- Add mechanism for changing the fee effort cost factor to make fees higher when the traffic is high. (a.k.a: surge pricing) -- Fee parameters fine tuning. -- adding more factors to both inclusion effort and execution effort +- Add mechanism for changing the surge factorto make fees higher when the traffic is high. +- Fine tuning of parameters. +- Adding more factors to both inclusion effort and execution effort to better match the actual resource consumption. ## Questions and Discussion Topics Q: While there is an incentive for access nodes not to include transactions from payers that potentially might not be able to pay for those transactions, what is the incentive in the opposite direction? If a payer was a bad payer and access nodes stop including his transactions, what motivates the access nodes to then start including transactions from that payer, if the payer has added funding to their account? +A: This is an open question, but is also out of scope for this FLIP + +Q: Adjusting the surge factor via a transaction might be to slow. If the network is already struggling to process transactions adding one more to the queue in order to make all that come after it more expensive, does not seem like the best solution. A different mechanic might be needed to control the surge factor. \ No newline at end of file From cdac8b54febb735bd6356a71266f473ab6fbe564 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Fri, 3 Dec 2021 19:56:05 +0000 Subject: [PATCH 10/15] wording changes --- flips/20211007-transaction-fees.md | 98 ++++++++++++++++-------------- 1 file changed, 51 insertions(+), 47 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 34dadc284..40aa844b3 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -24,29 +24,29 @@ Currently transaction fees are the same for all transactions and don't change ov [^2]: Except when an explicit decision is made to change them. -Transaction fees are deducted from the transaction payer's default Flow wallet automatically. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. +Transaction fees are deducted from the transaction payer's default Flow wallet automatically. This happens as the final step of transaction execution. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. -The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards, the leftover FLOW is kept in the pool for the next payout. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. +The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards, the leftover FLOW is kept in a pool for the next payout. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. -There is an existing concept of a execution limit (a.k.a.: gas limit or computation limit), but it is currently not tied to transaction fees. The execution limit is used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is very rudimentary and often doesn't reflect how long/how many the transaction will take to execute. +There is an existing concept of a execution limit (a.k.a.: gas limit or computation limit), but it is currently not tied to transaction fees. The execution limit is used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is also very rudimentary and often doesn't reflect how much time will the transaction take to execute. ## Design requirements -We can separate the requirements for transaction fees into two groups depending on perspective. +The requirements for transaction fees can be separated into two groups depending on perspective. From the protocol's perspective the following requirements can be set: -- Fees should be proportional to effort: Transactions that require more resources from the network, to be processed, should cost more. +- Fees should be proportional to effort: Transactions that require more resources (cpu, memory, bandwidth, storage, ...) from the network, to be processed, should cost more. - Surge pricing: If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. -The ultimate goal from the protocols perspective would be that transactions cost exactly the amount of FLOW needed to pay for the cost of the time/resources needed by the nodes to process the transaction. However this would include so many variables that it would not be feasible. Instead the goal is to make a reasonably good approximation, and improve that approximation over time. +The ultimate goal from the protocols perspective is that transactions cost exactly the amount of FLOW needed to pay for the cost of the resources needed by the nodes to process the transaction. However measuring the exact resource consumption would be difficult and costly in itself, instead the goal is to make a reasonably good approximation and improve that approximation over time. From the users perspective the implementation of transaction fees should satisfy the following criteria: - Fees should be easy to understand: - - It should be made clear why they exist in the form they do. + - It should be made clear why the transaction fees exist in the form they do. - It should be easy to explain how the fees are calculated. -- It should be possible to calculate the transaction fees (within a reasonable margin), before actually sending a transaction. +- It should be possible to calculate the minimum and maximum boundaries of the transaction fees for a transaction (within a reasonable margin), before actually sending a transaction. - The price of sending a specific transaction should not fluctuate over time so quickly that the user cannot respond to the fluctuation. - It should be possible to see the details of the transaction fee calculation after the transaction is executed. @@ -56,12 +56,12 @@ The idea behind this design is to break down the transaction fees into smaller p The motivation behind the first separation is to separate the fees into a part that can be known before executing the transaction script, inclusion fees, and a part that can only be know by executing the transaction script, execution fees. -The inclusion part of fees accounts for the resources needed by the transaction because of the transactions properties (that can be known without execution the transaction). Transaction properties are things like the byte size of the transaction and the number of signatures. Some examples of resources that will be needed are the bandwidth to transfer the transaction from node to node, the verification of signatures and the verification of the sequence number. +The inclusion part of fees accounts for the resources needed by a transaction because of the transaction's properties (that can be known without executing the transaction's script). Transaction properties are things like the byte size of the transaction and the number of signatures. Some examples of resources that will be needed are the bandwidth to transfer the transaction from node to node, the verification of signatures and the verification of the sequence number. -The execution part of fees account for the approximate operational cost of executing the transaction script, passing the results to the verification node and verifying the results. Execution fees cannot be know without actually executing the transaction. In order for the user to avoid having unexpectedly high execution fees each transaction should specify a stopping limit for the execution fees. +The execution part of fees accounts for the approximate operational cost of executing the transaction script, passing the results to the verification node and verifying the results. Execution fees cannot be know without actually executing the transaction script. In order to avoid having unexpectedly high execution fees, each transaction should specify a stopping limit for the execution fees. -The inclusion fees and the execution fees can be further split into effort and an the cost of that effort in units of FLOW. The effort required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. The cost of the effort required to execute a transaction can be a factor with the unit of FLOW per effort multiplied by the effort, however in general terms the price of effort would be a function of effort itself. This would allow increasing the price of effort if a lot of effort was used in a transaction. +The inclusion fees and the execution fees can be further split into effort and an the cost of that effort in units of FLOW. The effort required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. The cost of the effort required to execute a transaction can be a factor with the unit of FLOW per effort multiplied by the effort, however in general terms the price of effort should be a function of effort itself. This would allow increasing the price of effort if a lot of effort was used in a transaction. The way to specify the stopping limit for the execution fees is to define a maximum execution effort. @@ -92,18 +92,20 @@ The maximum fees a transaction can incur can be written with the equation below. The surge factor serves only one purpose, which is to give all transactions on the network a surcharge or potentially a discount when the network load is high or low respectively. It should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for anyone to read. -While the surge factor could be adjusted manually from the start, that would mean responding to network load would be very slow. A potential solution to adjusting the surge factor automatically would be a service that would monitor the network load and would send a transaction to update the surge factor when there was a change in the network load. +The surge factor could be adjusted manually from the start, by manually moniyoring the network and sending a transaction to update the surge factor. This would mean responding to network load would be very slow. -When automation will be in place for adjusting the surge factor it will be adjusted frequently. In the span it takes to run a few blocks and detect a surge. +A fist automated solution to adjusting the surge factor would be a service that would monitor the network load and send a transaction to update the surge factor when there was a change in the network load. + +When automation will be in place for adjusting the surge factor, the surge factor could be adjusted frequently (in the span it takes to run a few blocks and detect a surge), but it should not change to drastically otherwise users cannot respond to the change. ### Effort cost -In general there would be two different effort cost functions, one for inclusion fees and one for execution fees . This allows for fine tuning which transactions are the most cost optimal transactions. For example if the execution effort cost function is just a linear function, it would make sense to pack as much execution effort into one transaction as possible, but if the execution effort cost becomes quadratic at a certain effort threshold, it would make sense for users to break transactions into smaller transactions (if possible) that use effort up to that threshold. +In general there would be two different effort cost functions, one for inclusion fees and one for execution fees . This would allow for fine tuning what kind of transactions are the most cost optimal transactions. For example if the execution effort cost function is just a linear function, it makes sense to pack as much execution effort into one transaction as possible, but if the execution effort cost becomes quadratic at a certain effort threshold, it would make sense for users to break transactions into smaller transactions (if possible) that use effort up to that threshold. -In the fist iteration of variable transaction fees the effort cost functions can simply be coefficients: and . The coefficients are referred to as the inclusion effort cost parameter and the execution effort cost parameter, or together as effort cost parameters. +In the fist iteration of variable transaction fees the effort cost functions can simply be constant coefficients: and . The coefficients are referred to as the inclusion effort cost parameter and the execution effort cost parameter, or together as effort cost parameters. -The effort cost parameters should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. +The effort cost parameters should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. The effort cost parameters would need be adjusted infrequently. The reasons for adjusting them would be: @@ -113,7 +115,7 @@ The effort cost parameters would need be adjusted infrequently. The reasons for ### Inclusion effort -Inclusion effort can represent many different parts of the system. Listed below are some potential factors that may play into the inclusion effort part of the fees listed by node type. Adding factors to this list (and actually implementing them) can happen gradually. Not all of these factors play equally into the total inclusion effort and it is possible that some factors are completely negligible. +Inclusion effort can represent many different parts of the system. Listed below are some potential factors that may play into the inclusion effort part of the fees listed by node type. Adding factors to this list (and actually implementing them) can happen gradually. Not all of these factors play equally into the total inclusion effort and it is possible that some factors are completely negligible. 1. Access node: 1. Sending a transaction causes load on the access node (might depend on the byte size of the transaction). @@ -136,7 +138,11 @@ For the first iteration the inclusion effort could be defined to be a linear fun ### Execution effort -The execution effort represents the effort needed to execute the transaction script (the CPU time) and the effort to handle the execution results (transferring the necessary data to the verification node for the transaction to be verified). Because the execution path of the script depends on the current execution state, it is impossible to definitively predict the execution effort before the actual on chain execution of the transaction. +The execution effort represents the effort needed to execute the transaction script (the CPU time) and the effort to handle the execution results (transferring the necessary data to the verification node for the transaction to be verified). Because execution of the transactions script depends on the current execution state, it is impossible to definitively predict the execution effort before the actual on-chain execution of the transaction. + +Currently the execution effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the execution effort would be changed so that function calls would have a different execution effort cost assigned to them according to the time complexity of the method. + +Thi model should be replaced so that the execution effort better correlates with the time needed to execute the transaction. A possible replacement is to make some functions (that take longer to run) use more effort than others. The way execution effort is defined, means that the lower bound for execution effort is 0. Any constant part would just get counted under the inclusion effort. @@ -144,8 +150,6 @@ The upper bound of the execution fees (and thus also for transaction fees; assum If the transaction execution effort limit is reached during the execution of the transaction the execution stops and the state changes made up to that point are dropped. The transaction fees for this transaction are still collected at the maximum execution effort limit. -Currently the execution effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the execution effort would be changed so that function calls would have a different execution effort cost assigned to them according to the time complexity of the method. - ### Keeping the cost of transaction fees stable to a FIAT currency If desired the price of transaction fees could be kept relatively stable according to a FIAT currency. To achieved this the FIAT to FLOW ratio () would be added to the fee equation: @@ -158,9 +162,9 @@ This would require a periodic job to run and check the FIAT value of FLOW then u The purpose of fees is to prevent malicious actors sending transactions with the intention of overloading the network. To achieve this, it is necessary to check that the payer of a transaction has the funds to pay for that transaction as soon as possible. -Tho achieve this the following checks should happen on the access node, before the transaction is even included in a collection: +To achieve this the following checks should happen on the access node, before the transaction is even included in a collection: - Is the payers signature valid. -- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. +- Can the payer pay for the transaction. The payers default FLOW vault should have at least FLOW. It might still happen that this transaction will get included in a collection, either due to a bug or malicious behaviour from the access node, or due to the access node not having up to date information of the balance of the payer. In case it does still get included in a collection, even though it does not meet the condition, this check should be repeated in the execution node, and if the check fails on the execution node, the transaction fees should be deducted from the access nodes account instead of the payers account. @@ -180,26 +184,27 @@ The problem of providing economic incentive to include transactions of not part ### Failing transactions -There are a few reasons why a transaction fails, which can be split into four categories, according to who finally gets charged for the transaction fees and how much do they get charged. +There are a few reasons why a transaction fails, which can be split into four categories, according to who finally gets charged for the transaction fees and how much they get charged. 1. Transaction failed because the payers signature was incorrect, or the payer does not have enough funds to cover maximum transaction costs (transaction costs with the execution effort set to the execution effort limit). + + This transaction should have been rejected at the access node level. However since it was not and there is no way to charge the payer, the access node account is charged with the transaction fees. The transaction fees are computed with the execution effort set to 0. + 2. Transaction failed before the transaction script execution started (some non-payer signature was incorrect, or the sequence number was incorrect) -3. Transaction failed during transaction script parsing or during script execution, during fee deduction or during the storage used check. -4. Transaction failed because the execution effort limit was reached. -In scenario 1. the transaction should have been rejected at the access node level. However since it was not and there is no way to charge the payer, the access node account is charged with the transaction fees. The transaction fees are computed with the execution effort set to 0. + In this scenario the payer is charged for the transaction fees with the execution effort set to 0. -In the second scenario the payer is charged for the transaction fees with the execution effort set to 0. +3. Transaction failed during transaction script parsing or during script execution, during fee deduction or during the storage used check. -In the third case, the fees are charged normally with the execution effort set to the actual usage before the error was produced. + In this case, the fees are charged normally with the execution effort set to the actual usage before the error was produced. -Finally in the last case the payer is charged with the transaction fees where the execution effort is set to the execution effort limit. +4. Transaction failed because the execution effort limit was reached. -### Parameter calibration + For these transactions the payer is charged with the transaction fees where the execution effort is set to the execution effort limit. -When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate that the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. +### Parameter calibration -To achieve this the 2 different parts of variable transaction fees need to be calibrated accordingly. +When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate this, the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. The two different parts of variable transaction fees need to be calibrated accordingly. #### Inclusion fees @@ -207,15 +212,15 @@ Inclusion fees would be defined as a linear function of the byte size ( -To get the terms this linear function we need a way to compare the impact of the transaction byte size on the network. This can be done by taking a reference transaction that is at the 95th percentile of transaction byte sizes currently seen on mainnet, and does very little computation. This transaction should have inclusion fees of half of the current static fees. We then see how many of those the network can handle per second before it runs into problems. We can define this as the saturation point. The transactions saturation point is inversely proportionate to the fees that should be charged for the transaction. If the network can handle half as many transactions as transactions than should be twice as expensive as . Using this relation and getting the saturation points for a few transactions of different sizes, we can calibrate the linear dependency. +To get the terms of this linear function we need a way to quantify the impact of the transaction byte size on the network. This can be done by taking a reference transaction that is at the 95th percentile of transaction byte sizes currently seen on mainnet, and does very little computation. This transaction should have inclusion fees of half of the current static fees. We then see how many transactions like this the network can handle per second before it runs into problems. We can define this as the saturation point. The transaction's saturation point is inversely proportionate to the fees that should be charged for the transaction. If the network can handle half as many transactions as transactions than should be twice as expensive as . Using this relation and getting the saturation points for a few transactions of different sizes, we can calibrate the linear dependency. -The inclusion effort cost parameter is a free variable in the inclusion fees equation, so we can define the inclusion effort cost parameter such that the coefficient is 1/1000. This means the Inclusion fees are some base fee, plus 1 effort cost for each kilobyte. +The inclusion effort cost parameter is a free variable in the inclusion fees equation, so we can define the inclusion effort cost parameter such that the coefficient is 1/1000. This means the Inclusion fees can be interpreted as some base fee, plus 1 for each kilobyte. #### Execution fees Execution effort can be kept as the execution effort measuring that is currently in place. This can be later upgraded so the execution effort, more closely matches the resource usage of the transaction. -Effort cost parameter would be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then half of the current static transaction fees. +Effort cost parameter should be picked so that for 95% of transactions that have been seen so far the inclusion fees would cost less then half of the current static transaction fees. ### Viewing the deducted fees @@ -223,33 +228,32 @@ The breakdown of the transaction fee calculation for a transaction should be vis The proposed solution would allow anyone to reconstruct the transaction fees breakdown from two sources of information. -Some data can be retrieved from querying the blockchain at that height: +Some data can be retrieved from querying the blockchain at the height at which the transaction was executed: - effort cost parameters - surge factor -Some data will be on the fee deduction event emitted when the fees are deducted: +Some data will be on the emitted fee deduction event, when the fees are deducted: - inclusion effort - execution effort -From this data the entire breakdown and the final price can be deducted. - -The fee deduction event should also contain what the final fee price was. This is so the final fee value is more accessible and for extra redundancy. +From this data the entire breakdown and the final price can be deducted. However, the fee deduction event should also contain what the final fee price was, for extra redundancy and accessibility. ### Transaction Fee estimation -In order to estimate the minimum transaction fees needed for a transaction, some data needs to be retrieved from the state (access node): +In order to estimate the minimum transaction fees needed for a transaction, some data needs to be retrieved from the state (via the access node): - The effort cost parameters. - The surge factor. -The calculation of inclusion effort can be done locally. With this if we assume execution effort t is 0 we get the minimum possible transaction fees. The maximum possible transaction fees can be computed with the execution effort set to the chosen execution effort limit. +The calculation of inclusion effort can be done locally. -This could be done by the SDK with a script or a possibly over a dedicated endpoint on the access nodes. +If we assume execution effort is 0 we get the minimum possible transaction fees. The maximum possible transaction fees can be computed with the execution effort set to the chosen execution effort limit. -Due to the surge factor changing over time, this calculation will inherently be an estimation. +This estimation could be done by the SDK or a possibly over a dedicated endpoint on the access nodes. -If the user is unsure of what execution effort limit is good enough for their transaction an option is that they just go with the maximum execution effort limit as the transaction fees will only be deducted per execution effort usage. +Due to the surge factor changing over time, this calculation will inherently be an estimation. +If the user is unsure of what execution effort limit is good enough for their transaction an option is that they just go with the maximum execution effort limit as the transaction fees will only be deducted per execution effort usage. ## User impact @@ -305,12 +309,12 @@ Introduction of variable execution effort. Make access nodes ignore transactions that cannot be paid for. - Add payer eligibility checks to the access nodes. -- Add mechanism where the access node pays for transactions it includes in collections where the payer could not have paid. +- Add mechanism where the access node pays for transactions it includes where the payer could not have paid. ### Future steps - Improve execution effort calculation so that transactions are priced more fairly according to the load they cause. -- Add mechanism for changing the surge factorto make fees higher when the traffic is high. +- Add mechanism for changing the surge factor to make fees higher when the traffic is high. - Fine tuning of parameters. - Adding more factors to both inclusion effort and execution effort to better match the actual resource consumption. From d20f53eb5c28189501e4dd0188191a3faefb33b2 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 6 Dec 2021 18:11:47 +0000 Subject: [PATCH 11/15] fixed spelling mistakes --- flips/20211007-transaction-fees.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 40aa844b3..fe51d2036 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -26,9 +26,9 @@ Currently transaction fees are the same for all transactions and don't change ov Transaction fees are deducted from the transaction payer's default Flow wallet automatically. This happens as the final step of transaction execution. If the transaction fails the fees are still deducted and no other state change (except the fee deduction) is committed. -The transaction fees are collected on the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch then there were staking rewards, the leftover FLOW is kept in a pool for the next payout. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. +The transaction fees are collected in a FLOW Vault stored in the [FlowFees](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowFees.cdc) smart contract and are used as part of the staking rewards. If there were more fees collected during an epoch than the specified staking reward amount for that epoch, the leftover FLOW is kept in a pool for the next payout. This part would not be changed with this proposal, but it is important to note, as this means collecting more fees leads to less inflation or even deflation. -There is an existing concept of a execution limit (a.k.a.: gas limit or computation limit), but it is currently not tied to transaction fees. The execution limit is used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is also very rudimentary and often doesn't reflect how much time will the transaction take to execute. +There is an existing concept of a execution limit (a.k.a.: gas limit or computation limit), but it is currently not tied to transaction fees. The execution limit is used only to prevent transactions from running for a very long time (or indefinitely). The current measurement of execution is also very rudimentary and often doesn't reflect how much time the transaction will take to execute. ## Design requirements @@ -59,9 +59,11 @@ The motivation behind the first separation is to separate the fees into a part t The inclusion part of fees accounts for the resources needed by a transaction because of the transaction's properties (that can be known without executing the transaction's script). Transaction properties are things like the byte size of the transaction and the number of signatures. Some examples of resources that will be needed are the bandwidth to transfer the transaction from node to node, the verification of signatures and the verification of the sequence number. -The execution part of fees accounts for the approximate operational cost of executing the transaction script, passing the results to the verification node and verifying the results. Execution fees cannot be know without actually executing the transaction script. In order to avoid having unexpectedly high execution fees, each transaction should specify a stopping limit for the execution fees. +The execution part of fees accounts for the approximate operational cost of executing the transaction script, passing the results to the verification node and verifying the results. Execution fees cannot be known without actually executing the transaction script. In order to avoid having unexpectedly high execution fees, each transaction should specify a stopping limit for the execution fees. -The inclusion fees and the execution fees can be further split into effort and an the cost of that effort in units of FLOW. The effort required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. The cost of the effort required to execute a transaction can be a factor with the unit of FLOW per effort multiplied by the effort, however in general terms the price of effort should be a function of effort itself. This would allow increasing the price of effort if a lot of effort was used in a transaction. +The inclusion fees and the execution fees can be further split into effort and an the cost of that effort in units of FLOW. The effort required for a specific transaction should be independent of when the transaction is sent[^1] and should only depend on the properties of the transaction. + +The cost of the effort required to execute a transaction can be a factor with the unit of FLOW per effort multiplied by the effort, however in general terms the price of effort should be a function of effort itself. This would allow increasing the price of effort if a lot of effort was used in a transaction. The way to specify the stopping limit for the execution fees is to define a maximum execution effort. @@ -71,7 +73,7 @@ Finally we need to multiply the transaction fees by a (unit-less) surge factor t ### Terminology -- : Transaction fees. +- : Total transaction fees. - : Surge factor; unit-less factor that is higher if the network is experiencing heavy load. - : Inclusion part of the transaction fees. - : Execution part of the transaction fees. @@ -92,11 +94,11 @@ The maximum fees a transaction can incur can be written with the equation below. The surge factor serves only one purpose, which is to give all transactions on the network a surcharge or potentially a discount when the network load is high or low respectively. It should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for anyone to read. -The surge factor could be adjusted manually from the start, by manually moniyoring the network and sending a transaction to update the surge factor. This would mean responding to network load would be very slow. +The surge factor could be adjusted manually from the start, by manually monitoring the network and sending a transaction to update the surge factor. This would mean responding to network load would be very slow. -A fist automated solution to adjusting the surge factor would be a service that would monitor the network load and send a transaction to update the surge factor when there was a change in the network load. +A first automated solution to adjusting the surge factor would be a service that would monitor the network load and send a transaction to update the surge factor when there was a change in the network load. -When automation will be in place for adjusting the surge factor, the surge factor could be adjusted frequently (in the span it takes to run a few blocks and detect a surge), but it should not change to drastically otherwise users cannot respond to the change. +When automation will be in place for adjusting the surge factor, the surge factor could be adjusted frequently (in the span it takes to run a few blocks and detect a surge), but it should not change too drastically otherwise users cannot respond to the change. ### Effort cost In general there would be two different effort cost functions, one for inclusion fees and one for execution fees . This would allow for fine tuning what kind of transactions are the most cost optimal transactions. For example if the execution effort cost function is just a linear function, it makes sense to pack as much execution effort into one transaction as possible, but if the execution effort cost becomes quadratic at a certain effort threshold, it would make sense for users to break transactions into smaller transactions (if possible) that use effort up to that threshold. @@ -142,7 +144,7 @@ The execution effort represents the effort needed to execute the transaction scr Currently the execution effort is calculated by adding 1 for every loop or function call. This could be sufficient for the first implementation. In the future the calculation of the execution effort would be changed so that function calls would have a different execution effort cost assigned to them according to the time complexity of the method. -Thi model should be replaced so that the execution effort better correlates with the time needed to execute the transaction. A possible replacement is to make some functions (that take longer to run) use more effort than others. +This model should be replaced so that the execution effort better correlates with the time needed to execute the transaction. A possible replacement is to make some functions (that take longer to run) use more effort than others. The way execution effort is defined, means that the lower bound for execution effort is 0. Any constant part would just get counted under the inclusion effort. From 91d34eb2049b11afb87637d7b088a35a7651a5e7 Mon Sep 17 00:00:00 2001 From: Jan Bernatik Date: Thu, 9 Dec 2021 09:22:30 -0800 Subject: [PATCH 12/15] Update 20211007-transaction-fees.md --- flips/20211007-transaction-fees.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index fe51d2036..34c8f9ec1 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -105,7 +105,7 @@ In general there would be two different effort cost functions, one for inclusion -In the fist iteration of variable transaction fees the effort cost functions can simply be constant coefficients: and . The coefficients are referred to as the inclusion effort cost parameter and the execution effort cost parameter, or together as effort cost parameters. +In the first iteration of variable transaction fees the effort cost functions can simply be constant coefficients: and . The coefficients are referred to as the inclusion effort cost parameter and the execution effort cost parameter, or together as effort cost parameters. The effort cost parameters should be defined on a smart contract and adjustable via the service account admin resource and should be accessible for everyone to read. @@ -326,4 +326,4 @@ Q: While there is an incentive for access nodes not to include transactions from A: This is an open question, but is also out of scope for this FLIP -Q: Adjusting the surge factor via a transaction might be to slow. If the network is already struggling to process transactions adding one more to the queue in order to make all that come after it more expensive, does not seem like the best solution. A different mechanic might be needed to control the surge factor. \ No newline at end of file +Q: Adjusting the surge factor via a transaction might be to slow. If the network is already struggling to process transactions adding one more to the queue in order to make all that come after it more expensive, does not seem like the best solution. A different mechanic might be needed to control the surge factor. From 43cc9b2d04fdcc01893faf8ae93bc9ba2007a9d1 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 13 Dec 2021 19:27:02 +0000 Subject: [PATCH 13/15] The diagram isn't needed --- flips/20211007-transaction-fees/fees_diagram.svg | 1 - 1 file changed, 1 deletion(-) delete mode 100644 flips/20211007-transaction-fees/fees_diagram.svg diff --git a/flips/20211007-transaction-fees/fees_diagram.svg b/flips/20211007-transaction-fees/fees_diagram.svg deleted file mode 100644 index 25bb74c45..000000000 --- a/flips/20211007-transaction-fees/fees_diagram.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file From 97eabf644ba2e1230ab65406f5c0d1d92559b0c7 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Mon, 31 Jan 2022 17:05:49 +0000 Subject: [PATCH 14/15] Wording changes --- flips/20211007-transaction-fees.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index 34c8f9ec1..cf6f99a44 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -12,7 +12,7 @@ Change the calculation of transaction fees on the FLOW network to better secure ## Motivation -Transaction fees should allow the Flow blockchain to self regulate transaction throughput in a way where it would always tend to the optimum throughput, they should also discourage malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. +Transaction fees should allow the Flow blockchain to self-regulate transaction throughput in a way where it would always tend to the optimum throughput. Fees should also discourage malicious actors from trying to destabilize the network by sending computationally or network heavy transactions, as the transaction fees on such transactions would be appropriately higher. ## Note @@ -20,7 +20,7 @@ In this document I used the term **effort**, **effort cost**, and **effort limit ## Current design -Currently transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). +Currently, transaction fees are the same for all transactions and don't change over time [^2]. The transaction fee amount is defined in the `FlowServiceAccount` smart contract as the `transactionFee` field (this can be seen [here](https://github.com/onflow/flow-core-contracts/blob/master/contracts/FlowServiceAccount.cdc) or [here](https://flow-view-source.com/mainnet/account/0xe467b9dd11fa00df/contract/FlowServiceAccount)). [^2]: Except when an explicit decision is made to change them. @@ -39,9 +39,9 @@ From the protocol's perspective the following requirements can be set: - Fees should be proportional to effort: Transactions that require more resources (cpu, memory, bandwidth, storage, ...) from the network, to be processed, should cost more. - Surge pricing: If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. -The ultimate goal from the protocols perspective is that transactions cost exactly the amount of FLOW needed to pay for the cost of the resources needed by the nodes to process the transaction. However measuring the exact resource consumption would be difficult and costly in itself, instead the goal is to make a reasonably good approximation and improve that approximation over time. +The ultimate goal from the protocols perspective is that transactions cost exactly the amount of FLOW needed to pay for the cost of the resources needed by the nodes to process the transaction. However measuring the exact resource consumption would be difficult and costly in itself. Instead the goal is to make a reasonably good approximation and improve that approximation over time. -From the users perspective the implementation of transaction fees should satisfy the following criteria: +From the users' perspective the implementation of transaction fees should satisfy the following criteria: - Fees should be easy to understand: - It should be made clear why the transaction fees exist in the form they do. @@ -54,7 +54,7 @@ From the users perspective the implementation of transaction fees should satisfy The idea behind this design is to break down the transaction fees into smaller parts that are easier to define, control, and implement. -The motivation behind the first separation is to separate the fees into a part that can be known before executing the transaction script, inclusion fees, and a part that can only be know by executing the transaction script, execution fees. +The motivation behind the first separation is to partition the fees into a part that can be known before executing the transaction script, inclusion fees, and a part that can only be know by executing the transaction script, execution fees. The inclusion part of fees accounts for the resources needed by a transaction because of the transaction's properties (that can be known without executing the transaction's script). Transaction properties are things like the byte size of the transaction and the number of signatures. Some examples of resources that will be needed are the bandwidth to transfer the transaction from node to node, the verification of signatures and the verification of the sequence number. @@ -180,9 +180,9 @@ Assuming an honest and aware payer, the payer would realize their transaction fa With an unaware payer (for example due to bad automation) or a malicious payer, that wants to drain the access node account or wants to cause excess traffic without paying transaction fees, it is in the access nodes best interest to not include these transactions. -The access node can check the balance of the payer prior to including their transaction and if the balance is below the minimum account balance, it would not include the transaction. However this prevents the payer from topping up his own balance, after an hones mistake. Some thought is needed here to address this edge case. +The access node can check the balance of the payer prior to including their transaction and if the balance is below the minimum account balance, it would not include the transaction. However this prevents the payer from topping up his own balance, after an honest mistake. Some thought is needed here to address this edge case. -The problem of providing economic incentive to include transactions of not part of this FLIP. This FLIP only provides a way to discourage access nodes to include transactions that cannot be paid for by the payer. +The problem of providing economic incentive to include transactions is not part of this FLIP. This FLIP only provides a way to discourage access nodes to include transactions that cannot be paid for by the payer. ### Failing transactions @@ -206,7 +206,7 @@ There are a few reasons why a transaction fails, which can be split into four ca ### Parameter calibration -When turning on variable transaction fees the goal is to have a smooth transition to the new system. To help facilitate this, the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. The two different parts of variable transaction fees need to be calibrated accordingly. +When turning on variable transaction fees, a smooth transition to the new system is desired. To help facilitate this, the cost of transaction fees should be set so that ~95% of transactions should actually pay less transaction fees with variable transaction fees turned on. The two different parts of variable transaction fees need to be calibrated accordingly. #### Inclusion fees From 2f48991b8803b9b92b65719a20df1ce4e9814582 Mon Sep 17 00:00:00 2001 From: Janez Podhostnik Date: Tue, 1 Feb 2022 22:01:08 +0000 Subject: [PATCH 15/15] some wording changes --- flips/20211007-transaction-fees.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/flips/20211007-transaction-fees.md b/flips/20211007-transaction-fees.md index cf6f99a44..b1d0af107 100644 --- a/flips/20211007-transaction-fees.md +++ b/flips/20211007-transaction-fees.md @@ -39,7 +39,9 @@ From the protocol's perspective the following requirements can be set: - Fees should be proportional to effort: Transactions that require more resources (cpu, memory, bandwidth, storage, ...) from the network, to be processed, should cost more. - Surge pricing: If the protocol is experiencing a lot of traffic, all transactions should become more expensive, until the high traffic subsides. -The ultimate goal from the protocols perspective is that transactions cost exactly the amount of FLOW needed to pay for the cost of the resources needed by the nodes to process the transaction. However measuring the exact resource consumption would be difficult and costly in itself. Instead the goal is to make a reasonably good approximation and improve that approximation over time. +These requirements reflect both the need for fairness in pricing the transactions according to their impact on the machines processing them and the added security against resource exhaustion attacks (since they would cost more to perform). + +The ultimate pricing goal from the protocols perspective is that transactions cost exactly the amount of FLOW needed to pay for the cost of the resources needed by the nodes to process the transaction. However measuring the exact resource consumption would be difficult and costly in itself. It also would not account for the fact that each node (of the same type) will use a different amount of time to process the transaction. Instead, the goal is to make a reasonably good approximation, that is independent of the machine its running on, and improve that approximation over time. From the users' perspective the implementation of transaction fees should satisfy the following criteria: @@ -96,7 +98,7 @@ The surge factor serves only one purpose, which is to give all transactions on t The surge factor could be adjusted manually from the start, by manually monitoring the network and sending a transaction to update the surge factor. This would mean responding to network load would be very slow. -A first automated solution to adjusting the surge factor would be a service that would monitor the network load and send a transaction to update the surge factor when there was a change in the network load. +A first automated solution might be for the execution nodes to look at how full the blocks are and use the system chunk transaction to change the surge factor according to that. When automation will be in place for adjusting the surge factor, the surge factor could be adjusted frequently (in the span it takes to run a few blocks and detect a surge), but it should not change too drastically otherwise users cannot respond to the change. ### Effort cost