Add FIP: Premium Percentile Ratio

[ThatGuyLLC]

FIP PR: #1233
Author: https://github.com/wjmelements

Simple Summary
Replacing the utilization-based base fee adjustment with a premium-based target will stabilize gas markets and improve tipset efficiency.

Abstract
Instead of targeting 50% gas limit utilization, which is unreliable for multiple block proposers, congestion is detected by considering the ratio between the effective priority fee, called the premium, and the base fee.
When this ratio exceeds 1/8, the base fee increases; when it falls below this threshold (or if the blocks are sufficiently empty), the base fee decreases.
This approach aligns base fee adjustments with congestion signals, which should improve the predictability of gas markets, thereby reduce proposer duplications.

[wjmelements]

We had a side discussion on slack about whether we can abolish the 100 attoFIL base fee minimum with this change. I think we can, but there were some objections:

@magik6k:

I'd think we want to make it higher for faster response
100aFil is basically zero

@Kubuxu:

The lower you set it, the slower the response from sitting at the floor. IMO no reason to set it to 8.

I don't think we'll be sitting at the floor after this change. We're stuck on the floor because it is much harder to increase the base fee than to decrease it, even under congestion, as discussed here and in the FIP motivation.

I proposed that if we think the base fee adjusts too slowly, we can decrease the BaseFeeMaxChangeDenom, which could be reasonable because our block time is longer than Ethereum's, for whom 8 was configured.

[Kubuxu]

The lower you set it, the slower the response from sitting at the floor.

I didn't mean it as "now we have to walk from 8 to 100," but rather that the walk is slower than it should be due to loss of precision and rounding behaviours. Let's say we are on the floor, and the fee keeps increasing by half the available adjustment. If the floor is 8, the fee will stay at the floor due to rounding.

[wjmelements]

We could round up instead of down.

[wjmelements]

Per our discussions I have made the following changes (f2947f4):

• change adjustment from floor to ceil
• restore 100 base fee minimum

[wjmelements]

A weighted quickselect algorithm such as wselect0 can find the percentile premium in average linear time without sorting. It doesn't seem anyone's been able to get the worst case down to linear for weighted percentiles, but median-of-medians quickselect can get there for unweighted. It seems advanced pivot selection is not used in practice because it is much slower on average.

[wjmelements]

I found a bug in the protocol's current deduping logic for base fee. We only dedupe by Cid, not by sender+nonce, so it is possible for a griefer to drive up the base fee at no cost by sending conflicting messages to the proposers.

This change is a good chance to consider whether base fee calculation should dedupe by sender+nonce instead. I think it should, because such duplicates pay zero for their gas limit. I will update the FIP to clarify this change and make it explicit.

[wjmelements]

git blame says @Kubuxu might know if it was intentional

[Kubuxu]

Yeah, I think you are right that it is a bug.

[wjmelements]

Cool. I verified that all clients do it so we'll just fix it in the next network version.

[wjmelements]

Created Lotus PR

[wjmelements]

Today I ran some simulations with the new premium recommendation algorithm and I think the mempool based approach should not default to nblocksincl=10. The mempool approach might be more suited to lower nblocksincl. I am also deciding how recommendation should behave when the mempool is empty.

[magik6k]

I believe this proposal makes two fairly fatal assuptions:

• GasPremium is a reliable market signal
  • In reality many wallets still hardcode some value (see the amount of transactions at exactly 1nFil premium), and you have off-mempool pools where senders pay participating SPs with mechanisms that don't show up on-chain as GasPremium (maybe not a big thing in Filecoin now)
  • Tying BaseFee to Premium breaks the fundamental premise of BaseFee that it is preductable and reacts smoothly to demand and the 'walking' it does in equilibrium means that with a single knob of 'FeeCap' you get reasonably good statistical guarantees of inclusion times
  • It's especially good in Filecoin where the demant is generally very smooth (fixed rate operations like onboarding, PoSt)
• GasPremium can be used as a feedback mechanism for basefee
  • This breaks the pure self-balancing based on just throughput from EIP1559, "% of used blockspace" is the cleanest signal for blockspace demand, and 1559 is a really clean way to use this signal to control a much more stable transparent market.
  • In my opinion, the ONLY issue we have in filecoin with 1559 is that we don't correctly classify what "used gas" means, through overestimation and through message dedup.

[wjmelements]

In reality many wallets still hardcode some value

This is fine. The premium mechanism uses percentile and not mean, so what matters is some marginal price. In aggregate, it won't matter how individual users price their transactions. If there is congestion, the percentile premium will be higher. If there is not, they will be lower. This theory has been validated by the simulations.

Another way to view this is in terms of surplus. High surplus does signal that the base fee can be higher (and should be higher if we want to burn FIL). Low surplus signals that base fee should be lower and that we are constricting the quantity.

% of used blockspace" is the cleanest signal for blockspace demand

I disagree. Demand is a function of price yielding quantity, so price and quantity are codependent. Quantity only and price only are both fundamentally limited lenses of demand which is why I proposed the percentile premium, which takes both into account. If you consider only quantity, then if demand is highly elastic, the adjusted price might entirely zero out the quantity. This isn't true of the proposed mechanism.

the fundamental premise of BaseFee that it is preductable

The fundamental premise of base fee is not that it is predictable; however, the extent to which it is predictable is not changing. In practice it will become less predictable because right now it is very predictable that it will always be 100, but I don't think this is what you mean. I think what you mean is what you say after:

it does in equilibrium means that with a single knob of 'FeeCap' you get reasonably good statistical guarantees of inclusion times

This is not changing. In simulation, low-urgency transactions confirm within 10 epochs at the same rates as before, even under sustained congestion.

you have off-mempool pools where senders pay participating SPs with mechanisms that don't show up on-chain as GasPremium

I think this only becomes a problem only when such transactions are consuming ~42 gigagas per epoch. This is discussed in the design rationale section, which also suggests several mitigations we could consider if this scenario starts to emerge. We don't want transaction fees to be paid out-of-band anyway because it is bad for FIL. For low usage it should be fine; in fact private mempools would increase utilization.

In my opinion, the ONLY issue we have in filecoin with 1559 is that we don't correctly classify what "used gas" means, through overestimation and through message dedup.

There's no way to fix the utilization target within the TQ framework. I used to think it can be fixed but I was persuaded by @LinZexiao in this post. This is discussed in the motivation section. No utilization target can distinguish if overlap was caused by congestion or by an empty mempool. I also discuss this in slack and in this post where I tagged you.

[jennijuju]

from @magik6k (moving from a private channel )

I didn't see the simulation source code (is it released?) but one more simulation I'd love to see would be to gather a dataset of:

when messages first appear in the mempool and their fee-caps (potentially assign a certain pFIL/nFIL feecap to each message type, default feecap is 10x current price iirc which will produce a bad simulation)
blocks/TQs/tipsets

Then build a replay simulation that shows what happens with real-world data, with the assumption that Filecoin message sending patterns aren't hugely influenced by fee level, and for bulk senders (bulk sector/deal ops).

Basically I want the simulation to show:

SPs don't spend more for gas for priority messages when there is no reason to do that
PoSt messages (high feecap) have good QoS and are included within 5-10mins at least let's say 99.95% of the time
Basefee responds quickly and correctly to congestion caused by bulk senders
Single sender sending sustained e.g. 55% of chain throughput over a long time when there is chain bandwidth doesn't cause infinitely spiraling costs (the current mechanic iirc over-bids some percentile premium by some % so it suffers from this)
Most of the gas spend is captured by basefee burns (currently not significant, but with FOC I expect L1 will quite quickly become gas constrained again)

[wjmelements]

We should clarify in the FIP that the base fee should not update if there are no proposers, which is also the current behavior. This is an edge case I forgot to mention. I will verify that all of the fip-0115 client implementations already behave this way.

[wjmelements]

I have verified that the "null round" behavior has not changed in venus, forest, and lotus. I will now update the FIP.

[wjmelements]

#1251