Configurable sector deadline/partition assignment

[ZenGround0]

The frequently proposed idea (i.e. see: #32) of configurable sector deadline assigment came up again during a discussion last week with @rjan90, @Angelo-gh3990, @s0nik42, @Stebalien and others. Reminder this would help SPs:

• schedule posts at convenient working hours
• enforce minimal posts on chain given immutable deadline rules
• leave open maintenance windows

New to me from this discussion:

• SPs could schedule sector partitions based on similarities in sector expiration epoch. It was observed that holes in sector bitfields from partial partition expiration can get expensive. Operator assignment to a partition based on similar expiration time would help with keeping down bitfield size (one run to encode across sector life) and expiration queue processing (one expiration set across sector life). Even with partial faults this only adds one extra expiration queue.

Note that SPs are effectively configuring deadline assignment today by generating multiple miner actors and choosing the ones with deadline offsets that work best for them. Doing this explicitly would make the protocol more usable and force us to address over-scheduling concerns directly.

In particular there is risk of slow blocks in popular deadline windows with lots of cron work. After implementing safe cron this will be solved as SPs will have a cost signal for cron congestion.

There are two parts: allowing new sectors to be scheduled and moving existing sectors between deadlines / partitions.

Allowing new sectors to be scheduled should be as easy as providing an assignment config for prove commit. If configs are universal and compact (i.e. always assign my sectors to deadline 5 or assign my sectors anywhere but deadline 10 and 11) then we could put them in the miner info. The easiest thing is to add this to the message params.

Moving sectors / partitions across deadlines will take more work but is helpful both for all existing sectors / miner actors and for providing flexibility to future operations. Moving individual sectors will require sector info access which will be expensive though moving full partitions will be less expensive since we should just be able to use rolled up power values as in compaction. We'll need to address the security concern of moving your sectors out of a partition to avoid failing a post.

[geoff-vball]

@nicola might have opinions on this. It could have implications for proving finality for cross-chain interop.

[steven004]

SPs could schedule sector partitions based on similarities in sector expiration epoch.

Is this a real requirement from SPs? Usually, an SP will configure similar expiration epoch for a bunch of sectors with consecutive IDs, so basically they will be most likely be in one partition if there is still a room.

From implementation point of view, it looks moving partitions across deadlines is an easy way to achieve most of targets though it does not provide ideal flexibility.

[rjan90]

Is this a real requirement from SPs? Usually, an SP will configure similar expiration epoch for a bunch of sectors with consecutive IDs, so basically they will be most likely be in one partition if there is still a room.

It´s not a requirement, but on a longer time horizion we might see more SPs take on storage deals with very different durations that gets assinged into a single deadline. So when the partitions are compacted over time, that might compound into less ideal bitfields. So it would be good to explore other potentially better ideas for how we can schedule sector partitions in conjunction to the other improvements listed.

[Fatman13]

This could also be helpful for a SaaS using offline data transfer methods (ie, shipping hard drives) to have some relaxed time requirement for sectors' first windowPosts.

[FroghubMan]

oh... an interesting idea！
Storage providers across IDCs: Put sectors of the same IDC in the same partition.

[dd45e640b42e6da7da96faee3996ef7c]

does this include filling partition x +1 of deadline y before partition x of deadline y+1...47 is filled?

i doubt it is a good idea to allow free movement completely. we might see a constant pattern of moving sectors around to "flee" the congested times as the baseFee will not take this lightly

[rjan90]

This is just a discussion post to explore the idea further and see if there is some interest in such a proposal. So nothing has been written/drafted.

does this include filling partition x +1 of deadline y before partition x of deadline y+1...47 is filled?

We did not talk about this approach. There might be some concerns that such an approach lowers the security as generation attacks might be more feasible when all sectors are in a single deadline.

i doubt it is a good idea to allow free movement completely. we might see a constant pattern of moving sectors around to "flee" the congested times as the baseFee will not take this lightly

From my recollection of the conversation there would definetly need to be some constraints for when, at what times and how many times one could move sectors around.