How to avoid sector transfer among two workers?

[potato1992]

I have two docker worker running on the same machine.
They very frequently assign their sector to each other. And what drives me crazy is that, the GET method takes too long time:

ID        Sector  Worker    Hostname  Task  State        Time
2ae97f8e  49      cd6cbc9f  docker1   GET   running      2h24m49.2s
f2abc78a  48      2c3c92b4  docker0   GET   running      2h17m39.5s
00000000  52      2c3c92b4  docker0   PC2   assigned(0)  4h3m15.3s
00000000  50      cd6cbc9f  docker1   PC2   assigned(0)  3h59m58.2s
00000000  18      cd6cbc9f  docker1   PC2   assigned(1)  2h37m16.5s
00000000  39      2c3c92b4  docker0   C2    assigned(1)  2h24m45.4s

How to disable the sector transfer, and let them to do all the job themselves? I have tested iperf3 among this two docker (30Gbit speed can be get). I do not know why it takes more than 2 hours to transfer 500GB cache data.

I know it's useless to complain. But the developers really need to learn how to write efficient code for data transfer (use C/C++ please, Go is a garbage language). And the so called task/resources window is really stupidly designed.

Hardware:

AMD epyc 7502x2
SSD: Intel P4510 8T x3 (raid0)
memory: 1T 2933Mhz
GPU: RTX3080x2
Ethernet: Intel 10Gbit

[de-authority]

You can't stop the sectors transfer for now when you have more than two workers. The reason why it takes as long as 2Hrs to transfer 500Gb maybe is your disk write/read limit or you have your task parallel running.　Every language has its advantages. and i think mybe you can file an issue about how to schedule/asign task to different worker instead of complaining here.
lotus team are willing to hear any feedback.

[potato1992]

You can't stop the sectors transfer for now when you have more than two workers. The reason why it takes as long as 2Hrs to transfer 500Gb maybe is your disk write/read limit or you have your task parallel running.　Every language has its advantages. and i think mybe you can file an issue about how to schedule/asign task to different worker instead of complaining here.
lotus team are willing to hear any feedback.

I am sure there is not performance issue in my machine. I have tested CPU/GPU/memory/SSD test in my docker machine and they are all good.

Anyway, I have solved it, I have changed the code and added a switch to use "mv" to directly move the data from pre-configured path to current cache folder, instead of takes 2 hours to use the lotus to transfer it.

I have ugly code changes to achieve this, So I hope this will be considered to be implemented from the official side. Samba performs much much much better than the lotus built in data transfer, especially in docker environments.

[de-authority]

Great to hear that. i have filed an issue about this long ago: #6838
But to be honest. this change may can't meet your need. you can file an issue too if you have good ideas

[Fatman13]

@potato1992 venus implementation offers --bindp1p2 option to allow worker to get P1 from same machine.
https://venus.filecoin.io/guide/Using-venus-Shared-Modules.html#finding-optimal-configurations

Logic behind is simple. When selecting workers for P2, check if the sealed file from P1 already exists on the machine, if not reject the task, otherwise queue for the task. If resources on your machine allows, your P2 will proceed on the same machine.
filecoin-project/venus-sealer@f03d89e

[LexLuthr]

@potato1992 I think with new storage group coming into picture, this particular problem has been resolved. Please check the below link for details about the feature.

#7453
filecoin-project/lotus-docs#32