plot_loading_frequency_seconds issue

[long-pham]

Plot re-loading creates problem for harvester lookup time: WARNING Looking up qualities on ....
If I change the loading frequency to really long (126400), the problem with the harvester's lookup time goes away.
Is increasing num_threads to 120 helpful?
I only see this issue when the number of plots goes above a few thousand. Is this a bug that could be fixed?

I'm using macOS Big Sur on M1 Mac Mini & Chia-blockchain v1.1.7.

[rmica]

Generally speaking, writes to harvesters' drives have to be slow not to create an IO bottleneck. In my linux boxes scripted on my own, I use cgroup/blkio. The more plots you have, the more write speed impacts on the average lookup time. Above 512 plots you have an average 100% probability to pass the filter. In my context a limit to 70-90 MB/s helps, and I rarely see 5sec lookup warnings in the logs.

[long-pham]

All of the drives are full and just sit there for harvesting. So no IO bottleneck for my case.

[rmica]

In a full stable harvester you don't need reloading at all. If you add a new drive just restart the harvester. I'd increase the loading frequency and not the num threads. Reloading creates an IO bottleneck. I don't know if there's a way to disable reloading.

[long-pham]

Yeah, increase reloading time seems to be a temporary fix for me now.

[rmica]

If you find a way to disable it keep me updated. Regards.

[BrandtH22]

@long-pham , since this ticket was opened a number of years ago without further comments it seems this issue has been resolved.
We will be closing this ticket but if we have closed this ticket in error do not hesitate to reach out to us again with any followup questions or comments.

The best place to reach our support team is on Discord (https://discord.gg/chia) or by reopening this ticket.