500TB Setup optimization? Anyone? :(

[NipBoss]

Hey guys, it has been 7 days now since I hooked up my setup with 500TB but I am still unable to plot.
I have tried everything, 101.4GiB plot, custom RAM ram setting, different RAM sticks, different CPU, different risers
different drives (4x 5TB ones over usb instead of sata3/6) still nothing. it just crashes, the chia app.
First was the issue of running out of space but my temp folder now is a $400 samsung ssd, the os is on a 120gb kingston separate, and all the drives are 10tb's and I had returned the 4 x 5tb test units already...

it just died, the program crashes constantly and sometimes even the os reboots and its a windows 10 pro fresh install done like 12 times now, about twice a day, to reformat the drives so they are clean.

I am on a 1000/30 up/down dedicated link over wire just for this, brand new asus gaming motherboard, ryzen cpu x4 cores, 80gb ddr3 ram, I am going to cry myself to sleep again today.

GUYS, I will get us a powershell and a python developer if needed, just tell me how I can help so other people like me don't suffer in the future, please.

Here's the most recent log, this was set up like this - > Win 10 P (120gb ssd not pcie) ->Temp drive (2tb ssd - not pcie) -> 10TB seagate baracudda also not thru pcie, and it crashed twice again today, once at around 6% and second time just an hour ago with the same error.

Log file is attached, please help, this is runing my health because of the stress level :(
log for github.txt

[cccat6]

Never seen a crash in the system unless your hardware has some problems. You tried different CPU, RAM, etc. But did you try different power?
The Kingston system disk is also suspicious.
Take away some RAM, leave 8G in your system and retry. Run a single task, at least make it work once.

[NipBoss]

Hey! I really appriciate the help, I just put a new 8 core CPU, removed 2x 32GB ram sticks and put 2x 8 and left the prior 64GB.
All equipment I am putting is brand new for this project, it had crashed again with the new CPU and RAM, I will re-install windows again and use a samsung 240gb ssd fresh out of the box in the morning. Here's some more screenshots, I was running the simplest plot to test just now.

[cccat6]

What is your file system for your temp drive? Did you try to reformat it? Try both NTFS and EXFAT to see if it help.
I got a similar error before but is in Linux with EXT4 and I tried different versions of Linux. Then I use the exact same machine and switch it to Windows, it works well. So I just guessing the temp drive format has some problem.

[NipBoss]

It seems to be always happening and crashing at bucket #127 which is one of the parameters.
Bucket 127 uniform sort. Ram: 3.251GiB, u_sort min: 0.563GiB, qs min: 0.281GiB.
and then there's some sort of a mismatch.
Total matches: 4294762939
Caught plotting error: Matches do not match with number of write entries 4294762939 4294762944

[lvcivs]

That error message looks like the one in issue #2493 . a chia dev posted there that it points to a hardware problem. Might want to do a memtest86 and possibly some other tests to check whether all the hardware is setup correctly.

[NipBoss]

Just created another plot, gave it 16GB RAM, and increated the bucket size x20.
I'll let it run overnight and see if it helps.

[NipBoss]

Just ran into the same error at bucket #128, seems like you cannot get past that one.

[nomadengineer]

Your memory allocation for a single plot is too high which might be causing a problem!
Reduce it to 4400 may be! And start 8 parallel plots with a delay of 15 minutes each.
This blog should help.
https://thechiafarmer.com/2021/04/21/optimizing-plotters-in-windows/

[nomadengineer]

No need to use more than 128 buckets, its set to default 128 anyways!

[SackofFluor]

yeah.. go down with the buckets and start with a little more sensible ram settings.
Then, if it it works continue to tweak

[NipBoss]

I did the tweaks and got it to go all the way to 100% plotting and now it still is giving out an error at the very end...

It says Could not copy "F:\plot-k32-2021-05-09-18-16-6b313f55da52a722be9e8155c60f1b08acf37f7c6f769019f60b05c5be99b71b.plot.2.tmp" to "D:\plot-k32-2021-05-09-18-16-6b313f55da52a722be9e8155c60f1b08acf37f7c6f769019f60b05c5be99b71b.plot.2.tmp". Error Access is denied.. Retrying in five minutes.

It has been retrying for the past 4 hours. Thank you guys for the help, I really appriciate it.

[NipBoss]

I do, please check this post.
#3438

[NipBoss]

I did not manually copy anything there, I format the drives before every re-run.

[NipBoss]

and I am under the admin account, and I gave full permissions to all users.

[SackofFluor]

How is it formated? exFAT, NTFS?

Am i correct that it is a network drive?
If so, try to copy some other file there, just to be sure that you can actually access the drive for read/write purposes.

[NipBoss]

NTFS

[NipBoss]

I am considering an intel setup now, thats the only thing I have left to swap, argh.

[SackofFluor]

I bought an ryzen 3900x and samsung SSDs, sorry but THIS is not the problem.....

[NipBoss]

[NipBoss]

Sacko, I have 14 years exp. in IT, I have tried all things I could and everything suggested, I am going to wipe windows again and if it still doesnt work I am just going to try linux, I have had enough, thanks for your help everyone!

[NipBoss]

Anyways, any other idea? Just cleaned up everything and started a new plot, this is going to be the final one before linux and then if that doesn't work out I'll have to go with new motherboard and intel CPU.

[SackofFluor]

Did you try to start the software as "administrator"? Its not important for me, but I have some experience in network and datamangement myself and sometimes there are rediculous idiotic problems.
You did reinstall windows already right?
How big is D:\ `?

[NipBoss]

I am logged in as Administrator and yes I run Chia as admin everytime.
D:\ is 10TB seagate (9.09TB actual TB space), yes, fresh install is going again at the momment.
All 47 x 10TB are the same manufacturing batch and I plug in a new drive and format into NTFS every re-run.
I have background in networking and cybersecurity :) https://oleg.ua if you wanna get in touch ;)

[YahorL]

Try to use CLI instead of GUI. I don't have any other ideas

[NipBoss]

if I am using CLI for it I am going to do it with Cloud Linux CentOS 8... which I already tested and it works.

[jackykwandesign]

if you can they why not just linux, nothing you can do with window

[jackykwandesign]

i used to suffer from simliar problem that filesystem will unmount itself at some point, turns out it is Asmedia SATA controller driver compatible issue on my Motherboard, so stick with what you work and never drill in a hole

[leoliyuanduo]

I don't think CLI will help you. does the log from 'user.chia\mainnet\log\debeg.log' give you any hint? I guess you would go to Linux. Linux system plotting little faster.

[BrandtH22]

If you are still experiencing the issues it is recommended to upgrade to the latest version of chia. Please download it from here: https://www.chia.net/downloads/

Since this issue has been open for a number of years without additional comments, we will be closing this ticket but if we have closed this ticket in error do not hesitate to reach out to us again with any followup questions, comments, or if the issue persists after an update.

Please note that upgrading major versions of chia generally requires the config.yaml to be recreated using these steps:
1. Stop chia
2. Rename the config or move it to a different directory
3. In CLI run chia init
4. Copy any custom settings from the old config to the new (plot drive directories, pool info, trusted node info, etc) - be careful here as yaml is very picky about formatting including those leading spaces
5. Start Chia

The best place to reach our support team is on Discord (https://discord.gg/chia) or by reopening this ticket.