Check free space before DB Upgrade - v2

[petrkr]

rows, 77.6 GB, in 1699.721 secs
[2025-10-08 01:04:50.182] Progress: 1436000000 rows, 78.0 GB, in 1709.797 secs
[2025-10-08 01:04:51.858] FATAL: Caught exception: Error writing batch: IO error: No space left on device: While appending to file: /data/fulcrum/bitcoin/mainnet/fulc2_db/000818.sst: No space left on device
[2025-10-08 01:04:51.859] Stopping Controller ...
[2025-10-08 01:04:51.859] Closing storage ...
[2025-10-08 01:04:51.868] Flush of all column families returned: IO error: No space left on device: While appending to file: /data/fulcrum/bitcoin/mainnet/fulc2_db/000818.sst: No space left on device
[2025-10-08 01:04:54.627] Shutdown complete
fulcrum@fulcrum:~$

I had 90GB free, but it can not migrate, because out of space.

How much space I need to have? Can I have somehow temporary hard-drive, I do not want extend virtual hard drive just because of migration, because I can not shrink it after migration.. But I can attach temporary hard drive, which I will delete after migration.

Maybe add to documentation required free space for migration

[cculianu]

It does check free space and it estimates about ~87 GB is needed but I guess it was wrong. The code for that was written once and tested on a few machines but I guess it's inaccurate. Sorry about that. I bet you only need like 10GB more or something dumb like that and it will be fine.. so maybe ~100 GB?

For the record on my BTC mainnet install I tested it with ~90 GB free and it worked but the problem is the amount of space rocksdb uses "as it operates" is imprecise and highly variable.. only after compaction does it use minimal space otherwise sometimes it can do "space amplification" and eat up extra space.

Sorry about that.

I would give it maybe another 10GB and try again? (I hope you have a backup!).

[petrkr]

I meanwhile did 500GB and yes, seems it missed just few gigs, as here I can see it crashed at 78GB, but next run was done around 79 GB.

Yes I have 24TB HDD, where I did backup v1 before db migration, so I created new VM HDD on NVMe, copied backup and run again and after migration I copied back to raid ssd ( which is 256GB now), so It finished around 4am :)

[cculianu]

OMFG 1 GB was the difference? Ha ha. Yeah the real problem is rocksdb disk usage is a little bit unpredictable. I probably should have erred on the side of caution and added +10% to the estimate that would have covered 99.99% of cases. Hindsight is 20/20 I suppose. The irony is if I go back and fix it now and re-release Fulcrum, it won't even matter that much since like 80%-90% of servers will likely already have upgraded.

sigh, hindsight is 20/20. i fully acknowledge this was a mental error on my part in not realizing I should overshoot the requirement in the code. Hilarious that in your case was only 1GB. Ha ha ha. 😆

Glad you managed to find a solution by copying it back and forth to grow/shrink the virtual disks, etc.

[petrkr]

Yes I did it like>

Created new NVMe 500GB VM drive
Copied HDD->NVMe backup, which I created earlier
remount this NVMe as data dir
upgraded
copied NVMe --> SSD (where originaly data are)
remounted SSD back to original data dir
run server
Started copy NVMe --> HDD (backup)
goes to sleep

now deleted temporary NVMe 500GB image.

But I am lucky, because like year ago I does not had that NVMe yet, so I will not have space to do that.

btw ration on BTC registered electrum servers, there are still more 1.x than 2.x

83x v1
vs
51x v2

https://1209k.com/bitcoin-eye/ele.php?chain=btc