`D`-Gate destruction misses some blocks in some cases

[Thorinwasher]

Reproducible steps

• Generate loads of D gates using a random G gate
• Wait until destroyed
• In some cases the blocks do not get removed

[Thorinwasher]

I don't know why this is happening, and why it's only relevant for some blocks in the structure. Found this issue a while ago, but I can't seem to solve it at the moment