Update:

The initial goal of this was to greatly enhance performance. However, it has led us back to facing concurrency issues.

When running with more than a handful of workers, the process will not complete and will become locked indefinitely. I'm uncertain how to resolve this, so I'm open to any assistance!

If not, I may have to put this rework idea on hold for now and revisit it later in the development process.