Admin blocks (and times out) on long running unawaited tasks

[ssyberg]

Link to reproduction

No response

Describe the Bug

Apologies if this is expected behavior, my grasp of sync/async node is not stellar but this does smell a bit funny to me. I'm finding that if an afterChange hook does a lot of long running nonblocking work (in my case it's a few thousand record updates), it still blocks admin completing and generally leads to a timeout.

To Reproduce

Create the below collection and save a record, you'll see that the record itself is immediately created but then admin is stuck on the "loading" screen indefinitely and eventually times out.

const Sandbox = {
    slug: 'sandbox',
    fields: [
        {
            name: 'name',
            type: 'text',
            hooks: {
                afterChange: [
                    async (args) => {
                        console.log(`Starting after change hook`);

                        // Do 2000 payload writes but don't await them
                        for (let i = 0; i < 2000; i++) {
                            args.req.payload.create({
                                collection: 'sandbox',
                                data: { name: `name ${Math.random()}` },
                            });
                        }

                        console.log(`Ending after change hook`);
                    },
                ],
            },
        },
    ],
};

export default Sandbox;

Payload Version

2.7.0

Adapters and Plugins

db-mongodb

[jmikrut]

Hey @ssyberg, what is happening here is that you are not awaiting your create operation and effectively firing off 2000 create operations in parallel.

This will probably tie up your server / local Node process itself. There's not much Payload can do to solve this, so I will convert this to a discussion, but if you do need to do things like this, you have some Node-based options like split out into different threads / etc. to perform that work.

I'm also not hugely knowledgable on using multiple Node threads either to be honest but would be interested to hear what you find here!

[ssyberg]

Thanks @jmikrut I figured it was something to that effect, not awaiting the create is intentional here because I don't want these operations to block. I was specifically trying to harness this concept:

If your Hook simply performs a side-effect, such as updating a CRM, it might be okay to define it synchronously, so the Payload operation does not have to wait for your hook to complete.

(https://payloadcms.com/docs/hooks/overview#async-vs-synchronous)

But it appears that Payload is still waiting for the hook to complete?

[ssyberg]

Just want to add, if this is expected behavior, perhaps this is a documentation bug?