Is there a performance benchmark?

[CreatCodeBuild]

I use GraphQL and Dataloader in production. I have experienced some performance issues with Dataloader because our code might create 1,000+ promises per second by using Dataloader.

Not necessarily a performance issue with Dataloader itself, but I would like to understand more performance metrics of Dataloader itself in order to optimize my code.

[CreatCodeBuild]

If there isn't one already, are current tests a good approximation of the performance?

[Bajix]

I'm curious about this as well, just as a gauge of comparative performance. In Rust this could be done in as little as 8 microseconds, but I imagine in JS this is thousands of times slower.

[laurisvan]

DataLoaders are currently our main memory hog, indeed creating 1000s of Promises, JS context objects or array literals as we have a few levels of GraphQL resolvers, resolving up a few tens of thousands of objects. At this level, V8 garbage collection starts to become a bit of problem.

I traced some of the problems to the .load() function, creating a new Promise and such for every call. I am currently investingating if I could create an alternative that could somehow reuse the same Promise accross multiple load() calls.

[nicoabie]

DataLoaders are currently our main memory hog, indeed creating 1000s of Promises, JS context objects or array literals as we have a few levels of GraphQL resolvers, resolving up a few tens of thousands of objects. At this level, V8 garbage collection starts to become a bit of problem.

I traced some of the problems to the .load() function, creating a new Promise and such for every call. I am currently investingating if I could create an alternative that could somehow reuse the same Promise accross multiple load() calls.

Any findings?

[laurisvan]

@nicoabie Unfortunately not yet - we've been too busy with other things. We kicked the can forward by compiling node.js with pointer compression. That reduced our memory load enough that this is not currently our biggest challenge.

[laurisvan]

This morning I had a moment to think about this, yet still nothing concrete yet :(

I was wondering - perhaps async generators would work here. The problems raised at graphql-js repo seem a bit similar: graphql/graphql-js#2262

[nicoabie]

As far as I could understand, their problem is with blocking and here is with a lot of memory being used creating new promises. I don't see how the async generators would prevent creating the large amount of promises. 😕

[laurisvan]

I must say I'm not expert on async generators either - I don't know what the underlying mechanism is, e.g. does it create context objects in the same way as Promises do. At least in my case the biggest problem is not the memory used, but the fact that every object allocated from heap needs to be garbage collected.

[owenallenaz]

Running into a similar problem, there is a crucial element to dataloader that I think is overlooked in some of the above comments. Multiple calls to load() which result in the same cacheKey do not result in the creation of an additional promise.

So lets imagine the returned data is structured something like:

const users: [
  {
    id: 1,
    name: "foo",
    account: {
      id: 1
    }
  },
  {
    id: 2,
    name: "bar",
    account: {
      id: 1
    }
  },
  ... hundreds more records ...
]

And our simplistic resolver had something like

const resolvers = {
  users: {
    account: function(parent, args, context, info) {
      return context.relatedLoader.load(parent.account_id);
    }
  }
}

In that case, if 5 users return the same account_id then it doesn't create 5 promises, it creates 1 promise. That process can easily sideways if you declared account: async function, or if each user has a different unique account_id. In our situation we were returning 100k docs which all had the same parent account, and that resulted in 100k promises, until we properly refactored the code to ensure we're able to properly leverage dataloader same-request Promise caching behavior.

[laurisvan]

JFYI We found the promise creation might be difficult to solve with DataLoaders (or more specifically, with resolvers returning a promise). Hence we are investigating Grafast (https://grafast.org/) that should be able to reduce this "1000 promises for 1000 child resolver calls" into "1 promise and 1000 normal function calls for 1000 resolver calls".

We need to have the related graphql-modules PR merged (graphql-hive/graphql-modules#2618) until we can make progress on this one, but most of you probably won't use graphql-modules so you can try grafast and see if it solves your problem.