actions.createNode doesn't work from inside createPages

[xddz9]

Am I not allowed to create a node from inside createPages in gatsby-node.js? I can call actions.createNode from createPages, and when running Gatsby the total number of nodes reflects the nodes created from inside createPages, however, those don't seem to be added to GraphQL at all.

I've created a Code Sandbox which creates nodes both from createPages and from onCreateNode. As you can see, only nodes created within onCreateNode are added to GraphQL, while the ones created from createPages don't show up in GraphQL at all, even though Gatsby does seem to do something with them since adding new nodes there changes the output of "Total nodes" in the console when running "gatsby develop".

Is this a bug or am I misunderstanding something?

If it matters, I'm fetching all my data when Gatsby runs createPages, and from there I create all my custom pages and nodes, both of which depend on my data. If I have to split page and node creation into two functions then I'll need to fetch my data twice, which doesn't make sense.
I understand I don't have to create my pages/nodes manually, and that there are a whole bunch of plugins to help out with all of that, but in this particular case I'd like to do it manually. If createPages exposes actions.createNode then that should actually create a node, no?

[Richacinas]

Same problem here. Actually, executing createNode inside the body of createPages causes my process to restart again and again.

[tuube]

The documentation says the following about createPages (link):

Create pages dynamically. This extension point is called only after the initial sourcing and transformation of nodes plus creation of the GraphQL schema are complete so you can query your data in order to create pages.

So it would make sense that the GraphQl schema doesn't update anymore, because it's already created. I'm also in need of creating new nodes in createPages, but I don't know about any working ways to do that.

[Richacinas]

I ended up calling twice to createPages, each of those times with a different value for a certain process.env variable:

exports.createPages = async ({ actions, graphql }) => {
  if (!process.env.CREATING_PRODUCTS) {
    await createCategories({ actions, graphql });
  }
  if (process.env.CREATING_PRODUCTS) {
    await createProducts({ actions, graphql });
  }
}

exports.sourceNodes = async ({ actions, createNodeId, createContentDigest, cache }) => {
  if (process.env.CREATING_PRODUCTS) {
    await createProductNodes({ actions, createNodeId, createContentDigest, cache });
  }
}

This way, the second time I run my gatsby process, I pass CREATING_PRODUCTS=true on my package.json script, and then I can create my nodes that I previously stored on a separate database (or gatsby cache if it works for you).

[vladar]

Can you elaborate a bit on what you are trying to achieve? Why do you need to call createNode in createPages in the first place?

[xddz9]

Can you elaborate a bit on what you are trying to achieve? Why do you need to call createNode in createPages in the first place?

My use case is pretty much the same as @tuube and @Richacinas. I expected to be able to fetch some data and then call createNodes/createPages on any order I see fit, but that didn't work out. I ended up fetching my data during sourceNodes, calling createNodes from there, setting my data on Gatsby's cache and then using that cached data in createPages. It works but it feels incredibly hacky 😕

[vladar]

I see. I think you can restructure your apps a bit to behave as Gatsby would normally expect.

The general idea is that fetching of any data should happen inside sourceNodes call. Ideally, createPages shouldn't be calling the network at all. It should query local data that you've already fetched and stored in sourceNodes (use graphql call at this stage to fetch the data you need to create actual pages).

Or in other words, you can re-create your remote pages as nodes first in sourceNodes and then create actual Gatsby pages based on those nodes.

It is important because Gatsby has several assumptions about the various steps of the build. For example createPages must be very fast because we run this step in incremental builds (in Gatsby Cloud). And if they call network - it slows down incremental builds significantly.

But also keeping these semantics is important for future build optimizations we are currently working on. We will likely introduce an optional strict mode soon with additional guardrails to warn you in cases like this.

[Richacinas]

What you say makes a lot of sense, however, in my case, I can't make it work like that because I need to use graphql in order to know what products I have to create.

That is why my solution, even though feels totally hacky, is the only way I have to create my categories and then my products.

[vladar]

@Richacinas can you elaborate a bit or maybe show an example code? I don't see how graphql can be a blocker in this scenario. Can you create categories as nodes - and then create products as Gatsby nodes and only then - create a page for each product?

[Richacinas]

Because in order to create my product nodes, I first need to query my categories, and I can't use graphql in sourceNodes. That is why I'm first creating my category pages, storing all my products on a MySQL table in the process, and then calling again my gatsby-node.js script so I can create my product pages getting the data from that MySQL table.

It would be quite difficult to share some code because I would need to show you a few files. I could, but it wouldn't be something that you could run, of course.