Dynamic Routes with File System Route API via non-GraphQL data source

[mcclaskiem]

This is a continuation of some back and forth around related to #25171

I am trying to better understand the new File System Route API and how to implement it in order to solve an issue we have with profile pages being created by a user but because a build has not occurred it ends up 404'ing if that person shared or accessed that URL immediately.

This is a replacement for the explicit createPages API, yes.

@LekoArts mentioned this in the above thread and after looking through the API docs I am not seeing how I would accomplish what I want to do without using a combination of createPages and the new "client only fallback" approach.

Basically at build I make a GET request to our API to get all the profile objects and then call createPage on each object with its respecitve URL and component inside of gatsby-node.js, we all know this method. Now using only the File System Route API approach of a collection and fall-back files, I don't see where I would let Gatsby now about all the profiles that exist in our DB since we don't use GraphQL and it looks like this approach only accepts a query for a single Profile, when it is requested.

My question then becomes, since this is meant to replace the createPages API, how can I still generate pages (profiles) during builds and have a fallback component that will catch any slugs that have not yet had a static HTML page generated from a build. I'm hoping this can replace the current approach we have to use which requires doing a path check inside the 404 page component and serving content if a profile does in fact exist.

[mcclaskiem]

I just spun up version 2.26.x of Gatsby in order to test the above scenario. I can see all the static version of the pages being built so I know there are HTML files that exist for a given page. I also have added in a file pages/profiles/[id].js in order to serve as a fallback for when a page exists but has yet to be compiled during build.

After trying to access one of the pre-built html pages via its direct URL the routing is sending everything to my "fallback" component. As I stated before, I do not use GraphQL with our site and I don't see anywhere how one can create a bunch of pages during build similar to what createPages did previously. The collection routes feature does not show anywhere in the docs how one can source data to be used by this files without GraphQL.

This is a mission critical architecture problem for me to solve and any help would be greatly appreciated.

[LekoArts]

My question then becomes, since this is meant to replace the createPages API, how can I still generate pages (profiles) during builds and have a fallback component that will catch any slugs that have not yet had a static HTML page generated from a build

As the doc states quite early it expects you to have your data in our GraphQL data layer. Creating nodes with your data you get from your API is fairly easy with sourceNodes: https://www.gatsbyjs.com/docs/creating-a-source-plugin/

I'm hoping this can replace the current approach we have to use which requires doing a path check inside the 404 page component and serving content if a profile does in fact exist.

As mentioned in the previous issue also before the Route API this wasn't necessary. The Route API is some syntax sugar that translates into the same underlying Node APIs, so it also only uses createPages and sets the matchPath: https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-plugin-page-creator/src/create-client-only-page.ts

After trying to access one of the pre-built html pages via its direct URL the routing is sending everything to my "fallback" component. As I stated before, I do not use GraphQL with our site and I don't see anywhere how one can create a bunch of pages during build similar to what createPages did previously.

You can certainly continue to use createPages and use the [id] notation in the pages directory. A working example (with only Route API) is here: https://github.com/gatsbyjs/gatsby/tree/master/examples/route-api/src/pages/products

If you can provide a minimal reproduction of your setup I can look into it, maybe you're setting the matchPath incorrectly for example.

Side note: My colleague explained the same concept here: https://twitter.com/gill_kyle/status/1329228307104034822

[mcclaskiem]

@LekoArts Ok thanks for the in-depth explanation. I overlooked the matchPath aspect and I implemented that based on how I think it should be, however now no matter what profile URL I access it return the same page/data. Below is what I have in gatsby-node.js

  const profileTemplate = path.resolve('src/templates/profile.js');

  const allProfiles = await getProfiles().catch((error) => {
    console.log(error.message);
  });

  allProfiles.forEach(profiles => {
    createPage({
      path: `/profile/${profile.id}`,
      matchPath: `/profile/:id`,
      component: profileTemplate, 
      context: {
        profile: profile
      }
    })
  })

I have also configured profile/[id].js but that is not what is being used to render the route. Am I misunderstanding matchPaths usage?

Thanks!

[mcclaskiem]

@LekoArts So I just updated my createPage to the below and it seems to be working as you mentioned with createPage co-existing with the fallback comp. I need to just tweak how the data is passed, etc but regardless it seems to work fine. Appreciate your input/insights on this.

  allProfiles.forEach(profile => {
    createPage({
      path: `/profile/${profile.id}`,
      matchPath: `/profile/${profile.id}`,
      component: profileTemplate, 
      context: {
        profile: profile
      }
    })
  })

[LekoArts]

Glad you got it working! Sidenote: What happens when you drop the matchPath completely?

[mcclaskiem]

@LekoArts So I just ran a test which consisted of a simple console.log inside the template component and the fallback component. These both share the same "view" component inside of them.

For both instances I have cleared out my public and my .cache folders prior to running gatsby build followed by gatsby serve.

with matchPath:

I can see that after serving the build when I access a route directly for a page that existed prior to build, I see the console.log from the template component that was used in my createPage call, as expected. I then create a new page via our UI, copy the direct URL, open a new tab and when I access it, I see the console.log from my fallback component.

without matchPath:

Same preliminary steps as above however when I try to access a page that was created at build time, I see the console.log from the fallback component. The page still loads as it normally would however the browser is not being served the pre-rendered version.

Let me know if this test is sufficient or if you would like anymore info!