per-page data fetching like Next.js getStaticProps for Gatsby

[nibtime]

Last week I started writing my first Gatsby Theme. I want to include some nice patterns there that I learned from developing a multilingual website (7 languages) with Gatsby and MDX. Overall, this was a very pleasant experience, when it comes to meshing large content structures Gatsby is awesome and unbeatable. However, the thing I was missing most in this process was a proper solution for i18n content and pages with/and a per-page data fetching abstraction like Next.JS has with getStaticProps. Does anybody have similar requirements and is interested in a theme like this?

Goals

• streamline the page generation process via templates for i18n content (with a primary focus on MDX, but in principle every source/transformer plugin that exposes content through GraphQL nodes should be supportable). Both multilingual and multiregional content should be well representable and well maintainable with a Git-based CMS like Forestry. Page paths should never depend on filenames and folder hierarchy and should be sourced entirely from the content.

• create a getStaticProps-ish per-page data fetching abstraction that complements Gatsby's page queries and that is aware of the i18n page generation process. getStaticProps should re-fetch when data changes are triggered for a good developer experience.

• support the CDN-level i18n-rewrites of Firebase Hosting

I couldn't find any existing plugins or themes that would provide all of those things. I know that there are i18n plugins available, but I think none of them quite hits the spot, at least not in a way like i18n routing of Next.JS 10 does (which is unsurprisingly intertwined with getStaticProps).

First impression

I am still in the PoC phase and exploring how this could be done best. I managed to get all of the above to work in the first iteration. Here are some GIFs and images to give a gist on how that all sort of looks like:

getStaticProps (language-aware data fetching from a weather API)

pulling region-specific MDX content fragments into a page via query (this is where Gatsby is great)

on Firebase Hosting with i18n rewrites

the page

localized MDX page content

---
pageId: "/"
language: de
title: Startseite
---

# Startseite (de)

to language: <Link to="/en">Englisch</Link>

to region: <Link to="/de-DE">Deutsch (Deutschland)</Link> | <Link to="/en-US">Englisch (Vereinigte Staaten)</Link> | <Link to="/en-GB">Englisch (Vereinigtes Königreich)</Link>

#### <WeatherFromStaticProps />

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Ad unde in eaque, libero harum assumenda dolorem eveniet perspiciatis dignissimos officiis? Voluptatibus qui ipsum dolore est commodi magni repellendus beatae atque!

Lorem, ipsum dolor sit amet consectetur adipisicing elit. Corrupti, molestiae aliquid explicabo repellendus accusantium deserunt dolore numquam nostrum architecto esse corporis delectus dolor? Voluptatem fugit, assumenda dignissimos voluptates atque perspiciatis.

template

import { graphql, Link } from "gatsby"
import React from "react"
import tw from "twin.macro"
import { usePage } from "gatsby-theme-inhalt"
import { MDXProvider } from "@mdx-js/react"
import { MDXRenderer } from "gatsby-plugin-mdx"
import type { StaticProps as HomePageStaticProps } from "./getStaticProps"

const useHomePage = () => usePage<GatsbyTypesGraphQL.HomePageQuery, HomePageStaticProps>()

const Prose = tw.article`
  prose -lg:prose-lg lg:prose-2xl prose-primary
`

const WeatherFromStaticProps = () => {
  const { weatherData } = useHomePage()
  return weatherData ? <span>{`${weatherData?.name}: ${weatherData?.weather[0]?.description}`}</span> : <></>
}

const HomePage = () => {
  const { data } = useHomePage()
  return (
    <MDXProvider components={{ WeatherFromStaticProps, Link }}>
      <div tw="flex justify-between py-10">
        <div tw="w-1/12" />
        <Prose tw="block w-10/12">
          {data?.mdx?.body && <MDXRenderer>{data.mdx.body}</MDXRenderer>}
          {data?.germanLaw?.body && <MDXRenderer>{data.germanLaw.body}</MDXRenderer>}
          {data?.americanLaw?.body && <MDXRenderer>{data.americanLaw.body}</MDXRenderer>}
        </Prose>
        <div tw="w-1/12" />
      </div>
    </MDXProvider>
  )
}

export default HomePage

export const query = graphql`
  query HomePage($pageId: String, $language: String, $region: String) {
    mdx(fields: { pageId: { eq: $pageId }, language: { eq: $language }, region: { eq: $region } }) {
      body
    }
    germanLaw: mdx(fields: { dataId: { eq: "law" }, language: { eq: $language }, region: { eq: "DE" } }) {
      body
    }
    americanLaw: mdx(fields: { dataId: { eq: "law" }, language: { eq: $language }, region: { eq: "US" } }) {
      body
    }
  }
`

file and folder organization

configuration interface

page specification

import type { PageSpecification } from "gatsby-theme-inhalt"
import path from "path"

const templatePath = path.resolve(__dirname, "template.tsx")
const getStaticPropsPath = path.resolve(__dirname, "getStaticProps.ts")

const pageSpecification: PageSpecification = {
  templatePath,
  getStaticPropsPath
}

export default { "/": pageSpecification }

theme configuration

import type { Theme } from "gatsby-theme-inhalt"
import path from "path"
import home from "./pages/home"
import { GATSBY_DIR, REPO_ROOT } from "#env/dirs"

const i18n: Theme["options"]["i18n"] = {
  defaultLanguage: "de",
  supportedLanguages: ["de", "en", "fr"],
  supportedRegions: ["DE", "AT", "CH", "US", "GB"]
}

const pages: Theme["options"]["pages"] = {
  ...home
}

// poor mans syntax highlighting and formatting with Apollo VS Code Extension
const gql = (x: TemplateStringsArray) => x.join("")

const theme: Theme = {
  resolve: "gatsby-theme-inhalt",
  options: {
    firebase: {
      configPath: path.resolve(REPO_ROOT, "firebase.config.json"),
      firebaseJsonOutputPath: path.resolve(REPO_ROOT, "firebase.json"),
      gatsbyPublicDirPath: path.resolve(GATSBY_DIR, "public"),
      usei18nHosting: {}
    },
    contentProviders: {
      "gatsby-plugin-mdx": {
        pathsIntoData: [["frontmatter"]],
        pagesSeed: {
          pathIntoNodesArray: ["allMdx", "nodes"],
          query: gql`
            query GenerateMdxPages {
              allMdx(filter: { fields: { pageId: { ne: null }, published: { eq: true } } }) {
                nodes {
                  fields {
                    pageId
                    language
                    region
                    canonicalPath
                    canonicalPathWithSlug
                  }
                }
              }
            }
          `
        }
      }
    },
    pages,
    i18n
  }
}

export default theme

[herecydev]

What do you see this doing/solving differently than using source plugins and page queries?

For example, each node can be localization-aware and you can use this as a filter:

export const query = graphql`
  query ($locale: String) {
    myNode(locale: {eq: $locale}) {
      id
      title
    }
  }

[nibtime]

@herecydev
I know that each node can be implicitly localization-aware and I don't intend to solve anything differently from page queries and source plugins. I am trying to complement them. For the use case where you want to pull just an arbitrary piece of, let's say, localized MDX content into a page, Gatsby and page queries are great, that's exactly one of the things I don't want to miss. But that just works as long as there is a source plugin that perfectly supports the way you need to bring static data into a page and dependent data fetching is not possible with page queries. That's why I am trying to emulate getStaticProps with i18n routing of Next 10, so my i18n considerations are coming from experimenting with that.

https://nextjs.org/docs/migrating/from-gatsby explains the differences in data fetching between Gatsby and Next quite well. I believe that for pulling content from MDX, Gatsby is better, as with Next you would have to repeat reading in files and parsing frontmatter over and over again in getStaticProps. However, getStaticProps enables unopinionated data fetching of type () => Promise<T> per page. Since Next 10 with i18n routing, getStaticProps receives the page locale as a param, so the data fetching can be locale-aware as well.

I have an upcoming todo where I need static data on a single page that comes from Stripe and Firebase, and the data to return from Firebase is dependent on data fetched from Stripe. There is gatsby-source-stripe and gatsby-source-firebase, but they don't know each other and GraphQL queries are not chainable like () => Promise<T> is (which is why this does not work), so dependent fetching of static data per page is not possible with page queries. This would be super-easy with Next and cumbersome with Gatsby. Writing a source plugin for that or hacking that into createPages (with no hot reloading then I suppose) just feels wrong to me.

There is a good article by Jared Palmer about where Gatsby can be painful and with some problems that could have been solved way easier with Next and getStaticProps.

My concerns with i18n and what I'm trying to solve are about what has to happen in createPages, not about pulling localized content into a page via page query after createPages. That comprises somehow streamlining page and path generation based on metadata to be found in nodes, as there are no dynamic routes and getStaticPaths like in Next. There needs to be some sort of unique identifier for a page to identify it across locales, the locale, and optionally a localized slug.

When it comes to page generation in createPages from GraphQL, you need to execute at least one "seed query" to bring in an array of nodes with this metadata to kick off the page generation process from templates. If no pages are created, no page queries will be executed either. At this point, which source plugin owns the node is kind of irrelevant, as long as the metadata necessary for URL path generation with an association to a template is provided. Where this metadata is to be found will vary between source plugins. With MDX it will be somewhere in frontmatter, other source plugins, e.g. headless CMS with built-in i18n support will each have their own node model and that information could be found at an arbitrary location in the node object tree and with arbitrary field names. My theme extracts this metadata in onCreateNode (you can tell it the paths in the object tree where it should look for it) so every node that represents a localized page has it available in node.fields (and does checks if locale codes are malformed or unsupported and report warnings if so). I am currently only testing this with MDX, but in principle, this should not be bound to a certain source plugin (I will only add more the first time I see a potential use case for myself though).

Then you can add queries like this in the theme config (because this metadata will exist flat on node.fields):

query GenerateMdxPages {
  allMdx(filter: { fields: { pageId: { ne: null }, published: { eq: true } } }) {
    nodes {
      fields {
        pageId
        language
        region
        canonicalPath
        canonicalPathWithSlug
      }
    }
  }
}

Then all pages will be generated at canonicalPath and maybe canonicalPathWithSlug (built by a configurable URL scheme) and the per-page async data fetching for each page gets executed with this metadata as parameters, so it can return different data depending on the locale and spreads it into pageContext when calling createPage. That's my emulation attempt of getStaticProps with i18n routing of Next.JS. If this can or will be any good I have no idea yet myself 😄

I found that knowledge about the localized URL scheme is also necessary to make Gatsby work with the i18n rewrites of Firebase Hosting (that's how this is all somehow connected). If you use the i18n rewrites of Firebase Hosting, you can share links like https://mysite.web.app/about and each visitor will get served the localized version of the page depending on their IP location and browser language settings without any client-side code. I'd like to have that working out-of-box for every site I create. It's a fairly new feature of Firebase Hosting and I haven't found a single blog or discussion thread yet on how to use it together with Gatsby. If anybody already solved or attempted this, I'd be very interested to know about your solutions.

There are more aspects than this to i18n than I don't intend to touch in any way:

• translation strings + number and date formatting -> React.Intl, i18Next
• text direction, i.e. rtl. -> design system

The relationship of those things I see with Gatsby is just that pageContext will have locale information available and that get passed it into React Context Providers of libraries that handle those things in wrapPageElement.