Some how stitching two data sources together under one entry in Gatsby's graphql schema

[wildpow]

My company uses two sources for data. DatoCMS for page/product content (using gatsby-source-datocms) and Shopify for product pricing/inventory (using gatsby-source-Shopify). Product content is connected to product pricing through a Shopify-id plug-in on DatoCMS. This has been fine for single product pages. In the gatsby-node.js, we use createPages, pass the Shopify ids through context to the template, and the template queries the Shopify data.

The problems start when we need that same pricing in pages that are not programmatically created, like a page that listed all products. I run a page query to get the content and get the Shopify ids but, now I'm stuck.

My first thought was to pass the ids to the static query component, but this feature will never happen from what I have read. My next idea was to stitch these two data sources together in Gatsby's schema into one entry or create a field on the DatCMS nodes and inject the Shopify data. I've never dug into Gatsby's internal APIs or seen an example of this to know if it's even possible. Am I going down a fruitless rabbit hole, or is there another solution I'm not seeing?

[vladar]

@wildpow You can absolutely stitch types together. But that will require diving deep into schema customization API. See this blog post: https://www.gatsbyjs.com/blog/2019-03-18-releasing-new-schema-customization/

Maybe this example will be a good start:

gatsby/examples/using-type-definitions/gatsby-node.js

Lines 28 to 40 in 5bc84a9

	schema.buildObjectType({
	name: `CommentJson`,
	fields: {
	text: `String!`,
	blog: {
	type: `BlogJson`,
	resolve(parent, args, context) {
	return context.nodeModel.getNodeById({
	id: parent.author,
	type: `BlogJson`,
	})
	},
	},

[wildpow]

I'm having trouble wrapping my head around taking a list of products for one source and combining the data from another source based on a shared field shopifyId === shopifyConnection.
My solution since I made my original post has been to use useStaticQuery but, doing this for every page that needs this combination of data seems unmanageable.

export default () => {
  const { shopifyProducts, datoProduct } = useStaticQuery(
    graphql`
      query products {
        shopifyProducts: allShopifyProduct {
          nodes {
            shopifyId
            priceRange {
              minVariantPrice {
                amount
              }
              maxVariantPrice {
                amount
              }
            }
          }
        }
        datoProducts: allDatoCmsProduct {
            nodes {
              shopifyConnection
              slug
              id
              name
              saleInfo {
                saleBanner
              }
            }
          }
      }
    `,
  );
  const combinedProduct = [];
  shopifyProducts.nodes.forEach((shopifyItem) => {
    datoProducts.nodes.forEach((datoItem) => {
      if (datoItem.shopifyConnection === shopifyItem.shopifyId) {
        combinedProduct.push({ ...datoItem, ...shopifyItem.priceRange });
      }
    });
  });
  return { products: combinedProduct };
};

[wildpow]

My other thought was to use onCreatePage and pass combined data through pageContext but, it seems you can't get graphql inside the function to make the initial queries.

[vladar]

Have you tried the schema customization approach I've mentioned above? It will be something like this:

// in gatsby-node.js

exports.createSchemaCustomization = ({ actions, schema }) => {
  const ExtendedDatoCmsProduct = schema.buildObjectType({
    name: `DatoCmsProduct`,
    fields: {
      shopifyPriceRange: {
        type: `ShopifyProductPriceRange`,
        resolve(root, args, context, info) {
          const node = context.nodeModel.runQuery({
            query: { filter: { shopifyId: { eq: root.shopifyConnection } } },
            type: `ShopifyProduct`,
            firstOnly: true,
          })
          return node ? node.priceRange : null
        }
      }
    }
  })

  actions.createTypes([
    ExtendedDatoCmsProduct 
  ])
}

Writing blindly so there might be errors there but it should give you some direction. Also, check the blog post and the example above.

[wildpow]

With a little modification of your example, I was able to get a new field of shopifyPriceRange with a value of null, but both root.shopifyConnection and node.priceRange were coming up as undefined.

My first go at it using the docs and your example I came up with this:

exports.createSchemaCustomization = ({ actions, schema }) => {
  const ExtendedDatoCmsProuct = schema.buildObjectType({
    name: `DatoCmsMattress`,
    fields: {
      shopifyPriceRange: {
        type: `ShopifyProductPriceRange`,
        resolve(root, args, context, info) {
          const node = context.nodeModel.getAllNodes({
            type: `ShopifyProduct`,
          });
          return node.map((item) =>
            item.shopifyId ===
            root.entityPayload.attributes.shop_matt_connection
              ? item.priceRange
              : null,
          );
        },
      },
    },
  });
  actions.createTypes([ExtendedDatoCmsProuct]);
};

GraphiQL results:

 {
    "shopifyPriceRange": {
          "minVariantPrice": null,
          "maxVariantPrice": null
       }
 },

Which seem to be getting closer, still no data but, I'll keep messing with it.

Thanks for your help so far.

[wildpow]

Using either your example or the one from my last post. I get the new field but null for the data. Also, trying to console log the results of runQuery like in your example, I get back Promise { <pending> }. Is that what I should expect?

[wildpow]

Here is my only success so far:

exports.createResolvers = ({ createResolvers }) => {
  createResolvers({
    DatoCmsMattress: {
      shopifyInfo: {
        type: [`ShopifyProduct`],
        resolve(source, args, context, info) {
          const fieldValue =
            source.entityPayload.attributes.shop_matt_connection;
          const shopifyMatts = context.nodeModel.getAllNodes({
            type: "ShopifyProduct",
          });
          return shopifyMatts.filter((matt) => matt.shopifyId === fieldValue);
        },
      },
    },
  });
};

Which adds the entire Shopify product to Dato's entry. Not exactly what I was looking for, but it works. I got the idea from the same file you posted.

gatsby/examples/using-type-definitions/gatsby-node.js

Lines 88 to 106 in 5bc84a9

	AuthorJson: {
	// Add a field to an existing type by providing a field config.
	// Note that added fields will not be available in the input filter
	// when no type definitions are provided wth `createTypes`.
	posts: {
	type: [`BlogJson`],
	resolve(source, args, context, info) {
	// We use an author's `email` as foreign key in `BlogJson.authors`
	const fieldValue = source.email
	const posts = context.nodeModel.getAllNodes({
	type: `BlogJson`,
	})
	return posts.filter(post =>
	(post.authors || []).some(author => author === fieldValue)
	)
	},
	},
	},

Since my development build times are so long anyway, I'm not sure I could measure a negative impact if there were one.

[vladar]

I discourage you from using getAllNodes and looping through all nodes as it is expensive and will be pretty slow with many nodes. Have you tried using runQuery instead? It uses indexing under the hood to find the required data:

-          const shopifyMatts = context.nodeModel.getAllNodes({
-            type: "ShopifyProduct",
-          });
-          return shopifyMatts.filter((matt) => matt.shopifyId === fieldValue);
+          return context.nodeModel.runQuery({
+            query: { filter: { shopifyId: { eq: fieldValue } } },
+            type: `ShopifyProduct`,
+            firstOnly: true,
+          })

[wildpow]

This is what I get back in GraphiQL:

[wildpow]

@vladar This works:

exports.createResolvers = ({ createResolvers }) => {
  createResolvers({
    DatoCmsMattress: {
      shopifyInfo: {
        type: [`ShopifyProduct`],
        resolve(source, args, context, info) {
          const fieldValue =
            source.entityPayload.attributes.shop_matt_connection;
          return context.nodeModel.runQuery({
            query: { filter: { shopifyId: { eq: fieldValue } } },
            type: `ShopifyProduct`,
            // Not sure why removing this made it work?!?
            // firstOnly: true,
          });
        },
      },
    },
  });
};

[vladar]

Oh, yeah, I've missed that you expect to get an array of values. for this field. firstOnly simply extracts the first value from this array and returns it. So it would've worked if your type was type: "ShopifyProduct". But for type: ["ShopifyProduct"] the firstOnly flag is not needed.