Recommendation on including large third-party SDL

[kettanaito]

Hey. First of all, thank you for an amazing project!

I've got a GraphQL API that uses a third-party GraphQL API. The latter ships with the ready-to-consume TypeScript definitions and SDL, which I include in my graphql-codegen.yml:

schema:
  - node_modules/third-party-api/schema.graphql:
      assumeValid: true
      assumeValidSDL: true
  - server/schema/**/*.{ts,gql}
documents: src/**/*.
generates:
  graphql-schema.ts:
    config:
      contextType: server/schema/types/Context#Context
    plugins:
      - typescript
      - typescript-operations
      - typescript-resolvers

The issue is that the third-party schema (node_modules/third-party-api/schema.graphql) is a 30k+ lines long file. Importing it makes the generated graphql-schema.ts file subsequently large, which in turn slows down my project's compilation time quite significantly.

Please, what would you recommend in my case?

I don't know if importing from the node module would have performance implications rather than inlining the entire schema.graphql in the generated TypeScript file. Some sort of dead code elimination would be nice, but I understand it's rather difficult to implement, given types may be cross-referenced, resulting still in a large chunk of lines being included.

[dotansimha]

@kettanaito you can try onlyOperationTypes: true, preResolveTypes: true to get a smaller output. Can you please try and let me know if this helps?

[kettanaito]

Hey, @dotansimha. Thank you for a quick reply.

When using onlyOperationTypes, as the name suggests, some of my own GraphQL types get stripped away, resulting in broken TypeScript definitions. For example:

# server/schema/types/User.gql
type User {
  name: String!
}

extend type Query {
  user: User
}

Normally, the User type generates a corresponding TypeScript type User:

// graphql-schema.ts (generated)
export type User = {
  name: string
}

I suppose I'm not using the User type in operations somehow, that's why its TypeScript counterpart gets stripped entirely. This breaks my compilation, as I have code relying on the types derived from my schema (such as the User type).

// some/other/code.ts
import { User } from '@/graphql-schema'

Using the onlyOperationTypes: true option does reduce the size of the file in half (previous 30k, with the option 14k), but since it removes my own crucial types, it doesn't seem like I can use it.

Using the preResolveTypes: true option also has an impact on the generated schema, but far less significant (previously 30,200, with the option 30,000).

Is there a way to tell the code generator to import types from the third-party-package rather than inlining them? Since the package is already a generator's dependency (relative path to node_modules preset in the graphql-codegen.yml), it'd be a safe option to try.

[dotansimha]

If you are in a frontend context, no reason to use User type, since the client needs to use the types generated by the operations, so keep in mind that you might be using a type that you shouldn't and that's the reason for the error.
And you can use both options to get the most out of it.
You can tell codegen to use some generated types from external, but not as partial schema, so I'm not sure it's relevant for this use case?

[kettanaito]

I'm on the backend context and wish to have a typed query.

@dotansimha, is there a way to force GQL Code Generator to import types from a third-party node_modules instead of inlining them into the generated document?

I've tried using a custom plugin but plugins can modify the generated module only via addToSchema, which happens after the schema is validated. I don't think that'd help in my case anyway.

I have a strong suspicion that it's the size of the generated module that slows my build down. I'd like to try how it behaves if all the third-party types would be imported from that module.

// generated.ts
+import { User, Organization } from '@octokit/graphql-schema'

-type User { ... }
-type Organization { ... }

Why I think this may work:

• I don't need the type definitions from third-party schema. I only need those types to use them in the generated queries.

I've been trying to solve this for months now, perhaps you can give me an example of a simple setup that uses my own schema + a third-party schema. You may take @octokit/graphql-schema as an example.

My intention is to have type-safe queries and their results when querying a third-party API:

gql`
  query {
    viewer {
      login
    }
  }
`

• I don't define viewer and its type by myself, it comes from the third-party types.
• I still wish to get the result (type of the viewer.login) in my TypeScript code.

[kettanaito]

Once I thought better about how these generated types should behave, I came to a few useful conclusions:

• I don't need to combine third-party and my own schemas, they will never cross-reference.
• I use a separate GraphQL client to communicate with a third-party API (it's not schema stitching).
• GraphQL Code Generator rightfully fails when processing a documents glob and finding a GraphQL query that's meant to communicate with a third-party service (types are unknown for my own schema, which is correct).

It seems that my real question comes down to: how to use GraphQL Code Generator where multiple *.gql files may describe queries to different services/schemas.

What I'm trying to do right now is to generate separate type definitions for each service, and separate Typed Document Nodes as well. This does work, but the issue is how to configure the library so it distinguishes between different GraphQL queries and doesn't attempt to process third-party queries when generating operations for my own service.

generates:
  ./service.js:
    # This glob will also catch GraphQL queries written for the third-party API
    # which will fail this generation due to "X does not exist in type Y" errors.
    documents: **/*.gql

Let's try a loader

First, I've tried writing a custom documents loader to analyze if a loaded query has a specific comment (i.e. a name of the third-party service). If it doesn't, skip it. In the end, it looked something like this:

// my-loader.js
const { readFileSync } = require('fs')
const { parse } = require('graphql')

module.exports = function loader(docString, config) {
  const { config: pluginConfig } = config
  const { serviceEquals, serviceNotEquals } = pluginConfig || {}
  const contents = readFileSync(docString, { encoding: 'utf8' })

  const predicates = [
    serviceEquals ? contents.includes(`# service: ${serviceEquals}`) : true,
    serviceNotEquals
      ? !contents.includes(`# service: ${serviceNotEquals}`)
      : true,
  ]

  if (!predicates.every(Boolean)) {
    return
  }

  return parse(contents)
}

generates:
  ./service.js:
    # Still target all files
    documents:
      - **/*.gql:
        loader: my-loader.js
        config: 
          serviceNotEqual: github

Now whenever the library analyzes a query that has a # service: github comment in it, it will skip it. That's right what I need! There's only one implication though...

The loader cannot process non-GraphQL files. I'd also like to extract GraphQL queries from my ts,tsx files that write them in the gql tag. Turns out the loader cannot process anything else but GraphQL SDL files. The return type of the loader must be a Document.

Plugins, perhaps?

Okay, so to still be able to write GraphQL in TypeScript modules I've tried writing a custom plugin. My assumption was that a plugin accepts already parsed (extracted) GraphQL query, which includes queries from TS modules.

There comes another challenge: a plugin executes after the schema was validated. This means that when the library processes a GraphQL query that's meant for a third-party service, it will still fail. Since I cannot use loader outside of GraphQL files, there's no way to distinguish which query should be processed for a specific schema if those queries are written in TS modules.

// Component.ts
const QUERY = gql`
  # service: github
  query {
    viewer { login }
  }
`

For example, when processing Component.ts, GraphQL Code Generator will throw because there's no type viewer on my schema (which is correct). This query should be meant for third-party usage, but I don't see a way to tell the library how to skip certain queries if they are located in non-GraphQL modules.

@dotansimha, am I even moving to the right direction here? I feel like the things I try are way too complicated for a simple use case I have: generate types and typed documents for queries that are meant for different services, but from the code that mixes those queries at the same time.

It's worth mentioning that introducing a custom file pattern to distinguish between such queries should work (i.e. ThirdPartyQuery.github.gql), but I'm against this. This will lead to file pollution pretty quickly and I still need to create GraphQL queries via gql in my TS modules in some cases, I cannot isolate them.