Direct Schema Reference

[wasurdev]

Hello! I am curious as to whether or not graphql-mesh can be configured to reference schemas directly - that is, as opposed to 'linking' to schemas in the .meshrc.yaml file, simply referring to schema files.

To expand on this, the intent is to see if we can reduce our deployment needs from requiring a hook to update the introspectionCache.json and instead have the mesh update itself in real time when those schema files change.

[santino]

Can you expand more on this?

If I understand correctly you plan to update the data sources during runtime.
Is your idea to ping the Mesh service when one of your data sources gets updated so that Mesh can consume the newest API specifications and consume the new schemas straight away?

In this case, how do you plan to handle breaking changes, for example?
In my opinion, it is better to be explicit about how you consume data sources.
Updates can bring changes that might not necessarily be breaking in terms of the contract, but can still break type definitions of the auto-generated GraphQL schema, just to give an example; and that might break consuming clients that introspect your GraphQL schema.

Having code in production that self updates itself without a manual pass, which is normally provided through code review, QA, and automated tests sounds a bit dangerous.

[wasurdev]

Santino, your reply is greatly appreciated and I concur with your concerns.

As a followup question, how do you see 'breaking changes' handled in an optimal CI/CD process when leveraging a json schema cache? Apologies for my ignorance, I just want to ensure I am prepared to leverage mesh with its full capability before we start looking at processes.

[santino]

I would still advise on handling updated schema consumption through PRs.

How much you can automate is down to the data source you consume, assuming your JSON schema is remote and you're not controlling it directly.
F.i. you could have a polling system as part of CI on your graphql service that checks the version number on your data source and eventually raise a PR to consume the new version of the schema.
If the data source is built internally within your organisation you could implement a push mechanism, so that every time a new version is deployed it will push a notification to your GraphQL service's CI pipelines and those will eventually raise the PR.

Those are just some high-level ideas, it depends very much on how important it is to you to consume a fresh schema and your infrastructure setup.

The key point is that you end up with a PR that implements the new version.
This will probably be as simple as a single string change in your meshrc.yaml; so you might think is not much really.
But then, once you have this, you can take advantage of tools like graphql-inspector that are capable of creating full reports for your PRs, listing all the changes caused to the resulting GraphQL schema.

This is what I would do.
All the PRs are checked against schema changes and I can read the report to see what is breaking and eventually address it; for example, I can:

• Decide to absorb the downstream changes and use mesh transforms to keep my GraphQL schema as before
• Decide to accept the changes in my schema and work with consumers of my GraphQL service to handle these

If you don't have changes you could add a label to the PR, like "ready to merge".
Then again, you can set a job on your CI that looks for all the PRs marked as "ready to merge" and merges them automatically.

In this case, when all is good you don't have much (or maybe even none) manual work to do.
When there are changes you are informed and can make a deliberate choice.

Again, all of these are some answers that are coming to mind when thinking about how I would answer your questions.
Don't take it for gold, it's just the way I'm roughly thinking about the use case :)

[wasurdev]

Thanks again for your reply santino. Your insight is greatly appreciated!

A quick followup, just to make sure we're not missing anything here in regards to schema updates, etc - are you aware of how graphql-mesh is able to be notified that it's introspectionCache.json (for example) is updated? Wondering if we follow your advice and kick off processes to recreate that file, how is the mesh server aware there's been an update? Is this a configuration in mesh that we're overlooking?

[santino]

Mesh is not aware of when your data sources update, nor it does try to understand when this happens.
When you know your sources have updates, you can simply consume their latest specs.
The specs might have a version number within the document.
Or sometimes, when working with rest, they might include the version number in the endopint, such as http://myRestApi/v1/swagger, for example; this will be the case where you'll need to update the meshrc to change the endpoint your download your specs from.

It is up to you how you want to consume the data sources, but where Mesh can help is to understand the GraphQL schema.
Not really understand actually (that intelligence is out of scope), but generate.
Probably not really documented, but you can run:
graphql-mesh dump-schema --output ./static/schema.graphql

And this will save your resulting GraphQL schema combined from all your data sources into a static file.
Then you can setup your CI with graphql-inspector to compare the current schema (as of master) with the newer schema (from a PR).
That will give you the insights for changes.

In this case, when your data source update, you can simply generate a PR which scope is only to run graphql-mesh dump-schema and compare resulting GraphQL schemas through a step that involves graphql-inspector.

Hope this gives more info about my thoughts for your use case.

[wasurdev]

Interesting. So the purpose of the introspectionCache file is so mesh 'understands' the schema, correct?

Wouldn't that mean the mesh would need to be restarted upon any schema change from sub-graphs, after the introspectionCache json file is updated?

For example, if Mesh is serving service A and service B, and service A has a schema change, wouldn't we need to run write-introspection-cache, and then restart mesh so it gets the update?

[santino]

So, we're mixing a couple of concepts here.

If a schema changes, yes, you will need to restart your server, because Mesh needs to build the execution layer for your GraphQL operations, so is not just about "understanding" (or better "knowing") the schema/contract; but it is about building the instances that are able to talk to your data sources.

This is not really a problem after all, because if you take my advice, you raise a PR when your sources update, then after merging this, you'll need to deploy it. Hence as you're doing a deployment, you're starting new instances for your server anyway and those at startup will build the execution layer in line with latest schema changes that are committed as part of your build.

As about introspection-cache.
That is literally just a convenient thing to avoid downloading the specs from your data sources at startup.
Normally every time you start your server, Mesh will download the documents from your data sources and finally build a GraphQL schema and execution layer out of that.
This can potentially cause problems if one of your data sources is temporarily offline when your server is starting (e.g. after a new deployment) Mesh would not be able to download those documents and do its job.

To avoid this potential issue, you can pre-download the documents from your data sources with the introspection-cache command.
In this case, Mesh saves them to a static file and at startup it will try to read the file; if present will use its content as the source for the data sources documents; otherwise it will behave as usual and download the documents from the data sources during startup.