Skip to content

Commit ba5720b

Browse files
acaoTallTed
acao
and
TallTed
authored
debounce schema change events to fix codegen bugs (#3647)
* debounce schema change events to fix codegen bugs on mass file changes, network schema is requesting schema way too frequently because the schema cache is invalidated on every schema file change to address this, we debounce the onSchemaChange event by 400ms also, fix bugs with tests, and schemaCacheTTL setting not being passed to the cache * changeset * fix: docs update * fix: upgrade extension bundlers * Apply formatting suggestions from code review thanks @TallTed! Co-authored-by: Ted Thibodeau Jr <[email protected]>
1 parent e33af28 commit ba5720b

File tree

11 files changed

+582
-90
lines changed

11 files changed

+582
-90
lines changed

‎.changeset/eighty-maps-change.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
'graphql-language-service-server': patch
3+
'graphql-language-service-cli': patch
4+
'vscode-graphql': patch
5+
---
6+
7+
**Bugfixes**
8+
9+
debounce schema change events to fix codegen bugs to fix #3622
10+
11+
on mass file changes, network schema is overfetching because the schema cache is now invalidated on every watched schema file change
12+
13+
to address this, we debounce the new `onSchemaChange` event by 400ms
14+
15+
note that `schemaCacheTTL` can only be set in extension settings or graphql config at the top level - it will be ignored if configured per-project in the graphql config
16+
17+
**Code Improvements**
18+
19+
- Fixes flaky tests, and `schemaCacheTTL` setting not being passed to the cache
20+
- Adds a test to validate network schema changes are reflected in the cache

‎packages/graphql-language-service-server/README.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@ module.exports = {
158158
// a function that returns an array of validation rules, ala https://github.com/graphql/graphql-js/tree/main/src/validation/rules
159159
// note that this file will be loaded by the vscode runtime, so the node version and other factors will come into play
160160
customValidationRules: require('./config/customValidationRules'),
161+
schemaCacheTTL: 1000, // reduce or increase minimum schema cache lifetime from 30000ms (30 seconds). you may want to reduce this if you are developing fullstack with network schema
161162
languageService: {
162163
// this is enabled by default if non-local files are specified in the project `schema`
163164
// NOTE: this will disable all definition lookup for local SDL files
@@ -257,14 +258,16 @@ via `initializationOptions` in nvim.coc. The options are mostly designed to
257258
configure graphql-config's load parameters, the only thing we can't configure
258259
with graphql config. The final option can be set in `graphql-config` as well
259260

260-
| Parameter | Default | Description |
261-
| ----------------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
262-
| `graphql-config.load.baseDir` | workspace root or process.cwd() | the path where graphql config looks for config files |
263-
| `graphql-config.load.filePath` | `null` | exact filepath of the config file. |
264-
| `graphql-config.load.configName` | `graphql` | config name prefix instead of `graphql` |
265-
| `graphql-config.load.legacy` | `true` | backwards compatibility with `graphql-config@2` |
266-
| `graphql-config.dotEnvPath` | `null` | backwards compatibility with `graphql-config@2` |
267-
| `vscode-graphql.cacheSchemaFileForLookup` | `true` if `schema` contains non-sdl files or urls | generate an SDL file based on your graphql-config schema configuration for schema definition lookup and other features. enabled by default when your `schema` config are urls or introspection json, or if you have any non-local SDL files in `schema` |
261+
| Parameter | Default | Description |
262+
| ----------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
263+
| `graphql-config.load.baseDir` | workspace root or process.cwd() | the path where graphql config looks for config files |
264+
| `graphql-config.load.filePath` | `null` | exact filepath of the config file. |
265+
| `graphql-config.load.configName` | `graphql` | config name prefix instead of `graphql` |
266+
| `graphql-config.load.legacy` | `true` | backwards compatibility with `graphql-config@2` |
267+
| `graphql-config.dotEnvPath` | `null` | backwards compatibility with `graphql-config@2` |
268+
| `vscode-graphql.cacheSchemaFileForLookup` | `true` if `schema` contains non-SDL files or URLs | generate an SDL file based on your graphql-config schema configuration for definition lookup and other features. enabled by default when your `schema` config are URLs or introspection JSON, or if you have any non-local SDL files in `schema` |
269+
| `vscode-graphql.schemaCacheTTL` | `30000` | an integer value in milliseconds for the desired minimum cache lifetime for all schemas, which also causes the generated file to be re-written. set to 30s by default. effectively a "lazy" form of polling. if you are developing a schema alongside client queries, you may want to decrease this |
270+
| `vscode-graphql.debug` | `false` | show more verbose log output in the output channel |
268271

269272
all the `graphql-config.load.*` configuration values come from static
270273
`loadConfig()` options in graphql config.

‎packages/graphql-language-service-server/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@
6868
"@types/mkdirp": "^1.0.1",
6969
"@types/mock-fs": "^4.13.4",
7070
"cross-env": "^7.0.2",
71+
"debounce-promise": "^3.1.2",
7172
"graphql": "^16.8.1",
7273
"mock-fs": "^5.2.0"
7374
}

‎packages/graphql-language-service-server/src/GraphQLCache.ts

Lines changed: 20 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ import {
5555
} from './constants';
5656
import { NoopLogger, Logger } from './Logger';
5757
import { LRUCache } from 'lru-cache';
58+
// import { is } from '@babel/types';
5859

5960
const codeLoaderConfig: CodeFileLoaderConfig = {
6061
noSilentErrors: false,
@@ -83,12 +84,14 @@ export async function getGraphQLCache({
8384
loadConfigOptions,
8485
config,
8586
onSchemaChange,
87+
schemaCacheTTL,
8688
}: {
8789
parser: typeof parseDocument;
8890
logger: Logger | NoopLogger;
8991
loadConfigOptions: LoadConfigOptions;
9092
config?: GraphQLConfig;
9193
onSchemaChange?: OnSchemaChange;
94+
schemaCacheTTL?: number;
9295
}): Promise<GraphQLCache> {
9396
const graphQLConfig =
9497
config ||
@@ -105,6 +108,10 @@ export async function getGraphQLCache({
105108
parser,
106109
logger,
107110
onSchemaChange,
111+
schemaCacheTTL:
112+
schemaCacheTTL ??
113+
// @ts-expect-error TODO: add types for extension configs
114+
config?.extensions?.get('languageService')?.schemaCacheTTL,
108115
});
109116
}
110117

@@ -119,26 +126,29 @@ export class GraphQLCache {
119126
_parser: typeof parseDocument;
120127
_logger: Logger | NoopLogger;
121128
_onSchemaChange?: OnSchemaChange;
129+
_schemaCacheTTL?: number;
122130

123131
constructor({
124132
configDir,
125133
config,
126134
parser,
127135
logger,
128136
onSchemaChange,
137+
schemaCacheTTL,
129138
}: {
130139
configDir: Uri;
131140
config: GraphQLConfig;
132141
parser: typeof parseDocument;
133142
logger: Logger | NoopLogger;
134143
onSchemaChange?: OnSchemaChange;
144+
schemaCacheTTL?: number;
135145
}) {
136146
this._configDir = configDir;
137147
this._graphQLConfig = config;
138148
this._graphQLFileListCache = new Map();
139149
this._schemaMap = new LRUCache({
140150
max: 20,
141-
ttl: 1000 * 30,
151+
ttl: schemaCacheTTL ?? 1000 * 30,
142152
ttlAutopurge: true,
143153
updateAgeOnGet: false,
144154
});
@@ -602,10 +612,10 @@ export class GraphQLCache {
602612
}
603613

604614
getSchema = async (
605-
projectName: string,
615+
appName?: string,
606616
queryHasExtensions?: boolean | null,
607617
): Promise<GraphQLSchema | null> => {
608-
const projectConfig = this._graphQLConfig.getProject(projectName);
618+
const projectConfig = this._graphQLConfig.getProject(appName);
609619

610620
if (!projectConfig) {
611621
return null;
@@ -620,35 +630,18 @@ export class GraphQLCache {
620630
if (schemaPath && schemaKey) {
621631
schemaCacheKey = schemaKey as string;
622632

623-
try {
624-
// Read from disk
625-
schema = await projectConfig.loadSchema(
626-
projectConfig.schema,
627-
'GraphQLSchema',
628-
{
629-
assumeValid: true,
630-
assumeValidSDL: true,
631-
experimentalFragmentVariables: true,
632-
sort: false,
633-
includeSources: true,
634-
allowLegacySDLEmptyFields: true,
635-
allowLegacySDLImplementsInterfaces: true,
636-
},
637-
);
638-
} catch {
639-
// // if there is an error reading the schema, just use the last valid schema
640-
schema = this._schemaMap.get(schemaCacheKey);
641-
}
642-
633+
// Maybe use cache
643634
if (this._schemaMap.has(schemaCacheKey)) {
644635
schema = this._schemaMap.get(schemaCacheKey);
645-
646636
if (schema) {
647637
return queryHasExtensions
648638
? this._extendSchema(schema, schemaPath, schemaCacheKey)
649639
: schema;
650640
}
651641
}
642+
643+
// Read from disk
644+
schema = await projectConfig.getSchema();
652645
}
653646

654647
const customDirectives = projectConfig?.extensions?.customDirectives;
@@ -667,7 +660,9 @@ export class GraphQLCache {
667660

668661
if (schemaCacheKey) {
669662
this._schemaMap.set(schemaCacheKey, schema);
670-
await this._onSchemaChange?.(projectConfig);
663+
if (this._onSchemaChange) {
664+
this._onSchemaChange(projectConfig);
665+
}
671666
}
672667
return schema;
673668
};

‎packages/graphql-language-service-server/src/MessageProcessor.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ import { NoopLogger, Logger } from './Logger';
7575
import glob from 'fast-glob';
7676
import { isProjectSDLOnly, unwrapProjectSchema } from './common';
7777
import { DefinitionQueryResponse } from 'graphql-language-service/src/interface';
78+
import { default as debounce } from 'debounce-promise';
7879

7980
const configDocLink =
8081
'https://www.npmjs.com/package/graphql-language-service-server#user-content-graphql-configuration-file';
@@ -229,7 +230,7 @@ export class MessageProcessor {
229230
rootDir,
230231
};
231232

232-
const onSchemaChange = async (project: GraphQLProjectConfig) => {
233+
const onSchemaChange = debounce(async (project: GraphQLProjectConfig) => {
233234
const { cacheSchemaFileForLookup } =
234235
this.getCachedSchemaSettings(project);
235236
if (!cacheSchemaFileForLookup) {
@@ -241,7 +242,7 @@ export class MessageProcessor {
241242
return;
242243
}
243244
return this.cacheConfigSchemaFile(project);
244-
};
245+
}, 400);
245246

246247
try {
247248
// now we have the settings so we can re-build the logger
@@ -255,6 +256,7 @@ export class MessageProcessor {
255256
parser: this._parser,
256257
configDir: rootDir,
257258
onSchemaChange,
259+
schemaCacheTTL: this._settings?.schemaCacheTTL,
258260
});
259261
this._languageService = new GraphQLLanguageService(
260262
this._graphQLCache,
@@ -267,6 +269,7 @@ export class MessageProcessor {
267269
loadConfigOptions: this._loadConfigOptions,
268270
logger: this._logger,
269271
onSchemaChange,
272+
schemaCacheTTL: this._settings?.schemaCacheTTL,
270273
});
271274
this._languageService = new GraphQLLanguageService(
272275
this._graphQLCache,
@@ -1307,6 +1310,7 @@ export class MessageProcessor {
13071310
unwrapProjectSchema(project).map(async schema => {
13081311
const schemaFilePath = path.resolve(project.dirpath, schema);
13091312
const uriFilePath = URI.parse(uri).fsPath;
1313+
13101314
if (uriFilePath === schemaFilePath) {
13111315
try {
13121316
const file = await readFile(schemaFilePath, 'utf-8');

‎packages/graphql-language-service-server/src/__tests__/MessageProcessor.spec.ts

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,16 @@ import { serializeRange } from './__utils__/utils';
77
import { readFile } from 'node:fs/promises';
88
import { existsSync } from 'node:fs';
99
import { URI } from 'vscode-uri';
10-
import { GraphQLSchema, introspectionFromSchema } from 'graphql';
10+
import {
11+
GraphQLSchema,
12+
buildASTSchema,
13+
introspectionFromSchema,
14+
parse,
15+
} from 'graphql';
1116
import fetchMock from 'fetch-mock';
1217

18+
const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
19+
1320
jest.mock('@whatwg-node/fetch', () => {
1421
const { AbortController } = require('node-abort-controller');
1522

@@ -26,7 +33,7 @@ const mockSchema = (schema: GraphQLSchema) => {
2633
descriptions: true,
2734
}),
2835
};
29-
fetchMock.mock({
36+
return fetchMock.mock({
3037
matcher: '*',
3138
response: {
3239
headers: {
@@ -523,17 +530,37 @@ describe('MessageProcessor with config', () => {
523530
],
524531
schemaFile,
525532
],
533+
settings: { schemaCacheTTL: 500 },
526534
});
527535

528536
const initParams = await project.init('a/query.ts');
529537
expect(initParams.diagnostics[0].message).toEqual('Unknown fragment "T".');
530538

531539
expect(project.lsp._logger.error).not.toHaveBeenCalled();
532540
expect(await project.lsp._graphQLCache.getSchema('a')).toBeDefined();
541+
542+
fetchMock.restore();
543+
mockSchema(
544+
buildASTSchema(
545+
parse(
546+
'type example100 { string: String } type Query { example: example100 }',
547+
),
548+
),
549+
);
550+
await project.lsp.handleWatchedFilesChangedNotification({
551+
changes: [
552+
{ uri: project.uri('a/fragments.ts'), type: FileChangeType.Changed },
553+
],
554+
});
555+
await sleep(1000);
556+
expect(
557+
(await project.lsp._graphQLCache.getSchema('a')).getType('example100'),
558+
).toBeTruthy();
559+
await sleep(1000);
533560
const file = await readFile(join(genSchemaPath.replace('default', 'a')), {
534561
encoding: 'utf-8',
535562
});
536-
expect(file.split('\n').length).toBeGreaterThan(10);
563+
expect(file).toContain('example100');
537564
// add a new typescript file with empty query to the b project
538565
// and expect autocomplete to only show options for project b
539566
await project.addFile(

‎packages/graphql-language-service-server/src/__tests__/__utils__/MockProject.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ export class MockProject {
5757
}: {
5858
files: Files;
5959
root?: string;
60-
settings?: [name: string, vale: any][];
60+
settings?: Record<string, any>;
6161
}) {
6262
this.root = root;
6363
this.fileCache = new Map(files);
@@ -100,7 +100,11 @@ export class MockProject {
100100
});
101101
}
102102
private mockFiles() {
103-
const mockFiles = { ...defaultMocks };
103+
const mockFiles = {
104+
...defaultMocks,
105+
// without this, the generated schema file may not be cleaned up by previous tests
106+
'/tmp/graphql-language-service': mockfs.directory(),
107+
};
104108
for (const [filename, text] of this.fileCache) {
105109
mockFiles[this.filePath(filename)] = text;
106110
}

‎packages/graphql-language-service/package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,11 +35,13 @@
3535
"graphql": "^15.5.0 || ^16.0.0"
3636
},
3737
"dependencies": {
38+
"debounce-promise": "^3.1.2",
3839
"nullthrows": "^1.0.0",
3940
"vscode-languageserver-types": "^3.17.1"
4041
},
4142
"devDependencies": {
4243
"@types/benchmark": "^1.0.33",
44+
"@types/debounce-promise": "^3.1.9",
4345
"@types/json-schema": "7.0.9",
4446
"@types/picomatch": "^2.3.0",
4547
"benchmark": "^2.1.4",

‎packages/vscode-graphql-syntax/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@
155155
"vscode-oniguruma": "^1.7.0",
156156
"vscode-textmate": "^9.0.0",
157157
"ovsx": "^0.3.0",
158-
"@vscode/vsce": "^2.23.0"
158+
"@vscode/vsce": "^2.22.1-2"
159159
},
160160
"homepage": "https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/README.md",
161161
"scripts": {

0 commit comments

Comments
 (0)