Handle user errors for invalid `UserConfig`s and missing query files #3203

mbg · 2025-10-12T12:30:50Z

Classifies two additional types of errors as ConfigurationErrors:

Invalid config files
Missing query files

Best reviewed commit-by-commit.

Risk assessment

For internal use only. Please select the risk level of this change:

Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Advanced setup - Impacts users who have custom workflows.
Default setup - Impacts users who use default setup.
Code Scanning - Impacts Code Scanning (i.e. analysis-kinds: code-scanning).
Code Quality - Impacts Code Quality (i.e. analysis-kinds: code-quality).
GHES - Impacts GitHub Enterprise Server.

How did/will you validate this change?

Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).
End-to-end tests - I am depending on PR checks (i.e. tests in pr-checks).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

Rollback - Change can only be disabled by rolling back the release or releasing a new version with a fix.

How will you know if something goes wrong after this change is released?

Telemetry - I rely on existing telemetry or have made changes to the telemetry.
- Dashboards - I will watch relevant dashboards for issues after the release. Consider whether this requires this change to be released at a particular time rather than as part of a regular release.
- Alerts - New or existing monitors will trip if something goes wrong with this change.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Consider adding a changelog entry for this change.
Confirm the readme and docs have been updated if necessary.

- Throws a `ConfigurationError` if parsing the YAML fails - Add a couple of tests for it

Copilot

Pull Request Overview

This PR improves error handling for CodeQL configuration by classifying invalid config files and missing query files as ConfigurationErrors. The changes introduce JSON schema validation for config files and better error classification to provide users with clearer feedback when their configurations are malformed.

Adds JSON schema validation for user config files with structured error reporting
Expands CLI error classification to include missing query file scenarios
Updates type definitions to better handle query input formats

Reviewed Changes

Copilot reviewed 17 out of 19 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/error-messages.ts`	Adds new error message functions for config file parsing and validation errors
`src/db-config-schema.json`	Defines comprehensive JSON schema for CodeQL database configuration validation
`src/config/db-config.ts`	Implements `parseUserConfig` function with JSON schema validation and updates type definitions
`src/config/db-config.test.ts`	Adds tests for the new `parseUserConfig` function covering valid YAML and error cases
`src/config-utils.ts`	Updates config loading to use the new `parseUserConfig` function for better error handling
`src/cli-errors.ts`	Expands error pattern matching to include missing query file scenarios
`src/cli-errors.test.ts`	Adds test case for unknown query file error classification
`src/status-report.ts`	Updates query rendering to use new `renderQueryInput` helper function
`lib/*`	Generated JavaScript files reflecting the TypeScript changes

src/error-messages.ts

Copilot · 2025-10-12T12:31:51Z

src/config/db-config.ts

+        ),
+      );
+    }
+


The function should validate that the parsed document conforms to the UserConfig interface before casting. Consider adding runtime type checking or at least documenting the assumption that schema validation guarantees type safety.

Suggested change

// Assumption: The JSON schema used above is kept in sync with the UserConfig TypeScript interface.

// As long as schema validation passes, it is safe to cast to UserConfig.

Co-authored-by: Copilot <[email protected]>

henrymercer · 2025-10-13T09:56:42Z

src/config/db-config.ts

+      throw new ConfigurationError(
+        errorMessages.getInvalidConfigFileMessage(
+          pathInput,
+          `The configuration file contained ${result.errors.length} error(s)`,
+        ),
+      );


Could we list the errors?

henrymercer · 2025-10-13T11:11:54Z

src/config/db-config.ts

  name?: string;
  "disable-default-queries"?: boolean;
-  queries?: QuerySpec[];
+  queries?: QueryInput[];


The config file only accepts query specs. I double checked this here: https://github.com/github/codeql-action/actions/runs/18463233951

henrymercer · 2025-10-13T11:13:17Z

src/db-config-schema.json

+      "type": "array",
+      "description": "List of additional queries to run",
+      "items": {
+        "$ref": "#/definitions/QueryInput"


This should be just QuerySpec

henrymercer · 2025-10-13T11:13:40Z

src/db-config-schema.json

+    },
+    "paths-ignore": {
+      "type": "array",
+      "description": "Paths to ignore during analysis",


Suggested change

"description": "Paths to ignore during analysis",

"description": "Paths to exclude from analysis",

henrymercer · 2025-10-13T11:18:51Z

src/db-config-schema.json

+      "description": "Query filter that can either include or exclude queries",
+      "oneOf": [
+        {
+          "$ref": "#/definitions/ExcludeQueryFilter"
+        },
+        {
+          "$ref": "#/definitions/IncludeQueryFilter"
+        }
+      ]


It's possible to specify neither include nor exclude, and it's allowable, although unusual, to specify both an include and an exclude. I think we should just specify that the include and exclude properties are both optional.

mbg added 5 commits October 12, 2025 12:50

Fix type of queries in UserConfig to account for string values

f8153bb

Add and use parseUserConfig

b4b96a0

- Throws a `ConfigurationError` if parsing the YAML fails - Add a couple of tests for it

Add and validate UserConfig schema

60d9fe5

Add additional regex to CliConfigErrorCategory.PackCannotBeFound

15329e0

Add type annotation for return type of parseQueriesFromInput

71ddb40

mbg requested a review from henrymercer October 12, 2025 12:30

mbg self-assigned this Oct 12, 2025

mbg requested a review from a team as a code owner October 12, 2025 12:30

Copilot AI review requested due to automatic review settings October 12, 2025 12:30

Copilot AI reviewed Oct 12, 2025

View reviewed changes

Update src/error-messages.ts

c71b45c

Co-authored-by: Copilot <[email protected]>

mbg force-pushed the mbg/errors/more-user-errors branch from 6aa2711 to c71b45c Compare October 12, 2025 12:35

henrymercer reviewed Oct 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Handle user errors for invalid `UserConfig`s and missing query files #3203

Handle user errors for invalid `UserConfig`s and missing query files #3203

Uh oh!

mbg commented Oct 12, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

henrymercer Oct 13, 2025

Uh oh!

henrymercer Oct 13, 2025

Uh oh!

henrymercer Oct 13, 2025

Uh oh!

henrymercer Oct 13, 2025

Uh oh!

henrymercer Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants



	// Assumption: The JSON schema used above is kept in sync with the UserConfig TypeScript interface.
	// As long as schema validation passes, it is safe to cast to UserConfig.

	"description": "Paths to ignore during analysis",
	"description": "Paths to exclude from analysis",

Handle user errors for invalid UserConfigs and missing query files #3203

Are you sure you want to change the base?

Handle user errors for invalid UserConfigs and missing query files #3203

Uh oh!

Conversation

mbg commented Oct 12, 2025

Risk assessment

Which use cases does this change impact?

How did/will you validate this change?

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

How will you know if something goes wrong after this change is released?

Merge / deployment checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Handle user errors for invalid `UserConfig`s and missing query files #3203

Handle user errors for invalid `UserConfig`s and missing query files #3203