enhance(build): validate built `data.json` against schemas #27685

caugner · 2025-08-27T09:22:47Z

Summary

Validates the built data.json against the schemas.
Removes the maxProperties requirement for the browsers object from the browsers schema.

Test results and supporting details

This revealed two minor issues:

The browsers schema was requiring a maximum of one browser defined in browsers. This requirement only applies to the individual browser/*.json files, not the built data.json, so this requirement was removed.
The __meta object in the built data.json is not currently covered by the schema, so we cannot validate it.

Related issues

Fixes #27564.

github-actions · 2025-08-27T09:23:24Z

Tip: Review these changes grouped by change (recommended for most PRs), or grouped by feature (for large PRs).

caugner · 2025-08-27T09:31:10Z

scripts/build/index.ts

 const writeData = async () => {
  const dest = new URL('data.json', targetdir);
  const data = await createDataBundle();
+  validate(data);


Note
If validation fails, the validation errors are logged via console.error, but we don't bail out, to avoid disruption for contributors in the unlikely event that this ever happens on main. In either case, these errors won't get unnoticed, because they will appear on every npm install (via the prepare script).

Does this mean that a PR that breaks schema validation won't fail the build? I can kinda see the case for never failing during the prepare script, but it'd be nice if there were a test that does fail. I won't block on this though, as it would be a step up from what we have.

How about checking process.env.CI and failing if we're in CI?

That would help for local development, but not pull request checks. Maybe check if GITHUB_REF is refs/heads/main and, if true, convert errors to warnings, else do nothing?

In pull request checks, we always have CI=true in the environment, so we would notice it in a PR before merging. (In fact, the PR check would fail, preventing a merge.)

Let's tackle this afterwards.

ddbeck · 2025-08-28T08:45:31Z

The __meta object in the built data.json is not currently covered by the schema, so we cannot validate it.

FWIW, I raised this as a concern in #17574 and got shot down on doing anything about it.

caugner · 2025-08-28T15:17:31Z

FWIW, I raised this as a concern in #17574 and got shot down on doing anything about it.

FWIW, here's the meta schema (isolated, like the other two):

{
  "$schema": "http://json-schema.org/schema",

  "definitions": {
    "meta_block": {
      "type": "object",
      "properties": {
        "timestamp": {
          "type": "string",
          "format": "date-time"
        },
        "version": {
          "type": "string"
        }
      },
      "required": ["timestamp", "version"],
      "additionalProperties": false
    }
  },

  "properties": {
    "__meta": {
      "$ref": "#/definitions/meta_block"
    }
  },

  "title": "MetaData",
  "type": "object",
  "required": ["__meta"],
  "additionalProperties": false,
  "maxProperties": 1,
  "minProperties": 1
}

Unfortunately, ajv gives me an error when validating the data:

TYPE must be string

  2 |   "__meta": {
  3 |     "version": "7.1.0",
> 4 |     "timestamp": "2025-08-28T14:30:46.924Z"
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^ 👈🏽  type must be string
  5 |   }
  6 | }

That error doesn't make sense, but given this is not a priority, I would declare the scope of this PR to validate the built data against the existing schemas.

utils/ajv.ts

schemas/browsers.schema.json

scripts/build/index.ts

The built data.json contains multiple browsers.

caugner · 2025-10-17T11:43:07Z

@queengooborg If we can get this in, we can split up the schema into internal/public, and validate separately. Currently, it's a mix of both.

Elchi3

Approving the schema change 👍

ddbeck

I've reviewed the code. Thank you!

ddbeck · 2025-11-04T11:06:58Z

scripts/build/index.ts

+
+  for (const [key, value] of Object.entries(data)) {
+    if (key === '__meta') {
+      // Not covered by the schema.


See also: #27685 (comment)

ddbeck · 2025-11-04T11:10:26Z

scripts/build/index.ts

 const writeData = async () => {
  const dest = new URL('data.json', targetdir);
  const data = await createDataBundle();
+  validate(data);


Does this mean that a PR that breaks schema validation won't fail the build? I can kinda see the case for never failing during the prepare script, but it'd be nice if there were a test that does fail. I won't block on this though, as it would be a step up from what we have.

@argl

Approved by @argl

caugner changed the title ~~enhance(build): validate data against schemas~~ enhance(build): validate built data.json against schemas Aug 27, 2025

caugner requested review from Elchi3 and queengooborg August 27, 2025 09:23

github-actions bot added the size:m [PR only] 25-100 LoC changed label Aug 27, 2025

caugner commented Aug 27, 2025

View reviewed changes

caugner mentioned this pull request Aug 27, 2025

Schema forbids booleans for version_removed, but data contains false values #27437

Closed

queengooborg previously requested changes Aug 29, 2025

View reviewed changes

utils/ajv.ts Show resolved Hide resolved

schemas/browsers.schema.json Show resolved Hide resolved

caugner requested a review from a team as a code owner September 16, 2025 13:38

caugner requested review from queengooborg and removed request for Elchi3 September 16, 2025 13:43

caugner commented Oct 7, 2025

View reviewed changes

scripts/build/index.ts Show resolved Hide resolved

caugner force-pushed the 27564-validate-data-against-schema branch from b884975 to c046688 Compare October 7, 2025 12:32

github-actions bot added data:api Compat data for Web APIs. https://developer.mozilla.org/docs/Web/API size:l [PR only] 101-1000 LoC changed and removed size:m [PR only] 25-100 LoC changed labels Oct 7, 2025

caugner added 4 commits October 14, 2025 15:10

refactor(lint): extract createAjv() to utils

05e52bf

enhance(build): validate data against schemas

81d7b1d

fix(schemas/browsers): remove maxProperties constraint

fae5c18

The built data.json contains multiple browsers.

refactor(utils): move createAjv() to scripts/lib

fabb4f2

caugner force-pushed the 27564-validate-data-against-schema branch from c046688 to fabb4f2 Compare October 14, 2025 13:11

github-actions bot added size:m [PR only] 25-100 LoC changed and removed data:api Compat data for Web APIs. https://developer.mozilla.org/docs/Web/API size:l [PR only] 101-1000 LoC changed labels Oct 14, 2025

Merge branch 'main' into 27564-validate-data-against-schema

019e2e6

Merge branch 'main' into 27564-validate-data-against-schema

5622a61

Elchi3 approved these changes Nov 4, 2025

View reviewed changes

Merge branch 'main' into 27564-validate-data-against-schema

c0c83e3

caugner requested a review from a team as a code owner November 4, 2025 11:05

caugner requested a review from argl November 4, 2025 11:05

ddbeck approved these changes Nov 4, 2025

View reviewed changes

Merge branch 'main' into 27564-validate-data-against-schema

60b2389

argl approved these changes Nov 5, 2025

View reviewed changes

caugner merged commit 0dde3fe into main Nov 5, 2025
12 checks passed

caugner deleted the 27564-validate-data-against-schema branch November 5, 2025 13:49

enhance(build): validate built data.json against schemas #27685

enhance(build): validate built data.json against schemas #27685

Uh oh!

Conversation

caugner commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test results and supporting details

Related issues

Uh oh!

github-actions bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

caugner Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ddbeck commented Aug 28, 2025

Uh oh!

caugner commented Aug 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

caugner commented Oct 17, 2025

Uh oh!

Elchi3 left a comment

Choose a reason for hiding this comment

Uh oh!

ddbeck left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

enhance(build): validate built `data.json` against schemas #27685

enhance(build): validate built `data.json` against schemas #27685

caugner commented Aug 27, 2025 •

edited

Loading

github-actions bot commented Aug 27, 2025 •

edited

Loading

caugner Aug 27, 2025 •

edited

Loading

ddbeck left a comment •

edited

Loading