Correct JSON schema output #139

bennypowers · 2025-07-15T05:20:05Z

Closes Sum type ambiguities lead to incorrect or unclear validations #138 by swapping out the tool which generates JSON schema from Typescript
Prepares release v2.1.1

bennypowers · 2025-07-16T07:45:29Z

This is failing to validate known valid manifests:

❯ ajv --strict=false \
      -s ~/.cache/cem/schemas/2.1.1.json \
      -d ~/Developer/redhat-ux/red-hat-design-system/custom-elements.json
/var/home/bennyp/Developer/redhat-ux/red-hat-design-system/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/1',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'superclass' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'superclass' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1/members/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'attribute' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1/members/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'type' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/1/members/0',
    schemaPath: '#/anyOf',
    keyword: 'anyOf',
    params: {},
    message: 'must match a schema in anyOf'
  },
  {
    instancePath: '/modules/0/declarations/1/kind',
    schemaPath: '#/properties/kind/const',
    keyword: 'const',
    params: { allowedValue: 'mixin' },
    message: 'must be equal to constant'
  },
  {
    instancePath: '/modules/0/declarations/1',
    schemaPath: '#/anyOf',
    keyword: 'anyOf',
    params: {},
    message: 'must match a schema in anyOf'
  }
]

essentially, it's failing to validate CustomElementDeclarations in any circumstances

bennypowers · 2025-07-18T14:04:11Z

For future reference here's the fish script i'm using to test. i have to be on cem branch feat/verify from bennypowers/cem#53

npm run build:schema
for s in valid-manifest invalid-attribute
      set TEST_SCHEMA ~/Developer/cem/cmd/test/fixture/$s/custom-elements.json
      cat $TEST_SCHEMA
      ajv --strict=false \
                    -s ~/Projects/custom-elements-manifest/schema.json \
                    -d $TEST_SCHEMA
end

bennypowers · 2025-07-21T08:55:48Z

Ok so with some mild schema.d.ts changes, and the @discriminator jsdoc tag we can sssssorta rough this one in:

❯ for s in valid-manifest invalid-attribute
        set TEST_SCHEMA ~/Developer/cem/cmd/fixture/$s/custom-elements.json
        ajv --strict=false \
                      -s ~/Projects/custom-elements-manifest/schema.json \
                      -d $TEST_SCHEMA
  end
/var/home/bennyp/Developer/cem/cmd/fixture/valid-manifest/custom-elements.json valid
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-attribute/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/0/attributes/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/attributes/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/anyOf',
    keyword: 'anyOf',
    params: {},
    message: 'must match a schema in anyOf'
  }
]

with more aggressive if/then/else (maybe through a postprocessing step) we could do better: there really should only be one validation error here (attribute name missing), not several (also, it's not a function decl, etc)

bennypowers · 2025-07-27T14:41:07Z

I've patched ts-json-schema-generator in this PR to produce the kinds of output we want:

❯ for s in (ls ~/Developer/cem/cmd/fixture)
        set TEST_SCHEMA ~/Developer/cem/cmd/fixture/$s/custom-elements.json
      if test -e $TEST_SCHEMA
            ajv --strict=false \
                          -s ~/Projects/custom-elements-manifest/schema.json \
                          -d $TEST_SCHEMA
      end
  end
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-attribute/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/attributes/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-class/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/4/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-css-custom-property/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/cssProperties/0',
    schemaPath: '#/definitions/CssCustomProperty/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-css-part/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/cssParts/0',
    schemaPath: '#/definitions/CssPart/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-custom-element/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'tagName' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/4/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-event/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/events/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-field/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/properties/members/items/anyOf',
    keyword: 'anyOf',
    params: {},
    message: 'must match a schema in anyOf'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-function/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/0/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-method/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0/members/0',
    schemaPath: '#/properties/members/items/anyOf',
    keyword: 'anyOf',
    params: {},
    message: 'must match a schema in anyOf'
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-mixin/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/2/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-slot/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/slots/0',
    schemaPath: '#/definitions/Slot/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/invalid-variable/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/1/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/list-project/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0/slots/0',
    schemaPath: '#/definitions/Slot/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/5/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/unsupported-schema/custom-elements.json invalid
[
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/required',
    keyword: 'required',
    params: { missingProperty: 'name' },
    message: "must have required property 'name'"
  },
  {
    instancePath: '/modules/0/declarations/0',
    schemaPath: '#/allOf/1/if',
    keyword: 'if',
    params: { failingKeyword: 'then' },
    message: 'must match "then" schema'
  }
]
/var/home/bennyp/Developer/cem/cmd/fixture/valid-manifest/custom-elements.json valid

Use ts-json-schema-generator

98c7816

Closes webcomponents#138

bennypowers marked this pull request as draft July 16, 2025 07:44

justinfagnani self-requested a review July 16, 2025 15:30

Use discriminator jsdoc hint

7b9265b

bennypowers force-pushed the ts-json-schema-generator branch from 828cfdd to 7b9265b Compare July 21, 2025 08:51

chore: nvm

eb57e57

chore: update deps

2854edd

bennypowers marked this pull request as ready for review July 21, 2025 09:01

bennypowers added 3 commits July 27, 2025 17:06

Nest discriminators

c2f6432

chore: patch ts-json-schema-generator

8101972

Discriminate ClassMembers in CustomElement concrete types

7c827e6

bennypowers added 2 commits July 27, 2025 18:57

chore: patch-package

fb14944

chore: patch

fc4c955

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Correct JSON schema output #139

Correct JSON schema output #139

Uh oh!

bennypowers commented Jul 15, 2025 •

edited

Loading

Uh oh!

bennypowers commented Jul 16, 2025

Uh oh!

bennypowers commented Jul 18, 2025

Uh oh!

bennypowers commented Jul 21, 2025

Uh oh!

bennypowers commented Jul 27, 2025

Uh oh!

Uh oh!

Correct JSON schema output #139

Are you sure you want to change the base?

Correct JSON schema output #139

Uh oh!

Conversation

bennypowers commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bennypowers commented Jul 16, 2025

Uh oh!

bennypowers commented Jul 18, 2025

Uh oh!

bennypowers commented Jul 21, 2025

Uh oh!

bennypowers commented Jul 27, 2025

Uh oh!

Uh oh!

bennypowers commented Jul 15, 2025 •

edited

Loading