Skip to content

Commit 650cb66

Browse files
dingo-djrfnl
dingo-d
and
jrfnl
committed
Introduce a XSD schema for validation PHPCS XML docs (#80)
* Introduces an XSD schema file to allow for validating PHPCS XML docs files, i.e. the `Standard/Docs/Category/*Standard.xml` files. * Adds this file to the website build script so the file will also be available at a canonical URL via the website. * Adds validation of the XSD file against the schema for XSD to the CI/GH Actions workflows. * Adds validation of the PHPCSDebug XML docs against the new XSD schema file to the CI/GH Actions workflows. * Adds documentation about the XSD schema and how to use it to the README. * Adds extensive tests for the XSD schema to the integration test suite. * Adds a reference to the schema to the existing XML doc for the PHPCSDebug standard. Notes: * The XSD allows for multiple sets of `<standard>` elements, with each optionally having one or more `<code_comparison>`s. * The XSD allows for additional arbitrary attributes on select elements in the XML files. These use `lax` contents processing, as without it, the attributes must be explicitly defined in the XSD, and we don't know which attribute(s) some standard may want to add to their documentation. * The XSD has also been real-world tested against XML documentation files from PHPCS itself, WordPressCS, PHPCSExtra and more. Those all pass, except for a few were the XML doc actually contained errors, which confirms that the XSD will work in real-life situations. Co-authored-by: Juliette <[email protected]>
1 parent aec8d0f commit 650cb66

35 files changed

+942
-2
lines changed

‎.github/build/Website.php‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,8 @@ final class Website
5353
* @var array<string => string target>
5454
*/
5555
const FILES_TO_COPY = [
56-
'README.md' => 'index.md',
56+
'README.md' => 'index.md',
57+
'DocsXsd/phpcsdocs.xsd' => 'phpcsdocs.xsd',
5758
];
5859

5960
/**

‎.github/workflows/cs.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,11 @@ jobs:
5959
# @link https://github.com/marketplace/actions/xmllint-problem-matcher
6060
- uses: korelstar/xmllint-problem-matcher@v1
6161

62+
# Validate the Docs XSD file.
63+
# @link http://xmlsoft.org/xmllint.html
64+
- name: Validate Docs XSD against schema
65+
run: xmllint --noout --schema http://www.w3.org/2001/XMLSchema.xsd DocsXsd/phpcsdocs.xsd
66+
6267
# Validate the XML file.
6368
# @link http://xmlsoft.org/xmllint.html
6469
- name: Validate ruleset against schema
@@ -68,6 +73,11 @@ jobs:
6873
- name: Check XML code style
6974
run: diff -B ./PHPCSDebug/ruleset.xml <(xmllint --format "./PHPCSDebug/ruleset.xml")
7075

76+
# Validate the Docs XML file(s).
77+
# @link http://xmlsoft.org/xmllint.html
78+
- name: Validate docs against schema
79+
run: xmllint --noout --schema DocsXsd/phpcsdocs.xsd ./PHPCSDebug/Docs/*/*Standard.xml
80+
7181
# Check the code-style consistency of the PHP files.
7282
- name: Check PHP code style
7383
continue-on-error: true

‎.github/workflows/quicktest.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@ jobs:
3838
- name: Checkout code
3939
uses: actions/checkout@v3
4040

41+
- name: Install xmllint
42+
run: |
43+
sudo apt-get update
44+
sudo apt-get install --no-install-recommends -y libxml2-utils
45+
4146
# On stable PHPCS versions, allow for PHP deprecation notices.
4247
# Unit tests don't need to fail on those for stable releases where those issues won't get fixed anymore.
4348
- name: Setup ini config

‎.github/workflows/test.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,11 @@ jobs:
7070
- name: Checkout code
7171
uses: actions/checkout@v3
7272

73+
- name: Install xmllint
74+
run: |
75+
sudo apt-get update
76+
sudo apt-get install --no-install-recommends -y libxml2-utils
77+
7378
- name: Setup ini config
7479
id: set_ini
7580
run: |

‎DocsXsd/phpcsdocs.xsd‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
3+
<xs:element name="documentation">
4+
<xs:complexType>
5+
<xs:sequence maxOccurs="unbounded">
6+
<xs:group ref="rulegroup"/>
7+
</xs:sequence>
8+
<xs:attribute name="title" use="required" type="titleType"/>
9+
<xs:anyAttribute processContents="lax"/>
10+
</xs:complexType>
11+
</xs:element>
12+
13+
<xs:group name="rulegroup">
14+
<xs:sequence>
15+
<xs:element name="standard" type="standardType"/>
16+
<xs:element name="code_comparison" type="code_comparisonType" maxOccurs="unbounded" minOccurs="0"/>
17+
<xs:any minOccurs="0"/>
18+
</xs:sequence>
19+
</xs:group>
20+
21+
<xs:simpleType name="titleType">
22+
<xs:restriction base="xs:string">
23+
<xs:minLength value="1"/>
24+
<xs:maxLength value="58"/>
25+
</xs:restriction>
26+
</xs:simpleType>
27+
28+
<xs:complexType name="code_comparisonType">
29+
<xs:sequence>
30+
<xs:element name="code" type="codeType" maxOccurs="2" minOccurs="2"/>
31+
</xs:sequence>
32+
</xs:complexType>
33+
34+
<xs:complexType name="codeType">
35+
<xs:simpleContent>
36+
<xs:extension base="xs:string">
37+
<xs:attribute name="title" use="required" type="codeTitleType"/>
38+
</xs:extension>
39+
</xs:simpleContent>
40+
</xs:complexType>
41+
42+
<xs:complexType name="standardType">
43+
<xs:simpleContent>
44+
<xs:extension base="xs:string">
45+
<xs:anyAttribute processContents="lax"/>
46+
</xs:extension>
47+
</xs:simpleContent>
48+
</xs:complexType>
49+
50+
<xs:simpleType name="codeTitleType">
51+
<xs:restriction base="xs:string">
52+
<xs:minLength value="1"/>
53+
</xs:restriction>
54+
</xs:simpleType>
55+
</xs:schema>

‎PHPCSDebug/Docs/Debug/TokenListStandard.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<documentation title="Token List">
1+
<documentation xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://phpcsstandards.github.io/PHPCSDevTools/phpcsdocs.xsd" title="Token List">
22
<standard>
33
<![CDATA[
44
Lists how PHPCS tokenizes code.

‎README.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ This is a set of tools to assist developers of sniffs for [PHP CodeSniffer](http
2929
* [Features](#features)
3030
+ [Checking whether all sniffs in a PHPCS standard are feature complete](#checking-whether-all-sniffs-in-a-phpcs-standard-are-feature-complete)
3131
+ [Sniff Debugging](#sniff-debugging)
32+
+ [Documentation XSD Validation](#documentation-xsd-validation)
3233
* [Contributing](#contributing)
3334
* [License](#license)
3435

@@ -175,6 +176,48 @@ Ptr | Ln | Col | Cond | ( #) | Token Type | [len]: Content
175176

176177
PHPCS itself can also display similar information using the `-vv` or `-vvv` verbosity flags, however, when using those, you will receive a *lot* more information than just the token list and, while useful for debugging PHPCS itself, the additional information is mostly just noise when developing a sniff.
177178

179+
### Documentation XSD Validation
180+
181+
This project contains an [XML Schema Definition (XSD)](https://www.w3.org/standards/xml/schema) to allow for validation PHPCS documentation XML files. Following the XSD will make sure your documentation can be correctly displayed when using the PHPCS `--generator` option.
182+
183+
In order to use it, you'll need to add the schema related attributes to the `documentation` element of the sniff documentation file, like so:
184+
185+
```xml
186+
<documentation
187+
title="Name of the sniff"
188+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
189+
xsi:noNamespaceSchemaLocation="https://phpcsstandards.github.io/PHPCSDevTools/phpcsdocs.xsd"
190+
>
191+
```
192+
193+
If your IDE or editor supports automatic validation of XML files, you will be notified if your documentation XML file has the correct number of elements, correct type and number of certain attributes, and title length among other things.
194+
195+
#### Validating your docs against the XSD
196+
197+
You can validate your PHPCS XML documentation against the XSD file using [xmllint](https://gnome.pages.gitlab.gnome.org/libxml2/xmllint.html). This validation can be run locally if you have xmllint installed, as well as in CI (continuous integration).
198+
199+
An example of a workflow job for GitHub Actions CI looks like this:
200+
201+
```yaml
202+
jobs:
203+
validate-xml:
204+
runs-on: ubuntu-latest
205+
steps:
206+
- uses: actions/checkout@v3
207+
208+
- name: Install xmllint
209+
run: |
210+
sudo apt-get update
211+
sudo apt-get install --no-install-recommends -y libxml2-utils
212+
213+
# A Composer install is needed to have a local copy of the XSD available.
214+
- run: composer install
215+
216+
- name: Validate docs against schema
217+
run: xmllint --noout --schema vendor/phpcsstandards/phpcsdevtools/DocsXsd/phpcsdocs.xsd ./YourRuleset/Docs/**/*Standard.xml
218+
```
219+
220+
You'd need to replace the `YourRuleset` with the name of your ruleset of course.
178221

179222
Contributing
180223
-------

‎Tests/DocsXsd/DocsXsdTest.php‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
<?php
2+
/**
3+
* PHPCSDevTools, tools for PHP_CodeSniffer sniff developers.
4+
*
5+
* @package PHPCSDevTools
6+
* @copyright 2019 PHPCSDevTools Contributors
7+
* @license https://opensource.org/licenses/LGPL-3.0 LGPL3
8+
* @link https://github.com/PHPCSStandards/PHPCSDevTools
9+
*/
10+
11+
namespace PHPCSDevTools\Tests\DocsXsd;
12+
13+
use PHPCSDevTools\Tests\IOTestCase;
14+
15+
/**
16+
* Test the Docs XSD feature.
17+
*
18+
* @coversNothing
19+
*
20+
* @phpcs:disable Squiz.Arrays.ArrayDeclaration.DoubleArrowNotAligned -- If needed, fix once replaced by better sniff.
21+
*/
22+
final class DocsXsdTest extends IOTestCase
23+
{
24+
25+
/**
26+
* Command that is being run during the test
27+
*
28+
* @var string
29+
*/
30+
const COMMAND = 'xmllint --noout --schema DocsXsd/phpcsdocs.xsd Tests/Fixtures/DocsXsd/%s';
31+
32+
/**
33+
* Verify that the valid XSD doesn't throw errors
34+
*
35+
* Successful xmllint command will have the exit code 0, and contain the 'validates'
36+
* message in the stderr part of the result.
37+
*
38+
* @dataProvider dataValidXsd
39+
*
40+
* @param string $fixtureFile The name of the fixture file in the fixture directory.
41+
*
42+
* @return void
43+
*/
44+
public function testXsdValidationPassedWithValidXml($fixtureFile)
45+
{
46+
$command = \sprintf(self::COMMAND, $fixtureFile);
47+
$result = $this->executeCliCommand($command);
48+
49+
$this->assertSame('', $result['stdout'], 'Unexpected output in stdout');
50+
$this->assertStringContainsString(
51+
"Tests/Fixtures/DocsXsd/{$fixtureFile} validates",
52+
$result['stderr'],
53+
'Unexpected output in stderr'
54+
);
55+
$this->assertSame(0, $result['exitcode'], 'Exit code does not match 0');
56+
}
57+
58+
/**
59+
* Data provider for valid test cases.
60+
*
61+
* @return array
62+
*/
63+
public function dataValidXsd()
64+
{
65+
return [
66+
'Valid docs example with single standard in the file' => [
67+
'fixtureFile' => 'ValidSingleStandard.xml',
68+
],
69+
'Valid docs example with multiple standards in the file' => [
70+
'fixtureFile' => 'ValidMultipleStandard.xml',
71+
],
72+
'Valid docs example with multiple arbitrary attributes on the <documentation> element' => [
73+
'fixtureFile' => 'ValidDocumentationWithAdditionalAttributes.xml',
74+
],
75+
'Valid docs example with multiple arbitrary attributes on the <standard> element' => [
76+
'fixtureFile' => 'ValidDocumentationWithAdditionalAttributesOnStandardElement.xml',
77+
],
78+
'Valid docs example with multiple code examples' => [
79+
'fixtureFile' => 'ValidMultipleCodeExamples.xml',
80+
],
81+
'Valid docs example without code comparison element in the sequence group' => [
82+
'fixtureFile' => 'ValidEmptyCodeComparisonElement.xml',
83+
],
84+
];
85+
}
86+
87+
/**
88+
* Verify that an invalid PHPCS docs XML file will throw the correct validation errors
89+
*
90+
* @dataProvider dataInvalidXsd
91+
*
92+
* @param string $fixtureFile The name of the fixture file in the fixture directory.
93+
* @param string $expectedStdOut Expected output from the xmllint command.
94+
* @param string $expectedStdErr Expected validation error from the xmllint command.
95+
*
96+
* @return void
97+
*/
98+
public function testXsdValidationFailsForInvalidXml($fixtureFile, $expectedStdOut, $expectedStdErr)
99+
{
100+
$command = \sprintf(self::COMMAND, $fixtureFile);
101+
$result = $this->executeCliCommand($command);
102+
103+
$this->assertSame($expectedStdOut, $result['stdout'], 'Unexpected output in stdout');
104+
$this->assertStringContainsString($expectedStdErr, $result['stderr'], 'Unexpected output in stderr');
105+
$this->assertGreaterThan(0, $result['exitcode'], 'Exit code does not match 0');
106+
}
107+
108+
/**
109+
* Data provider for invalid test cases.
110+
*
111+
* @return array
112+
*/
113+
public function dataInvalidXsd()
114+
{
115+
return [
116+
'Title attribute too long on <documentation> element' => [
117+
'fixtureFile' => 'InvalidTitleTooLong.xml',
118+
'expectedStdOut' => '',
119+
'expectedStdErr' => "this exceeds the allowed maximum length of '58'",
120+
],
121+
'Documentation root element missing' => [
122+
'fixtureFile' => 'InvalidMissingDocumentationRoot.xml',
123+
'expectedStdOut' => '',
124+
'expectedStdErr' => "Element 'standard': No matching global declaration available for the validation root",
125+
],
126+
'Missing title attribute in the <documentation> root element' => [
127+
'fixtureFile' => 'InvalidMissingDocumentationTitleAttribute.xml',
128+
'expectedStdOut' => '',
129+
'expectedStdErr' => "Element 'documentation': The attribute 'title' is required but missing.",
130+
],
131+
'Missing standard element in the sequence group' => [
132+
'fixtureFile' => 'InvalidMissingStandardElement.xml',
133+
'expectedStdOut' => '',
134+
'expectedStdErr' => "Element 'code_comparison': This element is not expected. Expected is ( standard ).",
135+
],
136+
'Missing sequence group' => [
137+
'fixtureFile' => 'InvalidMissingRuleGroup.xml',
138+
'expectedStdOut' => '',
139+
'expectedStdErr' => "Element 'documentation': Missing child element(s). Expected is ( standard )",
140+
],
141+
'More than two code blocks in one comparison group' => [
142+
'fixtureFile' => 'InvalidMoreThanTwoCodeBlocksInComparison.xml',
143+
'expectedStdOut' => '',
144+
'expectedStdErr' => "Schemas validity error : Element 'code': This element is not expected.",
145+
],
146+
'Less than two code blocks in one comparison group' => [
147+
'fixtureFile' => 'InvalidLessThanTwoCodeBlocksInComparison.xml',
148+
'expectedStdOut' => '',
149+
'expectedStdErr' => "Element 'code_comparison': Missing child element(s). Expected is ( code ).",
150+
],
151+
'<code> element missing title attribute' => [
152+
'fixtureFile' => 'InvalidCodeElementMissingTitle.xml',
153+
'expectedStdOut' => '',
154+
'expectedStdErr' => "Element 'code': The attribute 'title' is required but missing.",
155+
],
156+
'Documentation contains non standard element' => [
157+
'fixtureFile' => 'InvalidContainsNonStandardElements.xml',
158+
'expectedStdOut' => '',
159+
'expectedStdErr' => "Element 'rule': This element is not expected. Expected is ( standard )",
160+
],
161+
'<standard> element contains non CDATA content' => [
162+
'fixtureFile' => 'InvalidStandardContainsElements.xml',
163+
'expectedStdOut' => '',
164+
'expectedStdErr' => "Element 'standard': Element content is not allowed, because the content type is a simple type definition.",
165+
],
166+
'Wrong order of <code_comparison> element (before <standard> element)' => [
167+
'fixtureFile' => 'InvalidWrongOrderOfElements.xml',
168+
'expectedStdOut' => '',
169+
'expectedStdErr' => "Element 'code_comparison': This element is not expected. Expected is ( standard ).",
170+
],
171+
'Empty <code_comparison> element' => [
172+
'fixtureFile' => 'InvalidEmptyCodeComparisonElement.xml',
173+
'expectedStdOut' => '',
174+
'expectedStdErr' => "Element 'code_comparison': Missing child element(s). Expected is ( code ).",
175+
],
176+
'<code_comparison> element contains non <code> elements' => [
177+
'fixtureFile' => 'InvalidCodeComparisonElementContainsNonCodeElements.xml',
178+
'expectedStdOut' => '',
179+
'expectedStdErr' => "Element 'sniff': This element is not expected. Expected is ( code ).",
180+
],
181+
'Empty title attribute in the <documentation> root element' => [
182+
'fixtureFile' => 'InvalidEmptyDocumentationTitleAttribute.xml',
183+
'expectedStdOut' => '',
184+
'expectedStdErr' => "Element 'documentation', attribute 'title': [facet 'minLength'] The value '' has a length of '0'; this underruns the allowed minimum length of '1'.",
185+
],
186+
'Title attribute in the <documentation> root element has wrong type (will throw parser error)' => [
187+
'fixtureFile' => 'InvalidDocumentationTitleAttributeType.xml',
188+
'expectedStdOut' => '',
189+
'expectedStdErr' => "parser error : AttValue: \" or ' expected",
190+
],
191+
'Multiple title attributes in the <documentation> root element (will throw parser error)' => [
192+
'fixtureFile' => 'InvalidDocumentationMultipleTitleAttributes.xml',
193+
'expectedStdOut' => '',
194+
'expectedStdErr' => 'parser error : Attribute title redefined',
195+
],
196+
'<code> element empty title attribute' => [
197+
'fixtureFile' => 'InvalidCodeElementEmptyTitle.xml',
198+
'expectedStdOut' => '',
199+
'expectedStdErr' => "Element 'code', attribute 'title': [facet 'minLength'] The value '' has a length of '0'; this underruns the allowed minimum length of '1'.",
200+
],
201+
'<code> element has has wrong type in the title attribute (will throw parser error)' => [
202+
'fixtureFile' => 'InvalidCodeElementTitleWrongType.xml',
203+
'expectedStdOut' => '',
204+
'expectedStdErr' => "parser error : AttValue: \" or ' expected",
205+
],
206+
'<code> element has has multiple title attributes (will throw parser error)' => [
207+
'fixtureFile' => 'InvalidCodeElementMultipleTitleAttributes.xml',
208+
'expectedStdOut' => '',
209+
'expectedStdErr' => 'Attribute title redefined',
210+
],
211+
'<code> element contains non string elements' => [
212+
'fixtureFile' => 'InvalidCodeElementContainsNonStringElements.xml',
213+
'expectedStdOut' => '',
214+
'expectedStdErr' => "Element 'code': Element content is not allowed, because the content type is a simple type definition.",
215+
],
216+
];
217+
}
218+
}

0 commit comments

Comments
 (0)