From 8ad6d7f2e964bf414e8d04ac78b8e48e86449a7b Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:02:02 -0700 Subject: [PATCH 1/7] Add test info to contriubting doc --- CONTRIBUTING.md | 59 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 96365b34..1484f95a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,6 +32,10 @@ - [How to verify which Spectral rules are running in Production and Staging LintDiff](#how-to-verify-which-spectral-rules-are-running-in-production-and-staging-lintdiff) - [Installing NPM dependencies](#installing-npm-dependencies) - [How to test](#how-to-test) + - [Run all tests](#run-all-tests) + - [Run a single test](#run-a-single-test) + - [Debugging tests](#debugging-tests) + - [Error running Nimma](#error-running-nimma) - [How to write a new validation rule using typescript](#how-to-write-a-new-validation-rule-using-typescript) - [Spectral rule](#spectral-rule) - [Native rule](#native-rule) @@ -555,12 +559,65 @@ using the command line. # How to test -To run all tests under the repo +## Run all tests + +To run all tests under the repo: ```bash rush test ``` +This runs both the `test-spectral` and `test-native` scripts as defined in `packages\azure-openapi-validator\core\package.json`. + +## Run a single test + +To run a single spectral test: + +```bash +cd packages/rulesets +npm run test-spectral -- /path/to/test.test.ts +``` + +To run a single native test: + +```bash +cd packages/rulesets +npm run test-native -- /path/to/test.test.ts +``` + +## Debugging tests + +To debug a test, make sure you set a breakpoint and use a Javascript Debug Terminal when running the test. See +[Run a single test](#run-a-single-test) to debug one test at a time. + +This project uses [Jest](https://jestjs.io/) for running tests. See [the Jest docs](https://jestjs.io/docs/cli) for more +CLI options, such as `watch` and coverage collection options. + +## Error running Nimma + +While running your test locally, if you run into an error with text: + +> Error running Nimma + +Try wrapping the `linter.run()` call in a try/catch block. This might help you determine where the error is coming from. +Remember to remove the try/catch block afterwards as it will interfere with test logic. E.g., + +```typescript +let result + + try { + result = await linter.run(oasDoc).then((results) => { + const paths = Object.keys(oasDoc.paths) + expect(results.length).toBe(2) + }) + } catch (error) { + if (error && error.errors && Array.isArray(error.errors)) { + throw new Error(`Errors found. ${error.errors}`) + } + } + return result +``` + # How to write a new validation rule using typescript ## Spectral rule From 736799d70a11ffc20d3e4c889df8521f136f1b65 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:02:41 -0700 Subject: [PATCH 2/7] Edit comment in .eslintrc.yaml --- .eslintrc.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.eslintrc.yaml b/.eslintrc.yaml index aa123553..be184c78 100644 --- a/.eslintrc.yaml +++ b/.eslintrc.yaml @@ -7,7 +7,7 @@ parserOptions: # To silence VS Code ESLint error of "cannot find 'tsconfig.json'", add entries like this # to your "Preferences: Open User Settings (JSON)" # "eslint.workingDirectories": [ - # {"directory": "./packages/azure-openapi-validator/autorest", "changeProcessCWD": true} + # { "mode": "auto" } # ], # This solution is based on: # https://github.com/typescript-eslint/typescript-eslint/issues/251#issuecomment-521142325 From 883cf0315699abdf529922b3b1de282fa965d281 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:03:44 -0700 Subject: [PATCH 3/7] add forward slashes as an allowable testMatch path --- packages/azure-openapi-validator/core/package.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/azure-openapi-validator/core/package.json b/packages/azure-openapi-validator/core/package.json index f85d768c..91e0b44b 100644 --- a/packages/azure-openapi-validator/core/package.json +++ b/packages/azure-openapi-validator/core/package.json @@ -55,6 +55,7 @@ "jest": { "testMatch": [ "**/tests/*.test.ts", + "**\\tests\\*.test.ts", "!**/*.test.d.ts" ], "globals": { @@ -71,4 +72,4 @@ ], "preset": "ts-jest" } -} +} \ No newline at end of file From 442ab3150fb52d00ab2f450bad6d5a3fd5b274d6 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:19:20 -0700 Subject: [PATCH 4/7] Update CHANGELOG.md --- packages/rulesets/CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/rulesets/CHANGELOG.md b/packages/rulesets/CHANGELOG.md index 3596c608..a6d23637 100644 --- a/packages/rulesets/CHANGELOG.md +++ b/packages/rulesets/CHANGELOG.md @@ -5,6 +5,8 @@ ### Patches - [AllTrackedResourcesMustHaveDelete][TrackedResourcePatchOperation] Skip the api paths if it has PrivateEndpointConnectionProxy +- Add documentation in CONTRIBUTING.md for running single tests and debugging tests for linter rules +- Add support for forward slashes in Jest `testMatch` regex ### Patches From 2bfe57cd7fd41ca7559c8ccb5bda46a033fb8779 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:20:16 -0700 Subject: [PATCH 5/7] Remove change log edits The change log is under a specific package --- packages/rulesets/CHANGELOG.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/packages/rulesets/CHANGELOG.md b/packages/rulesets/CHANGELOG.md index a6d23637..3596c608 100644 --- a/packages/rulesets/CHANGELOG.md +++ b/packages/rulesets/CHANGELOG.md @@ -5,8 +5,6 @@ ### Patches - [AllTrackedResourcesMustHaveDelete][TrackedResourcePatchOperation] Skip the api paths if it has PrivateEndpointConnectionProxy -- Add documentation in CONTRIBUTING.md for running single tests and debugging tests for linter rules -- Add support for forward slashes in Jest `testMatch` regex ### Patches From 9ff697278cefaa1f51662ad741cf6a445ee57b56 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:22:57 -0700 Subject: [PATCH 6/7] Fix indentation in CONTRIBUTING.md example code --- CONTRIBUTING.md | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1484f95a..68f39e11 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -604,18 +604,17 @@ Remember to remove the try/catch block afterwards as it will interfere with test ```typescript let result - - try { - result = await linter.run(oasDoc).then((results) => { - const paths = Object.keys(oasDoc.paths) - expect(results.length).toBe(2) - }) - } catch (error) { - if (error && error.errors && Array.isArray(error.errors)) { - throw new Error(`Errors found. ${error.errors}`) - } +try { + result = await linter.run(oasDoc).then((results) => { + const paths = Object.keys(oasDoc.paths) + expect(results.length).toBe(2) + }) +} catch (error) { + if (error && error.errors && Array.isArray(error.errors)) { + throw new Error(`Errors found. ${error.errors}`) } - return result +} +return result ``` # How to write a new validation rule using typescript From b26734872e768122eb76106bef8d312078abd316 Mon Sep 17 00:00:00 2001 From: Brett DeFoy Date: Tue, 28 Jan 2025 13:27:46 -0700 Subject: [PATCH 7/7] Add link to writeup on Nimma error --- CONTRIBUTING.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 68f39e11..3b5488e8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -600,7 +600,7 @@ While running your test locally, if you run into an error with text: > Error running Nimma Try wrapping the `linter.run()` call in a try/catch block. This might help you determine where the error is coming from. -Remember to remove the try/catch block afterwards as it will interfere with test logic. E.g., +Remember to remove the try/catch block afterwards as it will interfere with test logic. See the code example below: ```typescript let result @@ -617,6 +617,8 @@ try { return result ``` +This is an error from Spectral, which uses Nimma. See more [here](https://github.com/Azure/azure-sdk-tools/issues/6856). + # How to write a new validation rule using typescript ## Spectral rule