Create object validation (#435)

* started bulding the scraper

* Added component scraping support

* Added better signature handling

* Start collecting interface data

* Scrape roku docs websites for interfaces too

* rename script.

* fix bugs finding interface implementors.
generate ts file instead of json

* First pass for events

* remove eslint overrides for nonexistant data.ts file

* better data exporting

* fix lint issue

* collect basic descriptions

* Reference non-localized docs

* start parsing markdown instead of html

* Use markdown parser for `implementedBy`.

* Add better typing support for tables.

* Remove nested signatures array in favor of duplicate methods

* Convert interface method scraping to use markdown

* Removed default parameter value for components

* Collect all description markdown for components, interfaces and events

* Remove extraneous commas from descriptions

* Collect "availableSince" for main item

* move interfaces to bottom of the file

* Make more generic markdown retrieving methods

* use markdown parser for "implementedBy" table

* restructure class

* Fix "implemented by" bugs in some markdown docs

* Rename Implementor to Implementer

* Add interface implementers list.

* Remove jsdom entirely in favor of marked

* fix ci build issue

* Repair doc urls in descriptions.
Collect SGNode items.

* exclude "overview" component pages

* Collect scenegraph node fields

* Commiting files

* Tests

* Fixed tests - made sure to check actual unquoted string values in createObject validation

* small tsconfig fixes

* Added Diagnostic and validation for unknown brightscript component in createObject() call

* Added additional logic to check for mismatch in CreateObject argument count (#561)

* Added additional logic to check for mismatch in CrateObject argument count

* Fixed scraping code, and gracefully handles bad createObject docs

* Moved constructor push out of loop

* Updated to latest version of marked library

* Removed commented code

* Add merge logic for overrides

* Add doc url for RSGPalette

* Added missing deepmerge package

* Scapes docs for deprecation status, adds it to data.json, and adds a validation for CreateObject(<deprecated>)

* Lint fix

* Sorts data.json so nodes, compoenents, interfaces, events, properties, methods, implementors etc. are in alphabetical order

* add basic typedef generation (WIP)

Co-authored-by: Mark Pearce <mark.pearce@redspace.com>
Co-authored-by: Mark Pearce <markwpearce@gmail.com>

Author: TwitchBronBron

.eslintrc.js

@@ -209,5 +209,14 @@ module.exports = {
             '@typescript-eslint/no-require-imports': 'off',
             '@typescript-eslint/no-var-requires': 'off'
         }
+    }, {
+        files: ['src/roku-types/data.json'],
+        rules: {
+            '@typescript-eslint/no-unused-expressions': 'off',
+            '@typescript-eslint/quotes': 'off',
+            'no-template-curly-in-string': 'off',
+            'eol-last': 'off',
+            '@typescript-eslint/semi': 'off'
+        }
     }]
 };

.vscode/launch.json

@@ -64,6 +64,26 @@
                 "!**/node_modules/typescript/**",
                 "!**/node_modules/vscode-languageserver/**"
             ]
+        },
+        {
+            "name": "Debug Roku Docs Scraper",
+            "type": "node",
+            "request": "launch",
+            "runtimeExecutable": "node",
+            "runtimeArgs": [
+                "--nolazy",
+                "-r",
+                "ts-node/register/transpile-only"
+            ],
+            "args": [
+                "scripts/scrape-roku-docs.ts"
+            ],
+            "cwd": "${workspaceRoot}",
+            "internalConsoleOptions": "openOnSessionStart",
+            "skipFiles": [
+                "<node_internals>/**",
+                "node_modules/**"
+            ]
         }
     ]
-}
+}

.vscode/tasks.json

@@ -44,10 +44,28 @@
                 "test:nocover",
                 "--silent"
             ],
+            "presentation": {
+                "echo": true,
+                "reveal": "always",
+                "focus": true,
+                "panel": "shared",
+                "showReuseMessage": true,
+                "clear": true
+            },
             "group": {
                 "kind": "test",
                 "isDefault": true
             },
+            "problemMatcher": []
+        },
+        {
+            "label": "scrape-roku-docs",
+            "type": "shell",
+            "command": "npm",
+            "args": [
+                "run",
+                "scrape-roku-docs"
+            ],
             "presentation": {
                 "echo": true,
                 "reveal": "always",
@@ -59,4 +77,4 @@
             "problemMatcher": []
         }
     ]
-}
+}