Add slua_definitions.yaml

[tapple]

This PR adds all the information necessary to generate all metadata that a code editor needs to work with SLua

I'm marking this as ready for review because I've reached a milestone: as of this PR, this repository can fully re-generate these 2 files in the viewer:

• keywords_lsl_default.xml
• keywords_lua_default.xml

Additionally, to the best of my knowledge, these two files now accurately document all functions, events, and constants in lsl and slua, no more, and no less. You can review my improvements to these two files here:

• Type-fixes viewer#5288
• Updates and improvements to keywords_lua_default.xml viewer#5301
• fix keywords in keywords_lua_default.xml viewer#5325
• Change the keywords.xml generator from c to python viewer#5332

To be done in a follow up PR #42 :

1. add a slua.d.luau generator to gen_definitions.py (used by luau-lsp)
2. add a slua.docs.json generator to gen_definitions.py (used by luau-lsp)
3. add a lua_definitions.schema.json file (to catch errors in future modifications of this file)

---

How did I create this file?

1. I started by converting syntax_def_default.json (from Luau type fixes for luau-lsp sl-vscode-plugin#57) to yaml
   • This file started as a direct LLSD -> JSON conversion of these two files:
     1. keywords_lsl_78aaa42a-9d1f-b7bb-ac9a-856a2de7b3da.llsd.xml from the simulator capability
     2. types_lua_default.xml from the project/lua-editor viewer
   • I then made a few luau type fixes manually, primarily adding a way to represent type parameters (AKA generics)
2. I then applied all changes from the lsl half of the file to keywords_lsl.yaml
3. I then modified gen_definitions.py until it could generate a bit-identical file to keywords_lsl_78aaa42a-9d1f-b7bb-ac9a-856a2de7b3da.llsd.xml
   • this required modifying the llsd xml generator: Make the xml output better match files in the sim/viewer python-llsd#37
4. I renamed the lua half of syntax_def_default.yaml to lua_definitions.yaml
5. I then added a slua_syntax command to generate keywords_lua.xml
   • note that keywords_lua.xml was NOT an origin file of syntax_def_default.yaml. It is a copy of keywords_lsl.xml with hand edits mode
6. I simplified a few rough things about the format used by types_lua_default.xml:
   • I converted all luau types from llsd to strings. It's easier to read
   • I added basic regex validation that the types are valid and have been defined
   • I split classes in two: baseClasses and classes. The language generator in vscode-plugin has this distinction; I just formalized it in the schema. My type validator in the previous line needed this
   • I moved modules that were previously in globalVariables to modules
7. I merged all differences between the generated keywords_lua.xml and the original back into lua_definitions.yaml.
   1. I added these sections that were unique to keywords_lua.xml:
      • controls (direct conversion)
      • types (direct conversion, renamed to builtinTypes)
      • constants (converted to builtinConstants)
      • functions (converted to builtinFunctions)
   2. lua_definitions.yaml had better type annotations, because I'd previously enhanced them in Luau type fixes for luau-lsp sl-vscode-plugin#57
   3. keywords_lua.xml had better documentation comments
8. I made further improvements to the comments and type annotations that aren't found anywhere but this PR, and it's derivative Updates and improvements to keywords_lua_default.xml viewer#5301

---

Part of a set of PR's improving the definitions and keywords files in lsl and slua:

• Add slua_definitions.yaml #38
• Type-fixes viewer#5288
  • Updates and improvements to keywords_lua_default.xml viewer#5301
  • fix keywords in keywords_lua_default.xml viewer#5325
• Luau type fixes for luau-lsp sl-vscode-plugin#57
• Make the xml output better match files in the sim/viewer python-llsd#37
  • This needs to be accepted and released before the unit tests will pass

---

I verified that the output of this is bit-identical with the sim capability keywords_lsl_78aaa42a-9d1f-b7bb-ac9a-856a2de7b3da.llsd.test.xml, except for:

• sort order
• actual bugfixes
• some lost newlines in tooltips

I used the changes in fa208df (since reverted) to verify this. Here's the results, prettified, for inspection:

• from the sim:
  keywords_lsl_78aaa42a-9d1f-b7bb-ac9a-856a2de7b3da.llsd.autosorted.pretty.xml
• generated from this PR:
  keywords_dev.llsd.pretty.xml

[HaroldCindy]

@tapple You don't need to worry too much about generating a bit-identical version of the existing LLSD in the viewer repo once you've validated that you can, that file's not considered canonical, and we're okay with its form changing if it's not going to be edited directly anyway.

It seems that it does generate a bit-identical version of the slua definitions with the patch, so IMO that's good enough validation that the schema is at least consistent, and we shouldn't need the extended llsd functionality going forward. Would you be okay with removing it from this PR, and we can address python-llsd separately?

[tapple]

@HaroldCindy Done. I removed the dependency on the unreleased python-llsd version. Here's the changes that made to the generated files:

• Change the keywords.xml generator from c to python viewer#5332

[HaroldCindy]

Thanks muchly!

This seems broadly reasonable. I don't see any remaining issues that are worth blocking merging on, so I'll get this in