Add semantic token provider to Civet LSP

[Copilot]

Syntax highlighting via TextMate grammar patching is brittle and incomplete for Civet. Semantic tokens from the parser are the correct long-term solution, giving editors accurate, context-aware highlighting.

Changes

source/semantic-tokens.civet (new)

Core token walker. Traverses the pruned AST collecting {pos, length, group} tuples:

• Group is resolved by priority: explicit node.group → node type field → inherited parent group (handles RegularExpressionLiteral, TemplateLiteral) → keyword table lookup
• Whitespace-only and zero-length tokens are filtered out
• Result is sorted by source position

ast := compile(src, { ast: true, sync: true })
tokens := getSemanticTokens(ast)
// [{ pos: 0, length: 5, group: "keyword" }, { pos: 14, length: 7, group: "string" }, ...]

Token groups: "keyword" | "string" | "number" | "regexp" | "operator" | "comment" | "type"

source/parser/types.civet

Adds optional group?: string to ASTLeaf so the parser can explicitly annotate tokens in the future (as suggested in the issue) without requiring inference.

source/main.civet + types/types.d.ts

Exports getSemanticTokens, SemanticToken, and TokenGroup from the public API.

lsp/source/server.mts

• Registers semanticTokensProvider capability with a 7-type legend
• Handles textDocument/semanticTokens/full for .civet files: parses source, walks AST, converts byte offsets to line/character positions, returns LSP delta-encoded data array

test/semantic-tokens.civet (new)

13 tests covering keywords, all literal types, comments, sort order, and zero-length filtering.