Fix CST walker coverage for general multi-alt groups#768
Merged
Conversation
Previously, multi-alt groups that didn't match specialized patterns (simple CST, token-only, single-alt) were parsed but their results discarded. This meant tokens like INSERT keywords and VALUES clauses were missing from the CST walker output. Changes: - gen_util: Add is_general_group, general_group_type_name, general_group_field_name, general_group_alt_struct_name - generator: Generate variant type + struct per alt for general groups - parser_gen: Add gen_general_group_store_optional and gen_general_group_store with backtracking support - visitor_gen: Add WalkGeneralGroup action that walks per-alt fields - Update sqlite driver test expectations to include newly visible tokens (INSERT keyword, VALUES clause, FROM table, column defs) All 159 package-gale tests pass. All 1323 wado tests pass. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
The incomplete CST walker coverage issue has been fixed by adding general multi-alt group support. All three patterns (repeated separators, token-only groups, multi-element groups) now produce stored fields that the walker can traverse. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
Replace content-based naming (e.g., KInsertOrKReplaceOrKInsertK...Group) with concise rule-name+index naming (e.g., InsertStmtGroup0, group_0). Changes across 5 files: - gen_util.wado: general_group_type_name(rule_name, group_index), general_group_field_name(group_index), general_group_alt_struct_name(rule_name, group_index, alt_index) - gen_context.wado: add current_rule_name and group_counter fields to GenContext - generator.wado: thread rule_name and group_counter through type generation - parser_gen.wado: use ctx.current_rule_name and ctx.group_counter with save/restore - visitor_gen.wado: thread rule_name and group_counter parameters Note: triggers a cross-module codegen bug in the Wado compiler when linking via main.wado (type mismatch: expected (ref $type), found (ref $type)). Each file compiles individually without errors. Needs compiler fix first. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
Three interconnected issues fixed: 1. Group counter divergence between type gen, parser gen, and visitor gen. Non-storable groups (groups that don't match any specialized pattern) contain nested general groups that advance the counter in the type gen's gen_element_variant_types path, but not in collect_fields/element_to_field. Fixed by advancing the counter using count_element_groups for non-storable groups in all three passes. 2. Shared single-alt group type naming. When two rules (e.g., select_or_values and select_core) share the same single-alt group struct (e.g., KFromItemItem), the nested general group types were emitted only under the first rule's name. The parser gen for the second rule would reference non-existent types. Fixed by tracking which rule "owns" each single-alt group's types via GenContext.single_alt_owner map, and overriding current_rule_name in gen_consume_single_alt_group. 3. Compiler arity check for cross-module function calls was missing, causing confusing codegen errors instead of proper diagnostics. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
Working tree is dirtyIntegrity checks produced the following changes: Please run |
Add a check in resolve_struct_literal that all struct fields must be provided in the literal. Previously, omitting a field silently produced invalid Wasm (struct.new with wrong number of values on the stack), causing runtime type mismatches like "expected (ref $type), found (ref (exact $type))". Now the compiler emits a clear "missing field" compile error. Also adds e2e tests for missing fields, extra fields, and cross-module struct field types. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
Distinguish between "field not found" (field access like x.foo) and "extra field in struct literal" (struct literal with a field that doesn't exist on the struct). The new error message is: struct 'Foo' has no field 'bar' https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
Update struct_literal_unknown_field_error and
struct_literal_unknown_field_implicit_error fixtures to match the new
ExtraField error message format ("struct 'X' has no field 'y'").
Add golden fixture for cross_module_struct_field_type.
https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
gen_general_alt_parse_and_store and gen_general_alt_construct used gen_element for each element, which calls gen_repeat (greedy) for Repeat nodes. This lost the non-greedy while-loop guard (peek_at(1) disambiguation) that gen_elements_with_non_greedy provides. The regression caused CREATE TABLE with multiple columns and table constraints to fail parsing because the comma_column_def loop consumed commas greedily instead of stopping at table constraints. Add failing test first (TDD), then fix by using gen_elements_with_non_greedy in general group code generation. https://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
package-gale/TODO.md: multi-alt groups that didn't match specialized patterns (simple CST, token-only, single-alt) were parsed but their results discarded, causing tokens to be missing from CST walker outputsingle_alt_ownermappingalt_gc_startsfor out-of-order processingChanges
gen_util.wadois_general_group,count_general_groups,count_element_groups(now recurses into non-storable groups)gen_context.wadosingle_alt_ownermap +general_group_rule_name()+alt_gc_startsfor counter consistencygenerator.wadoparser_gen.wadogen_general_group_store_optional/gen_general_group_storewith backtracking; fix counter save/restore in all two-pass functions; use owner rule name in shared single-alt groupsvisitor_gen.wadoWalkGeneralGroupaction; advance counter for non-storable groups; fixbuild_general_group_walk_actioncounter scopingresolver/call.rsTest plan
on-task-donecompletes successfullyhttps://claude.ai/code/session_01VvNqsUaUQNBJkaUzWQFuZC