Compiler directives cause downstream errors

[MatchlessJoyride]

ast_error.py.txt
error.sv.txt
no_error.sv.txt

I have attached three files to help recreate this issue. You'll have to remove the .txt extension to run them locally.

When a compiler directive is inserted in the codebase, it has a potential to "break" the syntax. In error.sv I have made a contrived example that breaks the begin/end of an always_comb block:

    always_comb begin
        `ifndef MYDEFINE
            if (a) begin
        `else
            if (b) begin
        `endif
                c = a ^ b;
            end
    end

This has the downstream effect of breaking the ability of tree-sitter to properly parse the module instantiation.

In ast_error.py you will see this query:

MODULE_INST_QUERY = '''
(module_instantiation
  (simple_identifier) @module_name (#eq? @module_name "my_module")
  (hierarchical_instance
    (name_of_instance
      (simple_identifier) @instance_name
    )
  )
)
'''

When ast_error.py is run against no_error.sv, it properly returns the module instantiation. But it does not when you run it against error.sv.

Command lines:

python ast_error.py error.sv

python ast_error.py no_error.sv

[gmlarumbe]

Hi @MatchlessJoyride ,

From #11 (comment)

Currently this is the expected behavior since tree-sitter cannot preprocess code. Conditional compilation directives and macros make code hard to parse and usually result in parsing errors.

However tree-sitter-systemverilog provides basic preprocessing capabilites for code patterns that are supposed to be common, even though that means deviating from the LRM and increasing the size/complexity of the grammar.

I am afraid this is not an issue of the grammar but a tree-sitter limitation and is present in other tree-sitter grammars. You can check:

• Issue with handling preprocessor attributes tree-sitter/tree-sitter-c#13
• https://habr.com/en/articles/835192/

The parser recognizes the conditional_compilation_directive node but does not preprocess the code, leaving one extra begin and preventing closing of the always_comb block. This results in the instantiation being detected as a checker_instantiation instead of a module_instantiation.

The snippet below could be parsed properly instead:

`ifndef MYDEFINE
always_comb begin
    if (a) begin
        c = a ^ b;
    end
end
`else
always_comb begin
    if (b) begin
        c = a ^ b;
    end
end
`endif

I agree this approach does not look very practical. However I am afraid it would not be simple to modify the grammar to support this specific case.

[MatchlessJoyride]

Thanks for the reply - I had a feeling this was going to be a known limitation. Unfortunately modifying the RTL is going to be a no-go in this specific scenario.

For my application I think it would be sufficient if I knew there errors when parsing the AST. Is it possible to write a query that would return true if there were any parsing errors?

[gmlarumbe]

Hi @MatchlessJoyride ,

Is it possible to write a query that would return true if there were any parsing errors?

I believe this mostly depends on the tool you are using to run the query. When using tree-sitter-cli the tree-sitter parse <filename.sv> command returns 1 if there were errors during parsing:

$ tree-sitter parse <file_with_errors.sv>
> [...]
$ echo $?
> 1

However if you are using some language bindings you should look at its documentation.

A simple way would be grep for regexp containing "(ERROR)" in the returned AST but I am sure there are more accurate/efficient ways depending on the tool.

On the other hand if you are doing batch processing of code with preprocessor directives you can always preprocess original files and dump the output into intermediate preprocessed files before parsing these with tree-sitter. You have some open source tools like verilator, vppreproc or iverilog than can preprocess code.