`TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'` when processing UVVM file

[JHertz5]

Environment
Head of branch issue-1531 (needed to parse the file correctly)
Git hash: b565fdd

Describe the bug
#1531 was raised because a UVVM file couldn't be parsed. Once the parsing error had been corrected (#1532), the file still couldn't be processed by VSG. It would crash with the following log output.

$ ./bin/vsg -f test.vhd -a
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/apply_rules.py", line 126, in apply_rules
    oRules.check_rules(
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rule_list.py", line 229, in check_rules
    oRule.analyze(self.oVhdlFile)
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rule.py", line 147, in analyze
    lToi = self._get_tokens_of_interest(oFile)
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rules/multiline_structure.py", line 72, in _get_tokens_of_interest
    lMyReturn = remove_non_arrays(self.assignment_operator, lReturn)
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rules/multiline_structure.py", line 672, in remove_non_arrays
    if rules_utils.array_detected_after_assignment_operator(assignment_operator, oToi):
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rules/utils.py", line 422, in array_detected_after_assignment_operator
    if not open_paren_after_assignment_operator(assignment_operator, lTokens):
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rules/utils.py", line 457, in open_paren_after_assignment_operator
    return is_next_token_ignoring_whitespace(parser.open_parenthesis, iToken, lTokens)
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/rules/utils.py", line 414, in is_next_token_ignoring_whitespace
    iToken = utils.find_next_non_whitespace_token(iToken + 1, lTokens)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jhertzog/code/vhdl-style-guide-fork/./bin/vsg", line 16, in <module>
    main()
  File "/home/jhertzog/code/vhdl-style-guide-fork/bin/../vsg/__main__.py", line 154, in main
    for tResult in pool.imap(f, enumerate(commandLineArguments.filename)):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

This indicates a type error in multiline structure rule processing.

To Reproduce
Steps to reproduce the behavior:

1. Download the file mentioned above
2. Run VSG with -a or --fix so that it doesn't stop after raising errors in the first phase.
3. See error

Expected behavior
The tool should run completely without type errors.

[JHertz5]

With use of the --debug argument, I can see that this is occurring during the processing of variable_403.

[JHertz5]

I've narrowed the problem down to the source. There are a few functions that declare a variable in a particular way.

  function to_string(val : time_vector) return string is
    variable v_result : string(1 to 2 + -- parentheses
    2 * (val'length - 1) +              -- commas
    32 * val'length);
  begin
    return v_result(1 to v_width);
  end function;

The part of this that causes the problem is the fact the the array constraint for the string subtype goes over multiple lines. (In my opinion, this bit of code that calculates the string length would be much easier to read if it was refactored out to a function, especially because this file duplicates this code in three places. However, that is besides the point - it has exposed a bug in VSG that must be fixed.)

To demonstrate the issue -
This runs fine.

package body string_methods_pkg is

  function to_string(val : time_vector) return string is
    variable v_result : string(1 to 2);
  begin
  end function;

end package body string_methods_pkg;

This causes the failure.

package body string_methods_pkg is

  function to_string(val : time_vector) return string is
    variable v_result : string(1 to
    2);
  begin
  end function;

end package body string_methods_pkg;

I will look into why the tool can't handle this code, but it is already suspicious. The rule is designed to handle multi-line default values, but there is no default value here. It looks like the array constraint is being incorrectly picked up as an array default value.

[JHertz5]

After further investigation, my previous examples are not correct - the only reason that having no newline in the array constraint didn't cause an error was because the rule was configured to ignore single line assignments. Disabling that causes the tool to fail in both cases.

I'm using this config now:

rule:
  global:
    disable: true
  variable_403:
    disable: false
    ignore_single_line: false

[JHertz5]

I've arrived at the source of the problem - variable_403 is being too greedy! variable_403 is designed to act on variable initialisations, i.e. the tokens after the variable assignment token in a variable declaration, but they've been set to take everything after the variable keyword!

lTokenPairs.append([token.variable_keyword, token.semicolon])

correcting this to

lTokenPairs.append([token.assignment_operator, token.semicolon])

solves the problem.

Is it possible that there are cases in which this rule has been applying formatting to array constraints? Will this change remove formatting enforcement that users have been treating as a feature rather than a bug? Not sure. All tests passed with this change in place, so no intentional functionality has been broken at least.

We may need to create rules for multiple formatting of array constraints. I'll also look into whether other similar bugs exist for other types of assignment, e.g. signal, constant, etc.

[JHertz5]

I've had a check and this error is not present anywhere else outside of variable_403 as far as I can tell. I consider the issue to be closed, I'm merging the branch.