Mistake in smart init

[BramVanroy]

I think there is a small mistake that leads to a big difference in the smart initialization.

spring/spring_amr/utils.py

Line 106 in 3907994

s_ = s + tokenizer.INIT

The following line will never match becase the special INIT token comes before the token in BART, not after. So the match will only trigger for these special tokens, but more by accident than by intent I think (the ones after "match" of course).

match -00 Ġ
match -01 Ġ
match -02 Ġ
match -03 Ġ
match -04 Ġ
match -05 Ġ
match -06 Ġ
match -07 Ġ
match -08 Ġ
match -09 Ġ
match -10 Ġ
match -11 Ġ
match -12 Ġ
match -13 Ġ
match -14 Ġ
match -15 Ġ
match -16 Ġ
match -17 Ġ
match -18 Ġ
match -19 Ġ
match -21 Ġ
match -22 Ġ
match -23 Ġ
match -24 Ġ
match -25 Ġ
match -26 Ġ
match -27 Ġ
match -28 Ġ
match -29 Ġ
match -20 Ġ
match -31 Ġ
match -32 Ġ
match -33 Ġ
match -34 Ġ
match -35 Ġ
match -36 Ġ
match -37 Ġ
match -38 Ġ
match -39 Ġ
match -40 Ġ
match -41 Ġ
match -42 Ġ
match -43 Ġ
match -44 Ġ
match -45 Ġ
match -46 Ġ
match -47 Ġ
match -48 Ġ
match -49 Ġ
match -50 Ġ
match -51 Ġ
match -52 Ġ
match -53 Ġ
match -54 Ġ
match -55 Ġ
match -56 Ġ
match -57 Ġ
match -58 Ġ
match -59 Ġ
match -60 Ġ
match -61 Ġ
match -62 Ġ
match -63 Ġ
match -64 Ġ
match -65 Ġ
match -66 Ġ
match -67 Ġ
match -68 Ġ
match -69 Ġ
match -70 Ġ
match -71 Ġ
match -72 Ġ
match -73 Ġ
match -74 Ġ
match -75 Ġ
match -76 Ġ
match -77 Ġ
match -78 Ġ
match -79 Ġ
match -80 Ġ
match -81 Ġ
match -82 Ġ
match -83 Ġ
match -84 Ġ
match -85 Ġ
match -86 Ġ
match -87 Ġ
match -88 Ġ
match -89 Ġ
match -90 Ġ
match -91 Ġ
match -92 Ġ
match -93 Ġ
match -94 Ġ
match -95 Ġ
match -96 Ġ
match -97 Ġ
match -98 Ġ
match -of Ġ

They match because they of an empty token: when looking for subcomponents to use, the if-else statement before this loop will create components by splitting on -. But when you split on -01 the result will be ["", "01"]. And as a consequence the empty string will lead to this accidental match.

The fix would be to add the INIT before the token. And probably also to filter the tok_split on empty strings.