Commit db07acb
committed
Separation of concerns for Tokenizer
1 parent e59b554 commit db07acb
File tree
17 files changed
+837
-626
lines changed- src
- Helper
- Tokenizer
- Parser
- tests/Tokenizer
17 files changed
+837
-626
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | 16 | | |
| 17 | + | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | | - | |
88 | | - | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
89 | 90 | | |
90 | 91 | | |
91 | 92 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
| 50 | + | |
49 | 51 | | |
50 | 52 | | |
51 | 53 | | |
| |||
54 | 56 | | |
55 | 57 | | |
56 | 58 | | |
| 59 | + | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
| 64 | + | |
| 65 | + | |
61 | 66 | | |
62 | 67 | | |
63 | 68 | | |
| |||
66 | 71 | | |
67 | 72 | | |
68 | 73 | | |
| 74 | + | |
69 | 75 | | |
70 | 76 | | |
71 | 77 | | |
72 | 78 | | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
73 | 83 | | |
74 | 84 | | |
75 | 85 | | |
76 | 86 | | |
77 | 87 | | |
78 | 88 | | |
79 | 89 | | |
80 | | - | |
81 | | - | |
82 | | - | |
| 90 | + | |
83 | 91 | | |
84 | 92 | | |
| 93 | + | |
85 | 94 | | |
| 95 | + | |
86 | 96 | | |
87 | 97 | | |
88 | 98 | | |
| 99 | + | |
89 | 100 | | |
90 | 101 | | |
91 | 102 | | |
| |||
95 | 106 | | |
96 | 107 | | |
97 | 108 | | |
| 109 | + | |
98 | 110 | | |
99 | 111 | | |
100 | 112 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
148 | 149 | | |
149 | 150 | | |
150 | 151 | | |
151 | | - | |
152 | | - | |
| 152 | + | |
| 153 | + | |
153 | 154 | | |
154 | 155 | | |
155 | 156 | | |
| |||
174 | 175 | | |
175 | 176 | | |
176 | 177 | | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
177 | 186 | | |
178 | 187 | | |
179 | 188 | | |
| |||
184 | 193 | | |
185 | 194 | | |
186 | 195 | | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | 196 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
0 commit comments