Skip to content

Commit 7939aac

Browse files
spotandjakespotandjake
authored
chore: Codegen design doc (#13)
1 parent 695db1b commit 7939aac

1 file changed

Lines changed: 274 additions & 31 deletions

File tree

‎Codegen.md‎

Lines changed: 274 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -20,87 +20,330 @@ This document is just a high level overview more similar to compiling a parseTre
2020
Instead of directly emitting wasm instructions by traversing the anf tree, we are instead going to convert the anf tree to a wasm tree this will allow us to collect module types and other things in a more organized manner.
2121

2222
## Program
23-
Programs map directly onto wasm modules, so we are going to compile a program into a wasm module. This means that the program node is going to be the top level. Inside a program we will directly compile the classes inside.
23+
Programs are the top level of our language and map directly onto files. They are static elements and don't have any runtime representation. We are going to convert a program directly to a wasm module this transformation would convert a file like:
24+
```java
25+
<program_body>
26+
```
27+
to a wasm module like:
28+
```wasm
29+
(module <program_body>)
30+
```
31+
in reality we are going to be generating a wasm tree and instead of outputting `wat` which is the Webassembly text format we are going to be outputting `wasm` which is the binary format, which is more compact and easier for machines to parse and work with. We are documenting the process here in `wat` for simplicity the wasm spec can be followed for the conversion.
2432

25-
## Classes
26-
Given classes are static in our language like namespaces they don't really get compiled into a code unit instead we just compile the things inside.
33+
Documentation on wasm modules can be found here: https://developer.mozilla.org/en-US/docs/WebAssembly/Guides/Understanding_the_text_format#the_simplest_module
2734

28-
If classes we'rent static we have two options either compile them to a wasm gc struct or compile them to a struct in linear memory however this isn't very trivial and would affect a lot of other little details of compilation.
35+
## Classes
36+
Classes in our language are static that means they don't have any runtime representation and cannot be instantiated at runtime. In a sense they are equivalent to namespaces or modules in other languages. As classes have no runtime component they don't really get compiled into code instead they act as a literal namespace.
37+
38+
When compiling a class of the form:
39+
```java
40+
class Program {
41+
int x;
42+
void Main() {}
43+
}
44+
```
45+
we compile the properties as described in the properties section and the methods as described in the methods section. Notably we mangle the name to the class so `x` becomes something similar to `Program_x` and `Main` becomes something similar to `Program_Main` this ensures that every property and method has a unique name and we don't have to worry about name collisions.
2946

3047
### Properties
31-
Class properties can be thought of as global as they save state between function calls and are shared globally, all were going todo is compile them directly into globals so an `int x` might compile into `(global $x i32)` however we are going to need to mangle these names to avoid name collisions so lets say that the `int x` is within the `Program` class we might compile it to something like `(global $Program_x i32)` this is a pretty simple way to compile properties and it works well with the static nature of our classes.
48+
Class properties in our language are treated as globals given there is no instance this means the `int x` in the `Program` class above is going to become `(global $Program_x i32)` in wasm, which is a pretty simple compilation.
3249

3350
### Methods
34-
Methods in our language are also pretty simple, they are not first class and don't have closures which means they are pretty much a 1 to 1 compilation to a wasm function. [function example here](https://developer.mozilla.org/en-US/docs/WebAssembly/Guides/Understanding_the_text_format#our_first_function_body).
51+
Methods in our language are also pretty simple, as the are not first class and don't have closures which means they are pretty much a 1 to 1 compilation to a wasm function.
52+
53+
As an example the function:
54+
```java
55+
int Add(int a, int b) {
56+
<body>
57+
}
58+
```
59+
is going to compile into something like:
60+
```wasm
61+
(func $Program_Add (param $a i32) (param $b i32) (result i32)
62+
<body>
63+
)
64+
```
3565

36-
If we had first class functions things would get more complicated we would need to compute a closure which would essentially be at the time of creation we take all the variables used from outside the functions scope and put them into a struct in linear memory this struct also contains a function reference or table index that we can use to call the function without knowing it's name (for first class functions). We would then need to add an extra parameter to the function for the closure struct and compile the function body to extract the values from the struct.
66+
If we had first class functions things would get more complicated as we would need to build a closure which is essentially a record of all the variables used from the parent scope, we would also need to allow functions to be passed around as values which would require us to use indirect calls and function tables in wasm. This is a bit more complicated to implement however.
67+
68+
For more information on wasm classes see: https://developer.mozilla.org/en-US/docs/WebAssembly/Guides/Understanding_the_text_format#our_first_function_body
3769

3870
## Statements
39-
Statements are going to be lowered within a function to wasm instructions.
71+
Statements in our language are pretty simple to compile as most of them map pretty closely to wasm instructions.
4072

4173
### Assignments
42-
Assignments are pretty simple we need to compile the left hand side which is the location accessor to become a `global.get` if we are working on an array we are going to need to set a memory address.
74+
Assignments are one of the slightly more complicated statements to compile as we need to consider the location which can have a few different forms.
75+
76+
#### Simple Variable Assignment
77+
This is the simplest form of assignment and is pretty much a 1 to 1 compilation. Take the code:
78+
```java
79+
int x;
80+
x = <expr>;
81+
```
82+
This would compile into something like:
83+
```wasm
84+
(local.set $x <expr>)
85+
```
86+
87+
For more information on `local.set` see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Variables/local.set
88+
89+
#### Property Assignment
90+
This is also a rather simple case, given a property assignment like:
91+
```java
92+
Program.x = <expr>;
93+
```
94+
This would compile into something like:
95+
```wasm
96+
(global.set $Program_x <expr>)
97+
```
4398

44-
We are then either going to use `WasmI32.store` for arrays or `local.set` for local variables or `local.set` for global variables `global.set`
99+
For more information on `global.set` see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Variables/global.set
100+
101+
#### Array Assignment
102+
This is where assignments get a tiny bit more tricky, arrays are stored in linear memory and there are a few steps to generating them.
103+
It may be helpful to take a look at [ArrayInitialization](#ArrayInitialization) for more information on how arrays are stored in memory, in order to have enough context to understand how to compile array assignments. Given an array assignment like:
104+
```java
105+
arr[<index>] = <expr>;
106+
```
107+
This would compile into something like:
108+
```wasm
109+
; Load the array length
110+
(local.set $arr_length ; Set a temporary variable to hold the array length
111+
(i32.load
112+
(local.get $arr) ; Get the base pointer of the array
113+
0 ; offset is the first 4 bytes of the array data structure
114+
)
115+
)
116+
; Check if index is out of bounds and trap if it is
117+
(if
118+
(i32.ge_u (local.get $index) (local.get $arr_length)) ; Check if the index is out of bounds
119+
(then
120+
unreachable ; This causes wasm to trap (throw an exception) if the index is out of bounds
121+
)
122+
(else
123+
(i32.store
124+
(local.get $arr) ; Get the base pointer of the array
125+
(i32.add
126+
(i32.mul (local.get $index) 4) ; Calculate the offset for the index, each item is 4 bytes
127+
4 ; Add 4 to skip the length field at the start of the array data structure
128+
)
129+
<expr> ; The value to store at the index
130+
)
131+
)
132+
)
133+
```
134+
135+
For more information the [wasm spec](https://webassembly.github.io/spec/core/) is probably the best resource for understanding how these instructions work.
45136

46137
### Expression Statements
47-
These compile pretty simply as we compile the expression like normal and just `(drop)` the result if there is one.
138+
Expression statements are extremely easy to compile, we compile them like any other function, with one slight difference which is we don't want to leave the result no the stack so we most drop it. This means that the code:
139+
```java
140+
add(1, 2);
141+
```
142+
would compile into:
143+
```wasm
144+
(drop
145+
(call $Program_add (i32.const 1) (i32.const 2))
146+
)
147+
```
148+
or more generically:
149+
```
150+
<expr>
151+
```
152+
becomes:
153+
```wasm
154+
(drop <expr>)
155+
```
156+
157+
For more information on drop see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/drop
158+
159+
For more information on compiling expressions see: [Expressions](#Expressions)
48160

49161
### If Statements
50-
If statements are not to hard to compile they basically compile to a wasm if after we compile the expression.
162+
If statements are extremely simple to compile as they have a direct wasm equivalent, given an if statement like:
163+
```java
164+
if (<condition>) {
165+
<then_body>
166+
} else {
167+
<else_body>
168+
}
169+
```
170+
This would compile into something like:
171+
```wasm
172+
(if
173+
<condition>
174+
(then
175+
<then_body>
176+
)
177+
(else
178+
<else_body>
179+
)
180+
)
181+
```
182+
In the case that there is no else body we can just omit the else block.
183+
184+
For more information on if statements see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/if...else
51185

52186
### While Node
187+
`While` loops are pretty easy to compile however there is no directly equivalent to a while loop at the webassembly level, instead we need to use `loop` and `br` instructions to create a loop. Given a while loop like:
188+
```java
189+
while (<condition>) {
190+
<body>
191+
}
192+
```
193+
This would turn into something like:
194+
```wasm
195+
(block $break_label
196+
(loop $while_loop
197+
(if
198+
(i32.eqz <condition>) ; Check if the condition is false
199+
(then
200+
(br $break_label) ; If the condition is false, break out of the loop
201+
)
202+
)
203+
<body>
204+
(br $while_loop) ; Jump back to the start of the loop
205+
)
206+
)
207+
```
208+
This is a pretty standard way for compilers to implement loops, one thing to note is that while we do a lot of the lowering to wasm when performing codegen this is actually something that we simplify during the ANF conversion process, to unify any type of loop we have in the language. This means that codegen for `while`, `for`, and `do while` is all just syntax sugar on top of a loop.
209+
53210
This is going to compile to a basic loop see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/loop
54211

55212
### ContinueNode
56-
This is going to compile pretty cleanly to a `br` back to the top of the loop
213+
Continue statements are extremely easy to compile, given a continue statement `continue;` this would compile into:
214+
```wasm
215+
(br $while_loop) ; Jump back to the start of the loop
216+
```
217+
218+
For more information on `br` see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/br
57219

58220
### BreakNode
59-
This is going to compile pretty cleanly to a `br` to the end of the loop (I need to look at codegen for this a tiny bit more)
221+
Break statements are also pretty easy to compile, given a break statement `break;` this would compile into:
222+
```wasm
223+
(br $break_label) ; Jump to the end of the loop
224+
```
225+
226+
For more information on `br` see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/br
60227

61228
### ReturnNode
229+
Return statements are also extremely easy to compile, given the statement `return <expr>;` this would compile into:
230+
```wasm
231+
(return <expr>)
232+
```
233+
62234
This is going to compile to a return https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/return
63235

64236
## Expressions
65237

238+
Compiling expressions is pretty simple just like statements as most of them have a pretty direct mapping to wasm instructions.
239+
66240
### Call Node
67-
This is going to compile to a wasm call pretty cleanly
241+
Call nodes in our language are pretty simple to compile as they have a direct mapping to wasm function calls, given a call like:
242+
```java
243+
add(1, 2);
244+
```
245+
This would compile into:
246+
```wasm
247+
(call $Program_add (i32.const 1) (i32.const 2))
248+
```
249+
or more generally:
250+
```java
251+
<function_name>(<arg1>, <arg2>, ...)
252+
```
253+
becomes:
254+
```wasm
255+
(call $<function_name> <arg1> <arg2> ...)
256+
```
257+
258+
For more information on function calls see: https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/call
259+
260+
It's worth noting that this is only simple because we don't have closures or first class functions, for closures we would need to pass a closure pointer as an additional argument to the function, and for first class functions we would need to use indirect calls and function tables in wasm which is a bit more complicated to implement.
261+
262+
We are leaving the design for primitive callouts until we are at that point as they are going to be able to compile into a variety of different constructs from low level wasm instructions to regular function calls depending on the callout.
68263

69264
### Binop Node
70-
This is going to compile pretty cleanly into a wasm instruction we are going to need to use the signature and operator to determine what instruction to use from a lookup table.
265+
Binary operations are also pretty simple to compile as they have direct mappings in webassembly the exact instruction we are going to use is going to depend on both the operator and the types of the operands, but an example of compiling `a + b` would be:
266+
```wasm
267+
(i32.add (local.get $a) (local.get $b))
268+
```
269+
For subtraction it may become:
270+
```wasm
271+
(i32.sub (local.get $a) (local.get $b))
272+
```
273+
or more generally, `<expr1> <operator> <expr2>` becomes:
274+
```wasm
275+
(<wasm_operator> <expr1> <expr2>)
276+
```
71277

72278
### Prefix Node
73-
This is going to be done just like binops but with different instructions. I think for not we have a few option for compiling the simplest one is probably bitwise. It's also usually the fastest.
279+
Prefix nodes are going to be handled the exact same way as binops, given a prefix operation like: `!a` this would compile into:
280+
```wasm
281+
(i32.eqz (local.get $a))
282+
```
283+
or more generally, `<operator> <expr>` becomes:
284+
```wasm
285+
(<wasm_operator> <expr>)
286+
```
287+
As a note while we have described compilation of prefix nodes in a universal manner the only prefix operator in our language is `!` which is the logical not operator.
74288

75289
### New Class Node
76-
This isn't going to be compiled if it was we would store some sort of record in linear memory that points to everything.
290+
This is an object oriented feature as such it does not get compiled. If we had object oriented features it would likely get compiled into a memory allocation or wasm gc struct, of the form `{ <classID>, <field1>, <field2>, ...}`. I would describe a bit more but given we are not implementing them it seems like a waste.
77291

78-
### New Array Node
79-
This is going to probably use a very primitive bump allocator to compile some instructions that write a simple data structure to memory probably something like `<arrayTypeID>, <size>, ...<values>`
292+
### ArrayInitialization
293+
Array initialization is likely the hardest thing to compile in our language as it requires us to work with linear memory.
294+
Our current plan is to ship a basic bump allocator with our runtime which is essentially just a pointer to the end of the allocated memory, and when we want to allocate something we just move the pointer forward by the size of the allocation, this isn't the most efficient and fragments fast but it works for our use cases.
80295

81-
### LocationNode
82-
I think we still need todo some thoughts here but it's essentially going to be come a local.get or local.set depending on the use, this is probably going to be compiled in a somewhat context aware manner.
296+
Arrays are going to be stored in linear memory following the format, `<length> ...<items>` this means that:
297+
```
298+
ptr -> length
299+
ptr + 4 -> item 1
300+
ptr + 8 -> item 2
301+
...
302+
```
303+
304+
I don't show the compilation process here as it is a bit complex and will likely be lowered to use a few helper functions in a basic runtime.
83305

84306
### ThisNode
85-
I don't actually think we need to compile this for any real reason instead what we are probably going to be needing todo is just interpret this as a location client side, technically I guess we can assign each class an id. We may skip compiling this all together though do to the oop limiting nature.
307+
We don't actually compile `this` itself but instead we resolve it during compilation so the code:
308+
```java
309+
class Program {
310+
int x;
311+
void Main() {
312+
this.x = 5;
313+
}
314+
}
315+
```
316+
is directly equivalent to:
317+
```java
318+
class Program {
319+
int x;
320+
void Main() {
321+
Program.x = 5;
322+
}
323+
}
324+
```
86325

87-
### IdentiferNode
88-
This is just a subpart of location nodes. Handled by that.
326+
This only works because `this` is only valid in a static context, this approach would completely break with instance properties and methods as `this` would need to refer to the instance rather than the class, but given we don't have instance properties or methods this is a perfectly fine approach.
327+
328+
### LocationNode
329+
Location nodes are probably one of the more complex things to compile as well, we already gave a brief overview of them when discussing assignments. Compiling locations is going to be handled the exact same way as compiling the left hand side of an assignment however `set` will become `get` and `store` will become `load`.
89330

90331
### LiteralNode
91-
These are going to be compiled to either memory allocations or simple constants
332+
Literal nodes are rather simple to compile because they are just constants, exact compilation will depend on the type.
92333

93334
#### Integer
94335
This is going to become a `(i32.const <value>)`
95336

96337
### Character
97-
This is also going to become a `(i32.const <value>)` we will make no distinction between a character and an integer at runtime.
98-
99-
### String
100-
This is only going to be compiled one place the values will go into data sections and we will copy from the data section into memory passing around the pointer.
338+
This is also going to become a `(i32.const <value>)` we will make no distinction between a character and an integer at runtime. The value itself is going to be the unicode scalar value of the character.
101339

102340
### BooleanNode
103341
This is going to compile to a `(i32.const <value>)` with `1` for `true` and `0` for `false`.
104342

105343
### NullNode
106-
This is going to compile to a `(i32.const 0)` most likely though I need to consider this a bit more.
344+
`Null` is an interesting one as it is a special value it probably makes the most sense to compile it into `(i32.const 0)` as this would give it falsey semantics however there would be no distinction between `null` and `false` at runtime, which is a bit unfortunate.
345+
346+
We could also make the decision to not compile this given classes are static there really isn't much use for `null` as nothing can be `null`.
347+
348+
### String
349+
Strings are an interesting case you can think of them as arrays of characters, so we could compile them into linear memory following the same format as arrays. This would be pretty efficient however its worth noting this doesn't follow `utf-8` directly making ffi slightly annoying, we only need strings for callout nodes however so we can just convert them to `utf-8` when we pass them to the callouts if needed.

0 commit comments

Comments
 (0)