Skip to content

Commit ea66b21

Browse files
andrei-marinicaandrei-marinica
committed
sc architecture preliminary diagrams
1 parent cbe2eca commit ea66b21

File tree

2 files changed

+198
-0
lines changed

2 files changed

+198
-0
lines changed

‎docs/developers/architecture/architecture-overview.md‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
---
2+
id: architecture-overview
3+
title: The Big Diagram
4+
---
5+
6+
[comment]: # (mx-abstract)
7+
8+
This section aims to provide an overview of the main production and testing components involving smart contracts.
9+
10+
[comment]: # (mx-context-auto)
11+
12+
## Components overview
13+
14+
```mermaid
15+
graph TD
16+
protocol[Blockchain Protocol]
17+
protocol -->|VM Handler| vm[High-level VM]
18+
vm -->|Blockchain Hooks| protocol
19+
vm -->|Executor Interface| exec[Executor]
20+
exec --> |VM Hooks| vm
21+
```
22+
23+
To understand the architecture of the VM and of the entire contract development and testing infrastructure, a good place to start are the grand components of the system.
24+
25+
Whenever a transaction is processed that contains a smart contract call, the protocol needs call the VM to execute that call and to find the results. There are multiple versions of the VM, and in principle it is possible to also have other VMs with different architectures. For this reason, it is important that the protocol does not have access to the inner workings of the VM.
26+
27+
The only contact point is a pair of interfaces, one for calling the VM, and the other enabling the VM to interrogate the protocol from time to time:
28+
- The one moving _forwards_ is the "VM Handler", with only relevant method `RunSmartContractCall`;
29+
- The one pointing _backwards_ is known as the "Blockchain Hooks", it contains many methods, for instance for retrieving storage, or for interrogating balances.
30+
31+
Moving further down, we will see that the VM itself is split in two:
32+
1. A higher-level component, the high-level VM;
33+
2. The so-called executor.
34+
35+
This separation emphasizes the main responsibilities of the VM:
36+
1. On a high level, the VM needs to handle everything related to smart contract execution: transferring of funds, storage handling, execution context management, and even same-shard calls between smart contracts.
37+
2. On the lower level, it needs to execute smart contract code as fast as possible, and nothing else.
38+
39+
Similar to the layer above, here we also have a pair of interfaces that preserve the separation of components.
40+
- The one moving _forwards_ is the **Executor** interface. The Executor spawns **Instances**, which cache the compiled code and execute it.
41+
- The one pointing _backwards_ is known as **VM Hooks**, and contains a little over 250 methods, which are the only contact between the smart contract itself and the outside world. These VM hooks are used for everything, from retrieving arguments and balances, to pushing results and making calls to other contracts. They are also used to interact with managed types and cryptographic functions, more on them later.
42+
43+
[comment]: # (mx-context-auto)
44+
45+
## Component variation
46+
47+
Having this separation makes it easy to combine different components in various ways, when assembling production or test systems.
48+
49+
The diagram below shows the main options currently available:
50+
51+
```mermaid
52+
graph TD
53+
protocol[Blockchain Protocol] --- vmi
54+
worldmock[World Mock] --- vmi
55+
vmi["VM Handler ↓ ↑ Blockchain Hooks"]
56+
vmi --- vmgo[Go VM] --- execi
57+
vmi --- vmrust[Rust VM] --- execi
58+
vmi --- k[K Framework VM model*]
59+
execi["Executor ↓ ↑ VM Hooks"]
60+
execi --- wasmer2[Wasmer 2]
61+
execi --- wasmer3[Wasmer 3*]
62+
execi --- dbg[Rust Debugger Executor]
63+
execi --- execmock[Go SC Execution Mock*]
64+
```
65+
66+
To simplify, in this diagram the interfaces are represented as blocks instead of arrows, connected to them are the suitable existing components.
67+
68+
This diagram also ignores the added trouble of not having all these components written in the same language. Some are Go and some Rust. More on this later.
69+
70+
We will say a few things about all. On the protocol level we have:
71+
- The actual **protocol** running in production.
72+
- **World Mock** - a simplified representation of the state of the protocol, to be used in integration tests. This allows us to run various scenarios locally without having to spin an entire blockchain. There is both a Go and a Rust implementation.
73+
74+
The VMs are as follows:
75+
- The **Go VM** is the only VM currently in use in production. Using it in tests is the closest to the actual execution on the blockchain.
76+
- The **Rust VM** is currently only used for debugging and is not fully featured. By the time of writing this it can only be used in conjuction with the Rust Debugger Executor, but there are plans to make it work with Wasmer and thus bring the debugging experience closer to the real execution.
77+
- The **K Framework VM** is an executable specification of the VM. It contains the logic for both the high-level VM and the executor. It is not yet fully integrated into the MultiversX stack, but once it gets there it will allow symbolic execution, low-level coverage, fuzzing and many other tools.
78+
79+
There are several available executors:
80+
- **Wasmer 2.2** is the only executor currently available in production. To be more precise, the executor itself is a wrapper around the Wasmer 2.2 execution engine. Wasmer compiles and executes WebAssembly code, in our case the compiled smart contracts. This entire component is written completely in Rust.
81+
- **Wasmer 3** is the future upgrade of the production executor. There is no timeline yet for when it will be integrated.
82+
- The **Rust Debugger Executor** is the mechanism through which it is possible to test smart contracts without first compiling them to WebAssembly. It plugs smart contract Rust code directly to the Executor and VM hooks interfaces. This enables step by step debugging in Rust code directly, as well as Rust high-level code coverage.
83+
- There is also a **Go SC Execution Mock**, used for some integration tests in the VM, where performing the full WebAssembly execution would simply overcomplicate the tests. It is currently not directly tied to the executor interface, but there are plans to better integrate it.
84+
85+
86+
[comment]: # (mx-context-auto)
87+
88+
## Executing smart contracts
89+
90+
All until here refers exclusively to the VM and its low level execution. But now let's have a look at high level contracts and how they relate to this system.
91+
92+
93+
```mermaid
94+
graph TD
95+
protocol[Protocol] --- vmi
96+
vmi["VM Handler ↓ ↑ Blockchain Hooks"]
97+
vmi --- execi
98+
execi["Executor ↓ ↑ VM Hooks"]
99+
execi --- wasmer2[Wasmer]
100+
execi --- debugger-exec[Rust Debugger Executor]
101+
api["Endpoints ↓ ↑ SC API"]
102+
wasmer2 --- binary[SC .wasm binary]
103+
binary --- wasm-api[Wasm API]
104+
wasm-api --- api
105+
debugger-exec ---- debugger-api[Debugger API]
106+
debugger-api --- api
107+
api --- sc[Smart Contract]
108+
```
109+
110+
MultiversX smart contracts are written in Rust, using the `multiversx-sc` framework. Contracts need to be compiled to WebAssembly, so they can be executed by Wasmer, as part of the protocol.
111+
112+
However, there are other ways to execute a smart contract, and for this reason the actual build is abstracted away from the high-level SC code. The SC library itself (which contains the logic) is completely detached from the build system. It can only call a standard API, without knowing about the concrete implementation behind it.
113+
114+
There are several API implementations, but the most important are the WebAssemby API (crate `multiversx-wasm-adapter`) and the debugger API (in crate `multiversx-scenario`). The first is used for generating the necessary code to build the WebAssembly binary, the other one is able to connect to the debugging engine directly, in order to run tests, coverage, and to, well, debug.
115+
116+
117+
118+
119+
120+
121+
122+
123+
124+
125+
126+
127+
128+
```mermaid
129+
graph TD
130+
protocol[Protocol] --- vmi
131+
vmi["VM Handler ↓ ↑ Blockchain Hooks"]
132+
vmi --- execi
133+
execi["Executor ↓ ↑ VM Hooks"]
134+
execi --- wasmer2[Wasmer 2]
135+
execi --- debugger-exec[Rust Debugger Executor]
136+
wasmer2 --- binary
137+
binary[SC .wasm binary]
138+
binary --> endpoints[SC endpoints]
139+
debugger-exec --> func-selector[Function selector]
140+
func-selector --> sc
141+
sc-api[Smart Contract API]
142+
endpoints --> sc[Smart Contract]
143+
wasm-api --> binary
144+
wasm-api[Wasm API]
145+
sc-api --> wasm-api
146+
sc-api --> debugger-api[Debugger API]
147+
debugger-api --> debugger-exec
148+
sc[Smart Contract] --> sc-api
149+
150+
```
151+
152+
153+
154+
[comment]: # (mx-context-auto)
155+
156+
## Developer documentation
157+
158+
```mermaid
159+
graph TD
160+
Node ---> common[vm-common]
161+
common ----> vm-go[VM Go]
162+
vm-go ---> exec-go
163+
exec-go --> wasmer1[Wasmer 1]
164+
wasmer1 -->|endpoints| compiled
165+
exec-go[Executor Go] --> execcapigo[Rust Adapter ''Wasmer 2'']
166+
execcapigo -->|C API| execcapirs[Executor C API Rust]
167+
execcapirs --> exec-rs[Executor Rust + VMHooks interface]
168+
exec-rs --> wasmer2adapt[Wasmer 2 Adapter]
169+
wasmer2adapt --> wasmer2[Wasmer 2]
170+
wasmer2 -->|endpoints| compiled[Compiled 'contract.wasm']
171+
compiled -->|endpoints| contract-wasm[Contract wasm crate, auto-generated]
172+
contract-wasm -->|endpoints| contract-lib[Contract]
173+
contract-container -->|auto-gen function selector| contract-lib
174+
vm-rs[VM Rust]
175+
debugger[Framework: scenario debugger]
176+
vm-rs --> exec-rs
177+
exec-rs --> contract-container
178+
debugger --> vm-rs
179+
scenario-go[Scenario Go] ----> vm-go
180+
scenario-rs[Scenario Rust] --> debugger
181+
scen[Scenario 'x.scen.json']
182+
scen --> scenario-go
183+
scen --> scenario-rs
184+
intg-test-rs[Integragration test Rust]
185+
intg-test-rs --> scen
186+
intg-test-rs --> scenario-rs
187+
intg-test-rs --> whitebox[White Box]
188+
whitebox --> debugger
189+
```
190+
191+

‎sidebars.js‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,13 @@ const sidebars = {
123123
"developers/scenario-reference/embed",
124124
],
125125
},
126+
{
127+
type: "category",
128+
label: "Architecture",
129+
items: [
130+
"developers/architecture/architecture-overview",
131+
],
132+
},
126133
],
127134
},
128135
{

0 commit comments

Comments
 (0)