Skip to content

Commit bf18c6d

Browse files
miloskrizmiloskriz
committed
Add archive node page
1 parent 59cc32c commit bf18c6d

File tree

8 files changed

+327
-8
lines changed

8 files changed

+327
-8
lines changed

‎docs/6-members/5-nodes/2-bootnodes.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Wants=network-online.target
4040
User=polkadot
4141
Group=polkadot
4242
ExecStart=/usr/local/bin/polkadot \
43-
--name MyBootnode_01 \
43+
--name MY_BOOTNODE_01 \
4444
--chain polkadot \
4545
--base-path '/var/lib/polkadot/polkadot1' \
4646
--state-pruning 256 \
@@ -69,11 +69,11 @@ WantedBy=multi-user.target
6969

7070
Please note that this service file makes use of the following flags:
7171

72-
- `--state-pruning`: a typical value of 256 or less is possible for a bootnode, please note that using a value less than 256 does not imply savings of storage space.
73-
- `--sync`: the use of `warp` mode is supported for bootnodes, make use of it and speed up the syncing time!
74-
- `--listen-addr`: to open the relevant p2p ports in the node, here we have created four instances: combining IPv4 & IPv6 and vanilla p2p & Websocket p2p.
75-
- `--public-addr`: this are the addresses that the node will advertise to the network, note that they are not indicated in terms of IPv6 or IPv4 but in DNS addresses instead, and that although the ports match for the vanilla p2p service, they are different for the internal websocket one (ending in `/ws`) and the public secured one (ending in `/wss`).
76-
- `--ws-external`, `--rpc-external`, and `--rpc-methods safe` are used to restrict the type of commands made available in the `rpc` and `ws` ports
72+
- `--state-pruning`: a typical value of *`256`* or less is possible for a bootnode, please note that using a value less than 256 does not imply savings of storage space.
73+
- `--sync`: the use of *`warp`* mode is supported for bootnodes, make use of it and speed up the syncing time!
74+
- `--listen-addr`: to open the relevant p2p ports in the node, here we have created four instances: both IPv4 & IPv6 and both vanilla p2p & Websocket p2p.
75+
- `--public-addr`: these are the addresses that the node will advertise to the network, note that they are not indicated in terms of IPv6 or IPv4 but in DNS addresses instead, and that although the port is a match for the vanilla p2p service, it is different for the internal websocket (ending in *`/ws`*) that is advertised as secured in another port (ending in *`/wss`*).
76+
- `--ws-external`, `--rpc-external`, and `--rpc-methods safe` are used to restrict the type of commands made available in the **rpc** and **ws** ports
7777
- The rest of the flags are there for convenience and performance.
7878

7979
## Configure HAProxy Service
@@ -106,7 +106,7 @@ Known bugs: http://www.haproxy.org/bugs/bugs-2.7.0.html
106106
Running on: Linux 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2023 x86_64
107107
```
108108

109-
By default, HAProxy is not configured to listen on nay ports. In this step, since we are going to configure it as a reverse proxy, we are going to make changes to the default HAProxy configuration.
109+
By default, HAProxy is not configured to listen on any ports. In this step, since we are going to configure it as a reverse proxy, we are going to make changes to the default HAProxy configuration.
110110

111111
```shell
112112
# Make a copy of the current configuration for backup

‎docs/6-members/5-nodes/3-archives.md‎

Lines changed: 320 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,324 @@ sidebar_position: 3
44

55
# Archive Nodes
66

7-
This is a stub
7+
An Archive node stores all past blocks of the relevant blockchain, with complete state available for every block.
88

9+
The main purpose of Archive nodes is to be used by utilities, such as block explorers, wallets, discussion forums, etc. which need access to historical information.
10+
11+
These application often connect amd consult the archive nodes via RPC endpoints and WebSocket connections.
12+
13+
To provide archive services under the Infrastructure Builders' Programme, you will need to cater for the following use cases:
14+
15+
1. http / https requests.
16+
2. ws / wss sessions.
17+
18+
These will be possible by deploying the following solution per required chain:
19+
20+
![Archive node](assets/3-archive-01.png)
21+
22+
In the diagram above, a HAProxy instance will listen to the URL of the RPC service and will connect all the incoming requests to at least two different archive nodes in the backend, this configuration is meant to provide resiliency, throughput and maintainability to the services of the Infrastructure Builders' Programme.
23+
24+
## Configure Archive Service
25+
26+
The configuration of the substrate node (i.e. Polkadot, Kusama, etc) can be adjusted in the systemd service file:
27+
28+
```shell
29+
sudo nano /etc/systemd/system/polkadot1.service
30+
```
31+
32+
and the recommended binary flags are shown as follows:
33+
34+
```systemd title="/etc/systemd/system/polkadot1.service"
35+
[Unit]
36+
Description=Polkadot Node
37+
After=network-online.target
38+
Wants=network-online.target
39+
40+
[Service]
41+
User=polkadot
42+
Group=polkadot
43+
ExecStart=/usr/local/bin/polkadot \
44+
--name MY_ARCHIVE_NODE_01 \
45+
--chain polkadot \
46+
--base-path '/var/lib/polkadot/polkadot1' \
47+
--state-pruning archive \
48+
--block-pruning archive \
49+
--log sync=warn,afg=warn,babe=warn \
50+
--listen-addr /ip6/::/tcp/30333 \
51+
--listen-addr /ip6/::/tcp/30334/ws \
52+
--listen-addr /ip4/0.0.0.0/tcp/30333 \
53+
--listen-addr /ip4/0.0.0.0/tcp/30334/ws \
54+
--public-addr /dns/rpc.mydomain.tld/tcp/30333 \
55+
--public-addr /dns/rpc.mydomain.tld/tcp/30334/ws \
56+
--wasm-execution Compiled \
57+
--execution native-else-wasm \
58+
--no-hardware-benchmarks \
59+
--max-runtime-instances 256 \
60+
--runtime-cache-size 64 \
61+
--rpc-external \
62+
--prometheus-external \
63+
--rpc-methods safe \
64+
--rpc-cors all
65+
66+
Restart=always
67+
RestartSec=120
68+
69+
[Install]
70+
WantedBy=multi-user.target
71+
```
72+
73+
Note that this service file makes use of the following flags:
74+
75+
- `--state-pruning` and `--block-pruning` must both be set to *`archive`* for the node to be able so store the whole blockchain and serve historical information right from the genesis block.
76+
- `--listen-addr`: to open the relevant p2p ports in the node, here we have created four instances: both IPv4 & IPv6 and both vanilla http & websocket endpoints.
77+
- `--public-addr`: these are the addresses that the node will advertise to the network, note that they are not indicated in terms of IPv6 or IPv4 but in DNS addresses instead.
78+
- `--max-runtime-instances` set to `256` and `--runtime-cache-size` set to `64` will prevent the *`Ran out of free WASM instances`* error message occurring under load in the Polkadot and Kusama networks.
79+
- The rest of the flags are there for convenience and performance.
80+
81+
## Configure HAProxy Service
82+
83+
If not already installed, please follow the following commands to install the latest version of HAProxy
84+
85+
```shell
86+
# Install dependencies
87+
sudo apt install software-properties-common
88+
89+
# Add the repository of the haproxy project
90+
sudo add-apt-repository ppa:vbernat/haproxy-2.7
91+
92+
# Update the database of available packages
93+
sudo apt update
94+
95+
# install haproxy
96+
sudo apt install haproxy
97+
98+
# verify the installation
99+
sudo haproxy -v
100+
```
101+
102+
If HAProxy was correctly installed, the output will be similar to this one:
103+
104+
```
105+
HAProxy version 2.7.0-1ppa1~focal 2021/11/26 - https://haproxy.org/
106+
Status: stable branch - will stop receiving fixes around Q3 2023.
107+
Known bugs: http://www.haproxy.org/bugs/bugs-2.7.0.html
108+
Running on: Linux 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2023 x86_64
109+
```
110+
111+
By default, HAProxy is not configured to listen on any ports. In this step, since we are going to configure it as a reverse proxy, we are going to make changes to the default HAProxy configuration.
112+
113+
```shell
114+
# Make a copy of the current configuration for backup
115+
sudo cp -a /etc/haproxy/haproxy.cfg{,.bak}
116+
117+
# Edit the configuration file
118+
sudo nano /etc/haproxy/haproxy.cfg
119+
```
120+
121+
Add the following content to this file, but make sure to use the appropriate user, group, and details for your archive nodes:
122+
123+
```conf title="/etc/haproxy/haproxy.cfg"
124+
global
125+
log 127.0.0.1 local0
126+
chroot /var/lib/haproxy
127+
pidfile /var/run/haproxy.pid
128+
maxconn 250000
129+
user haproxy
130+
group haproxy
131+
daemon
132+
nbthread 8
133+
server-state-base /opt/haproxy-2.6.6/etc/state/
134+
tune.bufsize 131072
135+
tune.ssl.default-dh-param 4096
136+
stats socket /var/run/haproxy.sock mode 600 level admin
137+
stats timeout 2m
138+
maxcompcpuusage 50
139+
140+
defaults
141+
log global
142+
retries 3
143+
# option http-use-htx
144+
maxconn 250000
145+
timeout connect 5s
146+
timeout client 300s
147+
timeout server 300s
148+
timeout queue 25s
149+
150+
####
151+
#
152+
# RELAYCHAIN CONFIGURATIONS
153+
#
154+
####
155+
# Polkadot WSS configuration (for rpc nodes)
156+
157+
frontend relaychains-frontend
158+
bind *:443 ssl crt /etc/pki/certs
159+
mode http
160+
timeout client 300s
161+
162+
# Web sockets
163+
acl wss hdr(Upgrade) -i websocket
164+
165+
# Relay chains
166+
acl polkadot path_beg -i /polkadot
167+
acl kusama path_beg -i /kusama
168+
acl westend path_beg -i /westend
169+
acl westmint path_beg -i /westmint
170+
acl statemine path_beg -i /statemine
171+
acl statemint path_beg -i /statemint
172+
173+
# WSS Paths
174+
use_backend polkadot-wss-backend if polkadot wss
175+
use_backend kusama-wss-backend if kusama wss
176+
use_backend westend-wss-backend if westend wss
177+
use_backend westmint-wss-backend if westmint wss
178+
use_backend statemine-wss-backend if statemine wss
179+
use_backend statemint-wss-backend if statemint wss
180+
181+
# RPC Paths
182+
use_backend polkadot-rpc-backend if polkadot !wss
183+
use_backend kusama-rpc-backend if kusama !wss
184+
use_backend westend-rpc-backend if westend !wss
185+
use_backend westmint-rpc-backend if westmint !wss
186+
use_backend statemine-rpc-backend if statemine !wss
187+
use_backend statemint-rpc-backend if statemint !wss
188+
189+
190+
backend polkadot-rpc-backend
191+
mode http
192+
balance leastconn
193+
server polkadot-rpc-1 10.50.50.110:10000 check inter 2s maxconn 40
194+
server polkadot-rpc-2 10.50.50.111:11000 check inter 2s maxconn 40
195+
server polkadot-rpc-3 10.50.50.112:12000 check inter 2s maxconn 40
196+
server polkadot-rpc-4 10.50.50.113:13000 check inter 2s maxconn 40
197+
198+
backend polkadot-wss-backend
199+
mode http
200+
balance leastconn
201+
server polkadot-rpc-1 10.50.50.110:10001 check inter 2s maxconn 40
202+
server polkadot-rpc-2 10.50.50.111:11001 check inter 2s maxconn 40
203+
server polkadot-rpc-3 10.50.50.112:12001 check inter 2s maxconn 40
204+
server polkadot-rpc-4 10.50.50.113:13001 check inter 2s maxconn 40
205+
206+
backend kusama-rpc-backend
207+
mode http
208+
balance leastconn
209+
server kusama-rpc-1 10.50.50.120:10000 check inter 2s maxconn 40
210+
server kusama-rpc-2 10.50.50.121:11000 check inter 2s maxconn 40
211+
server kusama-rpc-3 10.50.50.122:12000 check inter 2s maxconn 40
212+
server kusama-rpc-4 10.50.50.123:13000 check inter 2s maxconn 40
213+
214+
backend kusama-wss-backend
215+
mode http
216+
balance leastconn
217+
server kusama-rpc-1 10.50.50.120:10001 check inter 2s maxconn 40
218+
server kusama-rpc-2 10.50.50.121:11001 check inter 2s maxconn 40
219+
server kusama-rpc-3 10.50.50.122:12001 check inter 2s maxconn 40
220+
server kusama-rpc-4 10.50.50.123:13001 check inter 2s maxconn 40
221+
222+
backend westend-rpc-backend
223+
mode http
224+
balance leastconn
225+
server westend-rpc-1 10.50.50.130:10000 check inter 2s maxconn 40
226+
server westend-rpc-2 10.50.50.131:11000 check inter 2s maxconn 40
227+
228+
backend westend-wss-backend
229+
mode http
230+
balance leastconn
231+
server westend-rpc-1 10.50.50.130:10001 check inter 2s maxconn 40
232+
server westend-rpc-2 10.50.50.131:11001 check inter 2s maxconn 40
233+
234+
backend westmint-rpc-backend
235+
mode http
236+
balance leastconn
237+
server westmint-rpc-1 10.50.50.140:10000 check inter 2s maxconn 40
238+
239+
backend westmint-wss-backend
240+
mode http
241+
balance leastconn
242+
server westmint-rpc-1 10.50.50.140:10001 check inter 2s maxconn 40
243+
244+
backend statemine-rpc-backend
245+
mode http
246+
balance leastconn
247+
server statemine-rpc-1 10.50.50.150:10000 check inter 2s maxconn 40
248+
249+
backend statemine-wss-backend
250+
mode http
251+
balance leastconn
252+
server statemine-rpc-1 10.50.50.150:10001 check inter 2s maxconn 40
253+
254+
backend statemint-rpc-backend
255+
mode http
256+
balance leastconn
257+
server statemint-rpc-1 10.50.50.160:10000 check inter 2s maxconn 40
258+
259+
backend statemint-wss-backend
260+
mode http
261+
balance leastconn
262+
server statemint-rpc-1 10.50.50.160:10001 check inter 2s maxconn 40
263+
264+
####
265+
# END RELAYCHAIN CONFIGURATIONS
266+
####
267+
```
268+
Now the setup is ready for:
269+
## Restart the Services
270+
271+
Apply the following commands in the shell to start the services.
272+
273+
```shell
274+
# Restart the reverse proxy service
275+
sudo systemctl restart haproxy
276+
277+
# Clear any previous blockchain database, if needed
278+
sudo rm -r /var/lib/polkadot/*
279+
280+
# Restart the bootnode service
281+
sudo systemctl restart polkadot1
282+
```
283+
284+
:::warn
285+
Before removing the old database, be warned that the full syncing of the main relay chains (i.e. Polkadot and Kusama) are taking between 2 to 4 weeks depending on node performance and peer availability.
286+
:::
287+
288+
## Test your Endpoints
289+
290+
Given that your services must cater for both https and websocket users, below is the procedure to test them accordingly
291+
292+
```shell
293+
# Request the following Remote Procedure Call (RPC) to the https endpoint of your node
294+
curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "system_health", "params":[]}' https://rpc.mydomain.tld/polkadot
295+
```
296+
297+
If everything went well, that command should return a response message similar to the one below.
298+
299+
```
300+
{"jsonrpc":"2.0","result":{"peers":20,"isSyncing":false,"shouldHavePeers":true},"id":1}
301+
```
302+
303+
To test the websocket endpoint, the easiest way is to add it to the polkadot.{js} interface and check that the app is connecting to the appropriate chain:
304+
305+
1. Head up to your preferred browser and navigate to https://polkadot.js.org/apps, you will be presented with the initial interface:
306+
307+
![Polkajs test 1](assets/3-archive-02.png)
308+
309+
1. In the top left corner, click to change the default chain/endpoint connected, this will display a menu showing the options, go to the bottom `DEVELOPMENT` field and enter your endpoint (e.g. wss://rpc.mydomain.tld/polkadot):
310+
311+
![Polkajs test 2](assets/3-archive-03.png)
312+
313+
3. The new connection will be attempted and, if successful, you will see the updated app showing blockchain information being populated:
314+
315+
![Polkajs test 3](assets/3-archive-04.png)
316+
317+
4. To obtain a similar information to what was received from the https endpoint, the wss equivalent is shown in the page `Network` -> `Explorer` -> `Node info`:
318+
319+
![Polkajs test 4](assets/3-archive-05.png)
320+
321+
And finally:
322+
323+
## Announce your Services
324+
325+
Let the administrators of the GeoDNS and Anycast services know when your endpoints are up and running so they can add you to the rotation set.
326+
327+
.-

‎docs/6-members/5-nodes/assets/3-archive-01.png‎

46.3 KB
Loading

‎docs/6-members/5-nodes/assets/3-archive-02.png‎

111 KB
Loading

‎docs/6-members/5-nodes/assets/3-archive-03.png‎

94.6 KB
Loading

‎docs/6-members/5-nodes/assets/3-archive-04.png‎

113 KB
Loading

‎docs/6-members/5-nodes/assets/3-archive-05.png‎

32 KB
Loading

‎docs/6-members/8-reporting/assets/1-grafana-02.png‎

714 Bytes
Loading

0 commit comments

Comments
 (0)