Skip to content

Commit 0b491bc

Browse files
fnikolaifnikolai
committed
Bump version
1 parent 0a09344 commit 0b491bc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+826
-41
lines changed

‎VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
v1.0.34
1+
v1.0.35

charts/federated-learning/fedbed/.gitignore renamed to charts/databases/parallax/.gitignore

File renamed without changes.

charts/federated-learning/fedbed/.helmignore renamed to charts/databases/parallax/.helmignore

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,5 +25,5 @@
2525
# Docs
2626
*.md
2727

28-
# Ignore any figures used in the README.md
29-
examples/README.assets
28+
# Tools
29+
build

‎charts/databases/parallax/Chart.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
apiVersion: v2
3+
appVersion: "1.16.0"
4+
description: "Tebis is a rack-scale key/value store optimized for replication over RDMA"
5+
6+
name: tebis
7+
type: application
8+
9+
maintainers:
10+
- name: Fotis Nikolaidis
11+
email: nikolaidis.fotis@gmail.com
12+
url: https://www.linkedin.com/in/fotis-nikolaidis-444a6634/
13+
14+
version: 0.0.0

‎charts/databases/parallax/README.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Tebis
2+
3+
This document focuses on setting up a development environment for the
4+
distributed of Tebis on a local machine.
5+
6+
7+
## Build Tebis Containers
8+
9+
10+
Build the Tebis initialization manager:
11+
12+
```shell
13+
docker build . -t tebis-init -f init.Dockerfile
14+
```
15+
16+
Build the Tebis nodes:
17+
18+
```shell
19+
docker build -t icsforth/tebis-node . -f tebis.Dockerfile
20+
```
21+
22+
23+
## Set up Execution Environment
24+
25+
Tebis uses RDMA for all network communication, which requires support from the
26+
network interface to run. A software implementation (soft-RoCE) exists and can
27+
run on all network interfaces.
28+
29+
30+
### Install dependencies
31+
32+
The `ibverbs-utils` and `rdma-core` packages are required to enable soft-RoCE.
33+
These packages should be in most distirbutions' repositories.
34+
35+
```
36+
apt install ibverbs-utils rdma-core perftest
37+
```
38+
39+
####
40+
41+
### Enabling soft-RoCE
42+
43+
Soft ROCE is a software implementation of RoCE that allows using Infiniband over any ethernet adapter.
44+
45+
46+
ROCE requires the `ethtool` to be installed and the `rdma_ucm` and `uverbs0` modules to be loaded in your system.
47+
48+
```
49+
sudo yum install ethtool
50+
sudo modprobe rdma_rxe rdma_ucm
51+
```
52+
53+
Then, where enp1s0 is the ethernet interface (e.g, eth0, en01, ...)
54+
55+
```
56+
rdma link add rxe0 type rxe netdev enp1s0
57+
```
58+
59+
validate it:
60+
61+
```
62+
>> rdma link
63+
link rxe0/1 state ACTIVE physical_state LINK_UP netdev eno1
64+
```
65+
66+
67+
#### Verify soft-RoCE is working
68+
To verify that soft-RoCE is working, we can run a simple RDMA Write throuhgput
69+
benchmark.
70+
71+
First, open two shells, one to act as the server and one to act as the client.
72+
Then run the following commands:
73+
* On the server: `ib_write_bw`
74+
* On the client: `ib_write_bw eth_interface_ip`, where `eth_interface_ip` is
75+
the IP address of a soft-RoCE enabled ethernet interface.
76+
77+
Example output:
78+
* Server process:
79+
```
80+
************************************
81+
* Waiting for client to connect... *
82+
************************************
83+
---------------------------------------------------------------------------------------
84+
RDMA_Write BW Test
85+
Dual-port : OFF Device : rxe0
86+
Number of qps : 1 Transport type : IB
87+
Connection type : RC Using SRQ : OFF
88+
CQ Moderation : 100
89+
Mtu : 1024[B]
90+
Link type : Ethernet
91+
GID index : 1
92+
Max inline data : 0[B]
93+
rdma_cm QPs : OFF
94+
Data ex. method : Ethernet
95+
---------------------------------------------------------------------------------------
96+
local address: LID 0000 QPN 0x0011 PSN 0x3341fd RKey 0x000204 VAddr 0x007f7e1b8fa000
97+
GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:205
98+
remote address: LID 0000 QPN 0x0012 PSN 0xbfbac5 RKey 0x000308 VAddr 0x007f70f5843000
99+
GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:205
100+
---------------------------------------------------------------------------------------
101+
#bytes #iterations BW peak[MB/sec] BW average[MB/sec] MsgRate[Mpps]
102+
65536 5000 847.44 827.84 0.013245
103+
---------------------------------------------------------------------------------------
104+
```
105+
106+
* Client process:
107+
```
108+
---------------------------------------------------------------------------------------
109+
RDMA_Write BW Test
110+
Dual-port : OFF Device : rxe0
111+
Number of qps : 1 Transport type : IB
112+
Connection type : RC Using SRQ : OFF
113+
TX depth : 128
114+
CQ Moderation : 100
115+
Mtu : 1024[B]
116+
Link type : Ethernet
117+
GID index : 1
118+
Max inline data : 0[B]
119+
rdma_cm QPs : OFF
120+
Data ex. method : Ethernet
121+
---------------------------------------------------------------------------------------
122+
local address: LID 0000 QPN 0x0012 PSN 0xbfbac5 RKey 0x000308 VAddr 0x007f70f5843000
123+
GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:205
124+
remote address: LID 0000 QPN 0x0011 PSN 0x3341fd RKey 0x000204 VAddr 0x007f7e1b8fa000
125+
GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:205
126+
---------------------------------------------------------------------------------------
127+
#bytes #iterations BW peak[MB/sec] BW average[MB/sec] MsgRate[Mpps]
128+
65536 5000 847.44 827.84 0.013245
129+
---------------------------------------------------------------------------------------
130+
```
131+
132+
133+
```shell
134+
server: ibv_rc_pingpong -d rxe0 -g 1
135+
client: ibv_rc_pingpong -d rxe0 -g 1 10.1.128.51
136+
```
137+
138+
139+
### Others
140+
141+
142+
Run rxe_cfg add ethN to configure an RXE instance on ethernet device ethN.
143+
144+
``` shell
145+
You should now have an rxe0 device:
146+
147+
# rxe_cfg status
148+
149+
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
150+
enp1s0 yes virtio_net 1500 192.168.122.211 rxe0 1024 (3)
151+
```
152+
153+
154+
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_infiniband_and_rdma_networks/index
155+
https://support.mellanox.com/s/article/howto-configure-soft-roce
156+
https://github.com/ememos/GiantVM/issues/24
157+
158+
159+
On /etc/security/limits.conf you must add
160+
161+
```
162+
* soft memlock unlimited
163+
* hard memlock unlimited
164+
```
165+
166+
https://bbs.archlinux.org/viewtopic.php?id=273059
167+
168+
https://ask.cyberinfrastructure.org/t/access-to-dev-infiniband-from-user-space/854/2
169+
170+
171+
## Parameters

‎charts/databases/parallax/example.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
apiVersion: frisbee.dev/v1alpha1
3+
kind: Scenario
4+
metadata:
5+
name: parallax
6+
spec:
7+
actions:
8+
- action: Cluster
9+
name: sequential-benchmarks
10+
cluster:
11+
templateRef: parallax.ycsb
12+
schedule:
13+
event:
14+
state: '{{.NumPendingJobs}} == 0 && {{.NumRunningJobs}} == 0'
15+
tolerate:
16+
failedJobs: 35 # number of tests
17+
inputs:
18+
- { test: test_small }
19+
- { test: test_medium }
20+
- { test: test_large }
21+
- { test: test_smalld }
22+
- { test: test_mediumd }
23+
- { test: test_larged }
24+
- { test: test_index_node }
25+
- { test: test_dirty_scans_sd_greater }
26+
- { test: test_dirty_scans_small }
27+
- { test: test_dirty_scans_medium }
28+
- { test: test_dirty_scans_large }
29+
- { test: test_dirty_scans_smalld }
30+
- { test: test_dirty_scans_mediumd }
31+
- { test: test_dirty_scans_larged }
32+
- { test: test_dirty_scans_smallp }
33+
- { test: test_dirty_scans_mediump }
34+
- { test: test_dirty_scans_largep }
35+
- { test: test_dirty_scans_smalldp }
36+
- { test: test_dirty_scans_mediumdp }
37+
- { test: test_dirty_scans_largedp }
38+
- { test: test_options }
39+
- { test: test_categories }
40+
- { test: test_sanitizers }
41+
- { test: test_gc }
42+
- { test: test_medium }
43+
- { test: test_mixes_99_small_1_medium }
44+
- { test: test_mixes_45_small_50_medium_5_big }
45+
- { test: simple_test_delete }
46+
- { test: test_leaf_root_delete_get_scan }
47+
- { test: test_region_allocations }
48+
- { test: test_redo_undo_log }
49+
- { test: test_optional_logging }
50+
- { test: test_par_format }
51+
- { test: test_par_put_serialized }
52+
- { test: test_par_put_metadata }
53+
- { test: tracer}
54+
55+
# Teardown
56+
- action: Delete
57+
name: teardown
58+
depends: { success: [ sequential-benchmarks ] }
59+
delete:
60+
jobs: []

0 commit comments

Comments
 (0)