Skip to content

Commit aca52cd

Browse files
committed
Add RocksDB
Add scripts to run ycsb on rocksdb and update readme
1 parent 28866c9 commit aca52cd

File tree

1,513 files changed

+502275
-3
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,513 files changed

+502275
-3
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@ SplitFS is under active development.
126126
6. Filebench
127127
7. LMDB
128128
8. FIO
129+
9. RocksDB (with YCSB)
129130

130131
## Testing
131132
[PJD POSIX Test Suite](https://www.tuxera.com/community/posix-test-suite/) that tests primarily the metadata operations was run on SplitFS successfully. SplitFS passes all tests.

dependencies/rocskdb_deps.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
3+
sudo apt-get update
4+
5+
sudo apt-get install libgflags-dev
6+
7+
sudo apt-get install libsnappy-dev
8+
9+
sudo apt-get install zlib1g-dev
10+
11+
sudo apt-get install libbz2-dev
12+
13+
sudo apt-get install liblz4-dev
14+
15+
sudo apt-get install libzstd-dev

experiments.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@ We evaluate and benchmark on SplitFS using different application benchmarks like
1818
* `$ export PATH=$PATH:$JAVA_HOME/bin`
1919
* Check installation using `java -version`
2020
* `$ sudo apt install maven`
21-
21+
3. RocksDB: Upgrade your gcc to version at least 4.8 to get C++11 support. For ubuntu, please run `cd dependencies ./rocksdb_deps.sh; cd..`
22+
If you face any dependency issues, please refer the [doc](https://github.com/utsaslab/SplitFS/blob/master/rocksdb/INSTALL.md#dependencies)
2223
---
2324

2425
### Kernel Setup
@@ -56,7 +57,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
5657

5758
### Workload Generation
5859

59-
1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB, because YCSB does not natively support LevelDB, and has been added to the benchmarks of LevelDB
60+
1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB & RocksDB, because YCSB does not natively support LevelDB & RocksDB(C++), and has been added to the benchmarks of LevelDB & RocksDB
6061
2. TPCC: `cd scripts/tpcc; ./gen_workload.sh; cd ../..` -- This will create an initial database on SQLite on which to run the TPCC workload
6162
3. rsync: `cd scripts/rsync/; sudo ./rsync_gen_workload.sh; cd ../..` -- This will create the rsync workload according to the backup data distribution as mentioned in the paper
6263
4. tar: `cd scripts/tar/; sudo ./gen_workload.sh; cd ../..` -- This will create the tar workload as mentioned in the paper
@@ -66,7 +67,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
6667

6768
### Run Application Workloads
6869

69-
1. YCSB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
70+
1. YCSB-LevelDB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
7071
2. TPCC: `cd scripts/tpcc; ./run_tpcc.sh; cd ../..` -- This will run the TPCC workload on SQLite3 with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX`
7172
3. rsync: `cd scripts/rsync; ./run_rsync.sh; cd ../..` -- This will run the rsync workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-sync`
7273
4. tar: `cd scripts/tar; ./run_tar.sh; cd ../..` -- This will run the tar workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX, SplitFS-sync, SplitFS-strict`
@@ -81,6 +82,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
8182
NOVA Relaxed, PMFS, SplitFS-POSIX`
8283
8. FIO: `cd scripts/fio; ./run_fio.sh; cd ../..` --
8384
This will run the random read-write workload with 50:50 reads and writes with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX`
85+
9. YCSB-RocksDB: `cd scripts/ycsb_rocksdb; ./run_ycsb_rocksdb.sh; cd ../..` -- This will run all the YCSB workloads on RocksDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS (based on the one found in splitfs/libnvp.so)`
8486

8587
---
8688

rocksdb/AUTHORS

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
Facebook Inc.
2+
Facebook Engineering Team
3+
4+
Google Inc.
5+
# Initial version authors:
6+
Jeffrey Dean <[email protected]>
7+
Sanjay Ghemawat <[email protected]>
8+
9+
# Partial list of contributors:
10+
Kevin Regan <[email protected]>
11+
Johan Bilien <[email protected]>
12+
Matthew Von-Maszewski <https://github.com/matthewvon> (Basho Technologies)

0 commit comments

Comments
 (0)