Skip to content

Commit b5e8b19

Browse files
committed
Add RocksDB
Add scripts to run ycsb on rocksdb and update readme
1 parent 5ce2ae4 commit b5e8b19

File tree

1,513 files changed

+502275
-4
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,513 files changed

+502275
-4
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ SplitFS is under active development.
124124
5. rsync
125125
6. Filebench
126126
7. LMDB
127+
8. RocksDB (with YCSB)
127128

128129
## Testing
129130
[PJD POSIX Test Suite](https://www.tuxera.com/community/posix-test-suite/) that tests primarily the metadata operations was run on SplitFS successfully. SplitFS passes all tests.

dependencies/rocskdb_deps.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
3+
sudo apt-get update
4+
5+
sudo apt-get install libgflags-dev
6+
7+
sudo apt-get install libsnappy-dev
8+
9+
sudo apt-get install zlib1g-dev
10+
11+
sudo apt-get install libbz2-dev
12+
13+
sudo apt-get install liblz4-dev
14+
15+
sudo apt-get install libzstd-dev

experiments.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@ We evaluate and benchmark on SplitFS using different application benchmarks like
1818
* `$ export PATH=$PATH:$JAVA_HOME/bin`
1919
* Check installation using `java -version`
2020
* `$ sudo apt install maven`
21-
21+
3. RocksDB: Upgrade your gcc to version at least 4.8 to get C++11 support. For ubuntu, please run `cd dependencies ./rocksdb_deps.sh; cd..`
22+
If you face any dependency issues, please refer the [doc](https://github.com/utsaslab/SplitFS/blob/master/rocksdb/INSTALL.md#dependencies)
2223
---
2324

2425
### Kernel Setup
@@ -55,7 +56,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
5556

5657
### Workload Generation
5758

58-
1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB, because YCSB does not natively support LevelDB, and has been added to the benchmarks of LevelDB
59+
1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB & RocksDB, because YCSB does not natively support LevelDB & RocksDB(C++), and has been added to the benchmarks of LevelDB & RocksDB
5960
2. TPCC: `cd scripts/tpcc; ./gen_workload.sh; cd ../..` -- This will create an initial database on SQLite on which to run the TPCC workload
6061
3. rsync: `cd scripts/rsync/; sudo ./rsync_gen_workload.sh; cd ../..` -- This will create the rsync workload according to the backup data distribution as mentioned in the paper
6162
4. tar: `cd scripts/tar/; sudo ./gen_workload.sh; cd ../..` -- This will create the tar workload as mentioned in the paper
@@ -65,7 +66,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
6566

6667
### Run Application Workloads
6768

68-
1. YCSB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
69+
1. YCSB-LevelDB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
6970
2. TPCC: `cd scripts/tpcc; ./run_tpcc.sh; cd ../..` -- This will run the TPCC workload on SQLite3 with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX`
7071
3. rsync: `cd scripts/rsync; ./run_rsync.sh; cd ../..` -- This will run the rsync workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-sync`
7172
4. tar: `cd scripts/tar; ./run_tar.sh; cd ../..` -- This will run the tar workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX, SplitFS-sync, SplitFS-strict`
@@ -78,7 +79,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
7879
7. Filebench: `cd scripts/filebench; ./run_filebench.sh; cd ../..` --
7980
This will run the varmail workload with `ext4-DAX, NOVA strict,
8081
NOVA Relaxed, PMFS, SplitFS-POSIX`
81-
82+
8. YCSB-RocksDB: `cd scripts/ycsb_rocksdb; ./run_ycsb_rocksdb.sh; cd ../..` -- This will run all the YCSB workloads on RocksDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS (based on the one found in splitfs/libnvp.so)`
8283
---
8384

8485
### Run Software Overhead Workloads

rocksdb/AUTHORS

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
Facebook Inc.
2+
Facebook Engineering Team
3+
4+
Google Inc.
5+
# Initial version authors:
6+
Jeffrey Dean <[email protected]>
7+
Sanjay Ghemawat <[email protected]>
8+
9+
# Partial list of contributors:
10+
Kevin Regan <[email protected]>
11+
Johan Bilien <[email protected]>
12+
Matthew Von-Maszewski <https://github.com/matthewvon> (Basho Technologies)

0 commit comments

Comments
 (0)