Skip to content

Commit 148930e

Browse files
KarthikSubbaraoKarthikSubbarao
committed
Cleaning up usage of valkey-test-framework (#57)
* Cleaning up usage of valkey-test-framework, small updates to build scripts, and cmd handler Signed-off-by: Karthik Subbarao <[email protected]> * Update tests Signed-off-by: Karthik Subbarao <[email protected]> --------- Signed-off-by: Karthik Subbarao <[email protected]>
1 parent c97e0c6 commit 148930e

20 files changed

+89
-58
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@ env:
88
CARGO_TERM_COLOR: always
99
VALKEY_REPO_URL: https://github.com/valkey-io/valkey.git
1010
TEST_FRAMEWORK_REPO: https://github.com/valkey-io/valkey-test-framework
11-
TEST_FRAMEWORK_DIR: tests/valkeytests
11+
TEST_FRAMEWORK_DIR: tests/build/valkeytestframework
1212

1313
jobs:
1414
build-ubuntu-latest:
1515
runs-on: ubuntu-latest
1616
strategy:
1717
fail-fast: false
1818
matrix:
19-
server_version: ['unstable', '8.0.0']
19+
server_version: ['unstable', '8.0', '8.1']
2020
steps:
2121
- uses: actions/checkout@v4
2222
- name: Set the server verison for python integeration tests
@@ -27,7 +27,7 @@ jobs:
2727
cargo clippy --profile release --all-targets -- -D clippy::all
2828
- name: Release Build
2929
run: |
30-
if [ "${{ matrix.server_version }}" = "8.0.0" ]; then
30+
if [ "${{ matrix.server_version }}" = "8.0" ]; then
3131
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release --features valkey_8_0
3232
else
3333
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release
@@ -36,8 +36,8 @@ jobs:
3636
run: cargo test --features enable-system-alloc
3737
- name: Make valkey-server binary
3838
run: |
39-
mkdir -p "tests/.build/binaries/${{ matrix.server_version }}"
40-
cd tests/.build
39+
mkdir -p "tests/build/binaries/${{ matrix.server_version }}"
40+
cd tests/build
4141
git clone "${{ env.VALKEY_REPO_URL }}"
4242
cd valkey
4343
git checkout ${{ matrix.server_version }}
@@ -81,7 +81,7 @@ jobs:
8181
strategy:
8282
fail-fast: false
8383
matrix:
84-
server_version: ['unstable', '8.0.0']
84+
server_version: ['unstable', '8.0', '8.1']
8585
steps:
8686
- uses: actions/checkout@v4
8787
- name: Set the server verison for python integeration tests
@@ -92,7 +92,7 @@ jobs:
9292
cargo clippy --profile release --all-targets -- -D clippy::all
9393
- name: Release Build
9494
run: |
95-
if [ "${{ matrix.server_version }}" = "8.0.0" ]; then
95+
if [ "${{ matrix.server_version }}" = "8.0" ]; then
9696
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release --features valkey_8_0
9797
else
9898
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release
@@ -101,8 +101,8 @@ jobs:
101101
run: cargo test --features enable-system-alloc
102102
- name: Make Valkey-server binary with asan
103103
run: |
104-
mkdir -p "tests/.build/binaries/${{ matrix.server_version }}"
105-
cd tests/.build
104+
mkdir -p "tests/build/binaries/${{ matrix.server_version }}"
105+
cd tests/build
106106
git clone "${{ env.VALKEY_REPO_URL }}"
107107
cd valkey
108108
git checkout ${{ matrix.server_version }}

‎.gitignore‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
Cargo.lock
22
target
3-
tests/.build
3+
tests/build
44
__pycache__
5+
.pytest_cache
56
test-data
67
.attach_pid*
7-
tests/valkeytests*
8+
src/commands/commands

‎README.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# valkey-bloom
22

3-
Valkey-Bloom (BSD-3-Clause) is a Rust based Valkey-Module which brings a Bloom Filter (Module) data type into Valkey and supports verions >= 8.0. With this, users can create bloom filters (space efficient probabilistic data structures) to add elements, perform “check” operation to test whether an element exists, auto scale their filters, customize bloom filter properties, perform RDB Save and load operations, etc.
3+
Valkey-Bloom (BSD-3-Clause) is a Rust based Valkey-Module which brings a Bloom Filter (Module) data type into Valkey and supports verions >= 8.0. With this, users can create bloom filters (space efficient probabilistic data structures) to add elements, check whether elements exists, auto scale their filters, customize bloom filter properties, perform RDB Save and load operations, etc.
44

55
Valkey-Bloom is built using `bloomfilter::Bloom` (https://crates.io/crates/bloomfilter which has a BSD-2-Clause license).
66

@@ -25,7 +25,10 @@ curl https://sh.rustup.rs -sSf | sh
2525
sudo yum install clang
2626
git clone https://github.com/valkey-io/valkey-bloom.git
2727
cd valkey-bloom
28-
cargo build --all --all-targets --release
28+
# Building for Valkey 8.1 and above:
29+
cargo build --release
30+
# Building for Valkey 8.0 specifically:
31+
cargo build --release --features valkey_8_0
2932
valkey-server --loadmodule ./target/release/libvalkey_bloom.so
3033
```
3134

‎build.sh‎

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -22,29 +22,29 @@ if [ -z "$SERVER_VERSION" ]; then
2222
export SERVER_VERSION="unstable"
2323
fi
2424

25-
if [ "$SERVER_VERSION" != "unstable" ] && [ "$SERVER_VERSION" != "8.0.0" ] ; then
25+
if [ "$SERVER_VERSION" != "unstable" ] && [ "$SERVER_VERSION" != "8.0" ] && [ "$SERVER_VERSION" != "8.1" ]; then
2626
echo "ERROR: Unsupported version - $SERVER_VERSION"
2727
exit 1
2828
fi
2929

3030
echo "Running cargo build release..."
31-
if [ "$SERVER_VERSION" == "8.0.0" ] ; then
31+
if [ "$SERVER_VERSION" == "8.0" ] ; then
3232
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release --features valkey_8_0
3333
else
3434
RUSTFLAGS="-D warnings" cargo build --all --all-targets --release
3535
fi
3636

3737

3838
REPO_URL="https://github.com/valkey-io/valkey.git"
39-
BINARY_PATH="tests/.build/binaries/$SERVER_VERSION/valkey-server"
40-
39+
BINARY_PATH="tests/build/binaries/$SERVER_VERSION/valkey-server"
40+
CACHED_VALKEY_PATH="tests/build/valkey"
4141
if [ -f "$BINARY_PATH" ] && [ -x "$BINARY_PATH" ]; then
4242
echo "valkey-server binary '$BINARY_PATH' found."
4343
else
4444
echo "valkey-server binary '$BINARY_PATH' not found."
45-
mkdir -p "tests/.build/binaries/$SERVER_VERSION"
46-
cd tests/.build
47-
rm -rf valkey
45+
mkdir -p "tests/build/binaries/$SERVER_VERSION"
46+
rm -rf $CACHED_VALKEY_PATH
47+
cd tests/build
4848
git clone "$REPO_URL"
4949
cd valkey
5050
git checkout "$SERVER_VERSION"
@@ -55,17 +55,18 @@ else
5555
make -j
5656
fi
5757
cp src/valkey-server ../binaries/$SERVER_VERSION/
58+
cd $SCRIPT_DIR
59+
rm -rf $CACHED_VALKEY_PATH
5860
fi
5961

6062

6163
TEST_FRAMEWORK_REPO="https://github.com/valkey-io/valkey-test-framework"
62-
TEST_FRAMEWORK_DIR="tests/valkeytests"
64+
TEST_FRAMEWORK_DIR="tests/build/valkeytestframework"
6365

6466
if [ -d "$TEST_FRAMEWORK_DIR" ]; then
65-
echo "valkeytest found."
67+
echo "valkeytestframework found."
6668
else
67-
68-
echo "Cloning test framework..."
69+
echo "Cloning valkey-test-framework..."
6970
git clone "$TEST_FRAMEWORK_REPO"
7071
mkdir -p "$TEST_FRAMEWORK_DIR"
7172
mv "valkey-test-framework/src"/* "$TEST_FRAMEWORK_DIR/"
@@ -128,8 +129,6 @@ if [ ! -z "${ASAN_BUILD}" ]; then
128129
rm test_output.tmp
129130
exit 1
130131
fi
131-
132-
133132
rm test_output.tmp
134133
else
135134
# TEST_PATTERN can be used to run specific tests or test patterns.

‎src/bloom/command_handler.rs‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@ fn handle_bloom_add(
2424
) -> Result<ValkeyValue, ValkeyError> {
2525
match multi {
2626
true => {
27-
let mut result = Vec::new();
28-
for item in args.iter().take(argc).skip(item_idx) {
29-
match bf.add_item(item.as_slice(), validate_size_limit) {
27+
let mut result = Vec::with_capacity(argc - item_idx);
28+
let mut curr_cmd_idx = item_idx;
29+
while curr_cmd_idx < argc {
30+
let item = args[curr_cmd_idx].as_slice();
31+
match bf.add_item(item, validate_size_limit) {
3032
Ok(add_result) => {
3133
if add_result == 1 {
3234
*add_succeeded = true;
@@ -38,6 +40,7 @@ fn handle_bloom_add(
3840
break;
3941
}
4042
};
43+
curr_cmd_idx += 1;
4144
}
4245
Ok(ValkeyValue::Array(result))
4346
}
@@ -299,7 +302,7 @@ pub fn bloom_filter_exists(
299302
let item = input_args[curr_cmd_idx].as_slice();
300303
return Ok(handle_item_exists(value, item));
301304
}
302-
let mut result = Vec::new();
305+
let mut result = Vec::with_capacity(argc - curr_cmd_idx);
303306
while curr_cmd_idx < argc {
304307
let item = input_args[curr_cmd_idx].as_slice();
305308
result.push(handle_item_exists(value, item));

‎src/commands/bf.exists.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@
2222
}
2323
]
2424
}
25-
}
25+
}

‎src/commands/bf.info.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,4 +60,4 @@
6060
}
6161
]
6262
}
63-
}
63+
}

‎tests/conftest.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
import subprocess
22
import pytest
3+
import sys
4+
import os
5+
6+
# Set the path to find and use the valkey-test-framework
7+
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), 'build')))
8+
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), 'build/valkeytestframework')))
39

410
@pytest.fixture(params=['random-seed', 'fixed-seed'])
511
def bloom_config_parameterization(request):

‎tests/test_bloom_acl_category.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
from valkeytests.conftest import resource_port_tracker
1+
from valkeytestframework.conftest import resource_port_tracker
22
from valkey_bloom_test_case import ValkeyBloomTestCaseBase
3-
from util.waiters import *
3+
from valkeytestframework.util.waiters import *
44

55
class TestBloomACLCategory(ValkeyBloomTestCaseBase):
66

‎tests/test_bloom_aofrewrite.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
from util.waiters import *
2-
from valkeytests.valkey_test_case import ValkeyAction
1+
from valkeytestframework.util.waiters import *
2+
from valkeytestframework.valkey_test_case import ValkeyAction
33
from valkey_bloom_test_case import ValkeyBloomTestCaseBase
4-
from valkeytests.conftest import resource_port_tracker
4+
from valkeytestframework.conftest import resource_port_tracker
55

66
class TestBloomAofRewrite(ValkeyBloomTestCaseBase):
77

0 commit comments

Comments
 (0)