Adding --test-time to memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values-pipeline-10.yml (#100)

* delete the stopped container by default at tear down

* fix test-time on 4 test-cases.

* removed the -n parameter, since it was incompatible with --test-time and was causing the benchmark to fail

* adding a test-time=180 to testcase memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values-pipeline-10.yml

* Re-order --preserve_temporary_client_dirs --client_aggregated_results_folder so that we can create an aggregated folder and still delete the temp files after. In addition, delete temp JSON files created by redis-benchmark (and not by memtier-benchmark)

Co-authored-by: root <[email protected]>
Co-authored-by: Martin Dimitrov <[email protected]>
Co-authored-by: filipe oliveira <[email protected]>

Author: 4 people

redis_benchmarks_specification/__runner__/runner.py

@@ -506,17 +506,6 @@ def process_self_contained_coordinator_stream(
                             )
                             pass
 
-                if preserve_temporary_client_dirs is True:
-                    logging.info(
-                        "Preserving temporary client dir {}".format(
-                            temporary_dir_client
-                        )
-                    )
-                else:
-                    logging.info(
-                        "Removing temporary client dir {}".format(temporary_dir_client)
-                    )
-                    shutil.rmtree(temporary_dir_client, ignore_errors=True)
                 if client_aggregated_results_folder != "":
                     os.makedirs(client_aggregated_results_folder, exist_ok=True)
                     dest_fpath = "{}/{}".format(
@@ -531,6 +520,23 @@ def process_self_contained_coordinator_stream(
                     shutil.copy(full_result_path, dest_fpath)
                 overall_result &= test_result
 
+                if preserve_temporary_client_dirs is True:
+                    logging.info(
+                        "Preserving temporary client dir {}".format(
+                            temporary_dir_client
+                        )
+                    )
+                else:
+                    if "redis-benchmark" in benchmark_tool:
+                        os.remove(full_result_path)
+                        logging.info(
+                            "Removing temporary JSON file".format(full_result_path)
+                        )
+                    shutil.rmtree(temporary_dir_client, ignore_errors=True)
+                    logging.info(
+                        "Removing temporary client dir {}".format(temporary_dir_client)
+                    )
+
     table_name = "Results for entire test-suite".format(test_name)
     results_matrix_headers = [
         "Test Name",

redis_benchmarks_specification/test-suites/memtier_benchmark-10Mkeys-load-hash-5-fields-with-100B-values-pipeline-10.yml

@@ -17,7 +17,7 @@ build-variants:
 clientconfig:
   run_image: redislabs/memtier_benchmark:edge
   tool: memtier_benchmark
-  arguments: '--test-time 180 "--pipeline" "10" "--data-size" "100" --command "HSET __key__ field1 __data__ field2 __data__ field3 __data__ field4 __data__ field5 __data__" "--command-key-pattern" "P" --key-minimum=1 --key-maximum 10000000 -c 50 -t 4 --hide-histogram'
+  arguments: '"--pipeline" "10" "--data-size" "100" --command "HSET __key__ field1 __data__ field2 __data__ field3 __data__ field4 __data__ field5 __data__" "--command-key-pattern" "P" --key-minimum=1 --key-maximum 10000000 -c 50 -t 4 --hide-histogram --test-time=180'
   resources:
     requests:
       cpus: "4"