Skip to content

Commit 261ab36

Browse files
feat: add parameter max_workers to StudioBenchmark.execute
* add parameter max_workers to StudioBenchmark.execute * to allow the control of parallel executions of various runs - needed when too many requests clog the inference API * default set to 10 for simple extension of functionality * apply linter
1 parent aa3a7ea commit 261ab36

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

src/intelligence_layer/evaluation/benchmark/studio_benchmark.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@ def execute(
9898
description: Optional[str] = None,
9999
labels: Optional[set[str]] = None,
100100
metadata: Optional[dict[str, Any]] = None,
101+
max_workers: int = 10,
101102
) -> str:
102103
start = datetime.now(timezone.utc)
103104

@@ -108,7 +109,11 @@ def execute(
108109
f"benchmark-{self.id}-runner",
109110
)
110111
run_overview = runner.run_dataset(
111-
self.dataset_id, description=description, labels=labels, metadata=metadata
112+
self.dataset_id,
113+
description=description,
114+
labels=labels,
115+
metadata=metadata,
116+
max_workers=max_workers,
112117
)
113118

114119
evaluation_overview = self.evaluator.evaluate_runs(

0 commit comments

Comments
 (0)