Skip to content

PYTHON-5144 - Add async performance benchmarks #2188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 13, 2025
Merged

PYTHON-5144 - Add async performance benchmarks #2188

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d22abeb
PYTHON-5144 - Add async performance benchmarks
NoahStapp Mar 5, 2025
d8f57ac
Address review
NoahStapp Mar 11, 2025
dcbf212
Add serial benchmarks
NoahStapp Mar 11, 2025
4ba8513
Merge branch 'master' into PYTHON-5144
NoahStapp Mar 11, 2025
ab8d3d6
Fix typing
NoahStapp Mar 11, 2025
847e534
Fix config
NoahStapp Mar 11, 2025
2f2536c
Clearer tests
NoahStapp Mar 13, 2025
363cc92
Better concurrent benchmarks
NoahStapp Mar 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 53 additions & 1 deletion .evergreen/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,7 @@ functions:
params:
working_dir: "src"
binary: bash
include_expansions_in_env: [SUB_TEST_NAME]
args:
- .evergreen/scripts/run-with-env.sh
- .evergreen/scripts/run-perf-tests.sh
Expand Down Expand Up @@ -1357,6 +1358,8 @@ tasks:
VERSION: "v6.0-perf"
TOPOLOGY: "server"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "sync"
- func: "attach benchmark test results"
- func: "send dashboard data"

Expand All @@ -1369,6 +1372,8 @@ tasks:
TOPOLOGY: "server"
SSL: "ssl"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "sync"
- func: "attach benchmark test results"
- func: "send dashboard data"

Expand All @@ -1380,9 +1385,52 @@ tasks:
VERSION: "8.0"
TOPOLOGY: "server"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "sync"
- func: "attach benchmark test results"
- func: "send dashboard data"

- name: "perf-6.0-standalone-async"
tags: [ "perf" ]
commands:
- func: "bootstrap mongo-orchestration"
vars:
VERSION: "v6.0-perf"
TOPOLOGY: "server"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "async"
- func: "attach benchmark test results"
- func: "send dashboard data"

- name: "perf-6.0-standalone-ssl-async"
tags: [ "perf" ]
commands:
- func: "bootstrap mongo-orchestration"
vars:
VERSION: "v6.0-perf"
TOPOLOGY: "server"
SSL: "ssl"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "async"
- func: "attach benchmark test results"
- func: "send dashboard data"

- name: "perf-8.0-standalone-async"
tags: [ "perf" ]
commands:
- func: "bootstrap mongo-orchestration"
vars:
VERSION: "8.0"
TOPOLOGY: "server"
- func: "run perf tests"
vars:
SUB_TEST_NAME: "async"
- func: "attach benchmark test results"
- func: "send dashboard data"


- name: "check-import-time"
tags: ["pr"]
commands:
Expand Down Expand Up @@ -1463,8 +1511,12 @@ buildvariants:
- name: "perf-6.0-standalone"
- name: "perf-6.0-standalone-ssl"
- name: "perf-8.0-standalone"
- name: "perf-6.0-standalone-async"
- name: "perf-6.0-standalone-ssl-async"
- name: "perf-8.0-standalone-async"


# Platform notes
# Platform notes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to ask if this was an intentional change but then I noticed all these platform notes are long out of date. Let's remove them?

# i386 builds of OpenSSL or Cyrus SASL are not available
# Debian 8.1 only supports MongoDB 3.4+
# SUSE12 s390x is only supported by MongoDB 3.4+
Expand Down
2 changes: 1 addition & 1 deletion .evergreen/run-perf-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ export OUTPUT_FILE="${PROJECT_DIRECTORY}/results.json"

export PYTHON_BINARY=/opt/mongodbtoolchain/v4/bin/python3

bash ./.evergreen/just.sh setup-tests perf
bash ./.evergreen/just.sh setup-tests perf "${SUB_TEST_NAME}"
bash ./.evergreen/just.sh run-tests
2 changes: 1 addition & 1 deletion .evergreen/scripts/run-perf-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash

PROJECT_DIRECTORY=${PROJECT_DIRECTORY}
bash "${PROJECT_DIRECTORY}"/.evergreen/run-perf-tests.sh
SUB_TEST_NAME=${SUB_TEST_NAME} bash "${PROJECT_DIRECTORY}"/.evergreen/run-perf-tests.sh
7 changes: 5 additions & 2 deletions .evergreen/scripts/setup_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
}

# Tests that require a sub test suite.
SUB_TEST_REQUIRED = ["auth_aws", "kms"]
SUB_TEST_REQUIRED = ["auth_aws", "kms", "perf"]

# Map the test name to test extra.
EXTRAS_MAP = {
Expand Down Expand Up @@ -374,7 +374,10 @@ def handle_test_env() -> None:
if test_name == "perf":
# PYTHON-4769 Run perf_test.py directly otherwise pytest's test collection negatively
# affects the benchmark results.
TEST_ARGS = f"test/performance/perf_test.py {TEST_ARGS}"
if sub_test_name == "sync":
TEST_ARGS = f"test/performance/perf_test.py {TEST_ARGS}"
else:
TEST_ARGS = f"test/performance/async_perf_test.py {TEST_ARGS}"

# Add coverage if requested.
# Only cover CPython. PyPy reports suspiciously low coverage.
Expand Down
Loading