Skip to content

benchmarks: Add a read all available benchmark #883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bkeryan merged 2 commits into from
Nov 10, 2025
Merged

benchmarks: Add a read all available benchmark #883

bkeryan merged 2 commits into from
Nov 10, 2025

Conversation

@bkeryan
Copy link
Collaborator

@bkeryan bkeryan commented Nov 7, 2025

  • This contribution adheres to CONTRIBUTING.md.
  • I've updated CHANGELOG.md if applicable.
  • I've added tests applicable for this pull request

What does this Pull Request accomplish?

Add AI read benchmarks for READ_ALL_AVAILABLE

Why should this Pull Request be merged?

I want to understand the effect that #881 has on performance.

What testing has been done?

Ran new test case along with read_many_sample.

@bkeryan bkeryan requested a review from mikeprosserni November 7, 2025 23:35
@bkeryan
Copy link
Collaborator Author

bkeryan commented Nov 7, 2025

Results with master branch (not Mike's PR) and a recent DAQmx 26.0 dev build:

PS D:\dev\nidaqmx-python> poetry run pytest -v .\tests\benchmark\test_analog_stream_readers.py -k "library and (read_all_available or read_many_sample)"
================================================= test session starts =================================================
platform win32 -- Python 3.11.7, pytest-8.4.2, pluggy-1.6.0 -- D:\dev\nidaqmx-python\.venv\Scripts\python.exe
cachedir: .pytest_cache
benchmark: 5.2.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: D:\dev\nidaqmx-python
configfile: pyproject.toml
plugins: benchmark-5.2.0, cov-7.0.0, mock-3.15.1
collected 88 items / 72 deselected / 16 selected

tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1] PASSED [  6%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000] PASSED [ 12%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1] PASSED [ 18%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000] PASSED [ 25%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1] PASSED [ 31%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2] PASSED [ 37%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8] PASSED [ 43%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1] PASSED [ 50%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2] PASSED [ 56%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8] PASSED [ 62%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1] PASSED [ 68%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2] PASSED [ 75%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8] PASSED [ 81%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1] PASSED [ 87%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2] PASSED [ 93%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8] PASSED [100%]


-------------------------------------------------------------------------------------------------------------- benchmark 'analog_readers': 16 tests --------------------------------------------------------------------------------------------------------------
Name (time in us)                                                                           Min                    Max               Mean              StdDev             Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1]          21.2000 (1.0)         393.1000 (1.89)     23.2368 (1.00)       7.4830 (1.10)     22.2000 (1.0)      0.4000 (1.33)      319;906       43.0351 (1.00)      10823           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2]          21.3000 (1.00)     23,171.5000 (111.51)   25.4475 (1.10)     218.6189 (32.14)    22.3000 (1.00)     0.4001 (1.33)        1;986       39.2966 (0.91)      11224           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8]          21.5000 (1.01)        500.9000 (2.41)     23.3722 (1.01)      10.3166 (1.52)     22.5000 (1.01)     0.5000 (1.67)      119;676       42.7858 (0.99)      10742           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1]           21.7000 (1.02)        624.0000 (3.00)     23.1948 (1.0)       12.5270 (1.84)     22.4000 (1.01)     0.3000 (1.0)        15;285       43.1130 (1.0)        2699           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1]       24.2000 (1.14)        535.9000 (2.58)     25.7000 (1.11)       8.1298 (1.20)     25.0000 (1.13)     0.5000 (1.67)       87;829       38.9105 (0.90)       8636           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000]        24.2000 (1.14)        356.0000 (1.71)     30.2948 (1.31)      13.2424 (1.95)     25.0000 (1.13)     0.9000 (3.00)    1576;2080       33.0089 (0.77)       9991           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2]       25.4000 (1.20)        405.7000 (1.95)     27.0135 (1.16)       7.6253 (1.12)     26.2000 (1.18)     0.5000 (1.67)      120;585       37.0185 (0.86)       8584           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8]       32.0000 (1.51)        434.8000 (2.09)     34.2159 (1.48)       8.7356 (1.28)     32.9000 (1.48)     0.6000 (2.00)     165;1052       29.2261 (0.68)       9268           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2]        37.4001 (1.76)        382.1000 (1.84)     40.1026 (1.73)       9.5643 (1.41)     38.8000 (1.75)     0.7000 (2.33)      168;619       24.9360 (0.58)       7542           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1]         37.6000 (1.77)        554.2000 (2.67)     40.4357 (1.74)      12.2701 (1.80)     38.6999 (1.74)     0.8000 (2.67)      132;485       24.7306 (0.57)       5033           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1]        37.6001 (1.77)        207.8000 (1.0)      39.7019 (1.71)       6.8804 (1.01)     38.8000 (1.75)     0.6999 (2.33)      153;535       25.1877 (0.58)       7548           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8]        37.8000 (1.78)        966.0000 (4.65)     41.4699 (1.79)      19.8548 (2.92)     39.1000 (1.76)     0.7000 (2.33)      199;673       24.1139 (0.56)       7231           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1]     42.7000 (2.01)        363.8000 (1.75)     45.7289 (1.97)       9.2563 (1.36)     44.0000 (1.98)     0.7000 (2.33)      242;770       21.8680 (0.51)       7033           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000]      42.7000 (2.01)        740.5000 (3.56)     51.0615 (2.20)      21.4029 (3.15)     43.9000 (1.98)     1.2000 (4.00)      689;926       19.5842 (0.45)       4944           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2]     44.6000 (2.10)        228.5000 (1.10)     47.1613 (2.03)       6.8030 (1.0)      46.2000 (2.08)     0.7000 (2.33)      221;592       21.2038 (0.49)       6911           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8]     57.2000 (2.70)        532.5000 (2.56)     59.5537 (2.57)      10.5398 (1.55)     58.4000 (2.63)     0.8000 (2.67)       96;520       16.7916 (0.39)       6132           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
========================================== 16 passed, 72 deselected in 6.92s ==========================================
PS D:\dev\nidaqmx-python> poetry run pytest -v .\tests\benchmark\test_analog_stream_readers.py -k "grpc and (read_all_available or read_many_sample)"
============================================================================================================================ test session starts ============================================================================================================================
platform win32 -- Python 3.11.7, pytest-8.4.2, pluggy-1.6.0 -- D:\dev\nidaqmx-python\.venv\Scripts\python.exe
cachedir: .pytest_cache
benchmark: 5.2.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: D:\dev\nidaqmx-python
configfile: pyproject.toml
plugins: benchmark-5.2.0, cov-7.0.0, mock-3.15.1
collected 88 items / 72 deselected / 16 selected

tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[grpc_init_kwargs-1] PASSED                                                                                                                                       [  6%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[grpc_init_kwargs-1000] PASSED                                                                                                                                    [ 12%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[grpc_init_kwargs-1] PASSED                                                                                                                                     [ 18%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[grpc_init_kwargs-1000] PASSED                                                                                                                                  [ 25%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-1] PASSED                                                                                                                                      [ 31%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-2] PASSED                                                                                                                                      [ 37%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-8] PASSED                                                                                                                                      [ 43%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-1] PASSED                                                                                                                                   [ 50%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-2] PASSED                                                                                                                                   [ 56%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-8] PASSED                                                                                                                                   [ 62%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-1] PASSED                                                                                                                                    [ 68%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-2] PASSED                                                                                                                                    [ 75%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-8] PASSED                                                                                                                                    [ 81%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-1] PASSED                                                                                                                                 [ 87%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-2] PASSED                                                                                                                                 [ 93%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-8] PASSED                                                                                                                                 [100%]


----------------------------------------------------------------------------------------------------------------- benchmark 'analog_readers': 16 tests ----------------------------------------------------------------------------------------------------------------
Name (time in us)                                                                           Min                   Max                  Mean              StdDev                Median                 IQR            Outliers         OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-8]            643.4000 (1.0)      2,640.4000 (1.10)       709.7314 (1.0)       79.5146 (1.11)       700.9500 (1.0)       20.7999 (1.29)        13;67  1,408.9837 (1.0)        1178           1
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-1]            644.2000 (1.00)     2,481.1000 (1.03)       723.0961 (1.02)      73.3083 (1.02)       708.0500 (1.01)      40.0000 (2.48)        75;72  1,382.9421 (0.98)       1176           1
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1-2]            653.2000 (1.02)     2,975.0000 (1.23)       711.5399 (1.00)      85.2178 (1.18)       702.9000 (1.00)      16.1250 (1.0)          8;86  1,405.4026 (1.00)       1269           1
test___analog_single_channel_reader___read_many_sample[grpc_init_kwargs-1]             660.1000 (1.03)     2,410.3000 (1.0)        736.7359 (1.04)      78.1365 (1.09)       729.3000 (1.04)      28.1750 (1.75)       33;110  1,357.3384 (0.96)        707           1
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-1]         761.4000 (1.18)     2,813.2000 (1.17)       812.3932 (1.14)      71.9302 (1.0)        803.4000 (1.15)      16.9000 (1.05)        11;67  1,230.9310 (0.87)        930           1
test___analog_single_channel_reader___read_many_sample[grpc_init_kwargs-1000]          763.6000 (1.19)     2,758.4000 (1.14)       833.8869 (1.17)     107.9822 (1.50)       816.9000 (1.17)      32.9500 (2.04)        33;76  1,199.2034 (0.85)        823           1
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-2]         833.4000 (1.30)     2,888.6000 (1.20)       894.8746 (1.26)      72.1170 (1.00)       888.0000 (1.27)      23.4000 (1.45)        13;50  1,117.4750 (0.79)        866           1
test___analog_multi_channel_reader___read_many_sample[grpc_init_kwargs-1000-8]       1,294.5000 (2.01)     3,265.6000 (1.35)     1,367.5057 (1.93)     104.2994 (1.45)     1,349.9000 (1.93)      24.8000 (1.54)        14;53    731.2584 (0.52)        601           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-8]        1,575.8000 (2.45)     3,756.3000 (1.56)     1,704.5850 (2.40)     130.0080 (1.81)     1,686.7000 (2.41)      31.5250 (1.96)         8;45    586.6531 (0.42)        533           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-2]        1,577.3000 (2.45)     3,744.3000 (1.55)     1,715.9493 (2.42)     133.5344 (1.86)     1,699.3500 (2.42)      40.9000 (2.54)        12;25    582.7678 (0.41)        540           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1-1]        1,586.6000 (2.47)     3,853.2000 (1.60)     1,748.6520 (2.46)     137.1975 (1.91)     1,711.5500 (2.44)      85.8500 (5.32)        39;31    571.8691 (0.41)        544           1
test___analog_single_channel_reader___read_all_available[grpc_init_kwargs-1]         1,610.7000 (2.50)     3,525.2000 (1.46)     1,705.1611 (2.40)      93.2523 (1.30)     1,691.8000 (2.41)      41.5750 (2.58)        16;21    586.4549 (0.42)        465           1
test___analog_single_channel_reader___read_all_available[grpc_init_kwargs-1000]      1,769.0000 (2.75)     3,941.3000 (1.64)     1,910.6388 (2.69)     138.9746 (1.93)     1,890.8500 (2.70)      46.4000 (2.88)        12;34    523.3852 (0.37)        436           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-1]     1,832.3000 (2.85)     4,100.0000 (1.70)     1,916.7978 (2.70)     146.2169 (2.03)     1,901.4000 (2.71)      34.5000 (2.14)         4;29    521.7034 (0.37)        463           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-2]     1,976.4000 (3.07)     3,885.3000 (1.61)     2,088.5548 (2.94)     105.4833 (1.47)     2,073.1000 (2.96)      36.9000 (2.29)        12;28    478.8000 (0.34)        416           1
test___analog_multi_channel_reader___read_all_available[grpc_init_kwargs-1000-8]     2,839.7000 (4.41)     4,124.0000 (1.71)     3,067.4762 (4.32)     182.5912 (2.54)     3,006.6000 (4.29)     109.9000 (6.82)        35;32    326.0009 (0.23)        302           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
==================================================================================================================== 16 passed, 72 deselected in 16.12s =====================================================================================================================

@bkeryan
Copy link
Collaborator Author

bkeryan commented Nov 7, 2025

Results with #881 and a recent DAQmx 26.0.0 dev build

PS D:\dev\nidaqmx-python> poetry run pytest -v .\tests\benchmark\test_analog_stream_readers.py -k "library and (read_all_available or read_many_sample)"
============================================================================================================================ test session starts ============================================================================================================================
platform win32 -- Python 3.11.7, pytest-8.4.2, pluggy-1.6.0 -- D:\dev\nidaqmx-python\.venv\Scripts\python.exe
cachedir: .pytest_cache
benchmark: 5.2.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: D:\dev\nidaqmx-python
configfile: pyproject.toml
plugins: benchmark-5.2.0, cov-7.0.0, mock-3.15.1
collected 88 items / 72 deselected / 16 selected

tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1] PASSED                                                                                                                                    [  6%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000] PASSED                                                                                                                                 [ 12%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1] PASSED                                                                                                                                  [ 18%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000] PASSED                                                                                                                               [ 25%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1] PASSED                                                                                                                                   [ 31%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2] PASSED                                                                                                                                   [ 37%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8] PASSED                                                                                                                                   [ 43%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1] PASSED                                                                                                                                [ 50%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2] PASSED                                                                                                                                [ 56%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8] PASSED                                                                                                                                [ 62%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1] PASSED                                                                                                                                 [ 68%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2] PASSED                                                                                                                                 [ 75%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8] PASSED                                                                                                                                 [ 81%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1] PASSED                                                                                                                              [ 87%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2] PASSED                                                                                                                              [ 93%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8] PASSED                                                                                                                              [100%]


-------------------------------------------------------------------------------------------------------------- benchmark 'analog_readers': 16 tests --------------------------------------------------------------------------------------------------------------
Name (time in us)                                                                           Min                    Max               Mean              StdDev             Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2]          21.3000 (1.0)       2,277.6000 (3.74)     24.3897 (1.06)      31.6750 (2.60)     22.3000 (1.00)     0.5000 (1.67)      60;1078       41.0009 (0.94)      10906           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1]          21.4000 (1.00)     15,783.8000 (25.92)    25.7863 (1.12)     155.8658 (12.80)    22.4000 (1.01)     0.5000 (1.67)       5;1049       38.7802 (0.89)      10406           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8]          21.7000 (1.02)      2,372.4000 (3.90)     24.2015 (1.05)      33.3025 (2.73)     22.5000 (1.01)     0.5000 (1.67)       53;885       41.3198 (0.95)      10918           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1]           21.7000 (1.02)        608.9000 (1.0)      22.9951 (1.0)       12.1791 (1.0)      22.2001 (1.0)      0.3001 (1.0)        19;264       43.4875 (1.0)        2679           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000]        24.2000 (1.14)      1,786.9000 (2.93)     27.7628 (1.21)      28.9060 (2.37)     25.1000 (1.13)     0.6001 (2.00)      282;954       36.0194 (0.83)       9833           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1]       24.3000 (1.14)      1,482.5000 (2.43)     27.2154 (1.18)      19.4070 (1.59)     25.1000 (1.13)     0.4999 (1.67)      396;813       36.7439 (0.84)       7711           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1]         24.9000 (1.17)      2,471.4000 (4.06)     28.2299 (1.23)      39.3373 (3.23)     26.1000 (1.18)     0.5999 (2.00)       48;759       35.4235 (0.81)       8504           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1]        25.0000 (1.17)      2,434.6000 (4.00)     27.5313 (1.20)      36.6897 (3.01)     25.9000 (1.17)     0.5001 (1.67)       75;767       36.3223 (0.84)       9644           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2]        25.1000 (1.18)      2,444.3000 (4.01)     29.0313 (1.26)      42.0275 (3.45)     26.0000 (1.17)     0.6000 (2.00)       41;821       34.4456 (0.79)       7337           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8]        25.3000 (1.19)      2,202.5000 (3.62)     27.4837 (1.20)      24.0327 (1.97)     26.2000 (1.18)     0.4750 (1.58)      111;761       36.3852 (0.84)       9355           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2]       25.5000 (1.20)      2,023.7000 (3.32)     27.7882 (1.21)      26.8567 (2.21)     26.4000 (1.19)     0.6999 (2.33)      101;530       35.9865 (0.83)       8749           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1]     29.9001 (1.40)      2,230.9000 (3.66)     33.0988 (1.44)      29.0350 (2.38)     30.9000 (1.39)     0.5000 (1.67)       83;946       30.2126 (0.69)       9150           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000]      30.2000 (1.42)      1,885.3000 (3.10)     35.0710 (1.53)      27.5310 (2.26)     31.1000 (1.40)     0.6500 (2.17)     483;1298       28.5136 (0.66)       9100           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8]       32.3000 (1.52)      1,453.6000 (2.39)     34.3823 (1.50)      17.3763 (1.43)     33.2000 (1.50)     0.4001 (1.33)      149;662       29.0847 (0.67)       9200           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2]     32.6000 (1.53)      1,574.4000 (2.59)     36.7150 (1.60)      19.9699 (1.64)     33.6000 (1.51)     0.7000 (2.33)     597;1061       27.2369 (0.63)       9497           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8]     45.2000 (2.12)      2,365.2000 (3.88)     47.7599 (2.08)      27.7892 (2.28)     46.1000 (2.08)     0.5000 (1.67)      123;700       20.9381 (0.48)       7758           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
===================================================================================================================== 16 passed, 72 deselected in 6.89s =====================================================================================================================

read_all_available went from a mean of 40-60 us to a mean of 28-46 us.

@bkeryan
Copy link
Collaborator Author

bkeryan commented Nov 7, 2025

Results with #881 and changing _calculate_num_samps_per_chan to use 0x217f (do_num_booleans_per_chan) to generate errors.

PS D:\dev\nidaqmx-python> poetry run pytest -v .\tests\benchmark\test_analog_stream_readers.py -k "library and (read_all_available or read_many_sample)"
============================================================================================================================ test session starts ============================================================================================================================
platform win32 -- Python 3.11.7, pytest-8.4.2, pluggy-1.6.0 -- D:\dev\nidaqmx-python\.venv\Scripts\python.exe
cachedir: .pytest_cache
benchmark: 5.2.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: D:\dev\nidaqmx-python
configfile: pyproject.toml
plugins: benchmark-5.2.0, cov-7.0.0, mock-3.15.1
collected 88 items / 72 deselected / 16 selected

tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1] PASSED                                                                                                                                    [  6%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000] PASSED                                                                                                                                 [ 12%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1] PASSED                                                                                                                                  [ 18%]
tests/benchmark/test_analog_stream_readers.py::test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000] PASSED                                                                                                                               [ 25%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1] PASSED                                                                                                                                   [ 31%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2] PASSED                                                                                                                                   [ 37%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8] PASSED                                                                                                                                   [ 43%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1] PASSED                                                                                                                                [ 50%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2] PASSED                                                                                                                                [ 56%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8] PASSED                                                                                                                                [ 62%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1] PASSED                                                                                                                                 [ 68%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2] PASSED                                                                                                                                 [ 75%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8] PASSED                                                                                                                                 [ 81%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1] PASSED                                                                                                                              [ 87%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2] PASSED                                                                                                                              [ 93%]
tests/benchmark/test_analog_stream_readers.py::test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8] PASSED                                                                                                                              [100%]


------------------------------------------------------------------------------------------------------------------- benchmark 'analog_readers': 16 tests -------------------------------------------------------------------------------------------------------------------
Name (time in us)                                                                              Min                    Max                  Mean              StdDev                Median                 IQR            Outliers          OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-2]             21.8000 (1.0)         256.7000 (1.15)        23.4959 (1.0)        7.5896 (1.16)        22.6000 (1.0)        0.5000 (1.25)      113;592  42,560.6225 (1.0)        7068           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1]              22.0000 (1.01)        529.7000 (2.37)        23.5918 (1.00)      10.6762 (1.63)        22.9000 (1.01)       0.5000 (1.25)       20;256  42,387.6617 (1.00)       2722           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-1]             22.1000 (1.01)        468.6000 (2.09)        24.1235 (1.03)       9.2967 (1.42)        22.8000 (1.01)       0.5000 (1.25)      304;838  41,453.4120 (0.97)      10385           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1-8]             22.1000 (1.01)     16,949.3000 (75.73)       26.0300 (1.11)     165.4784 (25.26)       23.0000 (1.02)       0.5000 (1.25)       2;1028  38,417.2536 (0.90)      10707           1
test___analog_single_channel_reader___read_many_sample[library_init_kwargs-1000]           24.8000 (1.14)        299.9000 (1.34)        26.7389 (1.14)       8.0469 (1.23)        25.6000 (1.13)       0.4001 (1.00)      267;942  37,398.7238 (0.88)       9843           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-1]          25.0000 (1.15)        228.7000 (1.02)        27.1525 (1.16)       7.5008 (1.15)        25.7000 (1.14)       0.4000 (1.0)       417;820  36,829.0182 (0.87)       9653           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-2]          26.0000 (1.19)        223.8000 (1.0)         27.6114 (1.18)       6.5506 (1.0)         26.9000 (1.19)       0.4000 (1.0)       142;639  36,216.8755 (0.85)       9075           1
test___analog_multi_channel_reader___read_many_sample[library_init_kwargs-1000-8]          32.8000 (1.50)      1,515.9000 (6.77)        34.6427 (1.47)      16.6577 (2.54)        33.7000 (1.49)       0.5000 (1.25)      121;533  28,866.1147 (0.68)       9329           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-8]        2,901.5000 (133.10)    4,576.3000 (20.45)    2,975.2064 (126.63)   172.0575 (26.27)    2,919.5000 (129.18)    35.1000 (87.75)       27;52     336.1111 (0.01)        312           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-1]     2,904.6000 (133.24)    5,150.5000 (23.01)    3,011.6247 (128.18)   318.9577 (48.69)    2,927.8500 (129.55)    58.3500 (145.87)      12;38     332.0467 (0.01)        312           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-2]        2,904.9000 (133.25)    5,377.2000 (24.03)    3,026.1032 (128.79)   279.4141 (42.65)    2,942.4000 (130.19)    91.9500 (229.87)      19;35     330.4580 (0.01)        313           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1-1]        2,905.9000 (133.30)    4,867.8000 (21.75)    3,015.5084 (128.34)   237.5993 (36.27)    2,933.3000 (129.79)    83.8000 (209.50)      20;39     331.6190 (0.01)        261           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1000]      2,907.3000 (133.36)    5,797.1000 (25.90)    3,030.0079 (128.96)   317.3613 (48.45)    2,940.6500 (130.12)    62.7500 (156.87)      17;47     330.0321 (0.01)        316           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-2]     2,909.9000 (133.48)    5,761.4000 (25.74)    2,968.0112 (126.32)   221.1040 (33.75)    2,927.5000 (129.54)    24.4500 (61.12)        8;37     336.9260 (0.01)        304           1
test___analog_multi_channel_reader___read_all_available[library_init_kwargs-1000-8]     2,922.3000 (134.05)    5,533.8000 (24.73)    3,002.6458 (127.79)   249.2597 (38.05)    2,946.0500 (130.36)    67.1000 (167.75)       6;28     333.0396 (0.01)        310           1
test___analog_single_channel_reader___read_all_available[library_init_kwargs-1]         2,932.6000 (134.52)    5,488.4000 (24.52)    3,213.0034 (136.75)   523.9661 (79.99)    3,025.9000 (133.89)   216.0250 (540.06)        3;3     311.2353 (0.01)         29           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
==================================================================================================================== 16 passed, 72 deselected in 11.01s =====================================================================================================================

read_all_available went from a mean of 40-60 us to a mean of 3 ms.

@bkeryan bkeryan mentioned this pull request Nov 7, 2025
Merged
3 tasks
Base automatically changed from users/bkeryan/fix-benchmark-buffer-size to master November 10, 2025 14:54
@bkeryan bkeryan force-pushed the users/bkeryan/read-all-available-benchmark branch from 35e5700 to 2b90a49 Compare November 10, 2025 14:55
@github-actions
Copy link
Contributor

github-actions bot commented Nov 10, 2025 •
edited
Loading

Test Results

    42 files      42 suites   1h 2m 7s ⏱️
 3 119 tests  2 459 ✅    660 💤 0 ❌
51 448 runs  40 990 ✅ 10 458 💤 0 ❌

Results for commit e2efe97.

♻️ This comment has been updated with latest results.

@bkeryan bkeryan merged commit f62b0ca into master Nov 10, 2025
29 checks passed
@bkeryan bkeryan deleted the users/bkeryan/read-all-available-benchmark branch November 10, 2025 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhindes zhindes zhindes approved these changes

@mikeprosserni mikeprosserni mikeprosserni approved these changes

@maxxboehme maxxboehme Awaiting requested review from maxxboehme maxxboehme is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bkeryan @zhindes @mikeprosserni