Skip to content

Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet101-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs  #33751

@github-actions

Description

@github-actions

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet101-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 101.
Test link -

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7


timestamp: Tue Jan 21 04:20:51 2025, metric_value: 82596.67
timestamp: Mon Jan 20 04:23:31 2025, metric_value: 85858.23
timestamp: Sun Jan 19 04:21:21 2025, metric_value: 82086.12
timestamp: Sat Jan 18 04:24:51 2025, metric_value: 86418.52
timestamp: Fri Jan 17 04:19:36 2025, metric_value: 84872.85
timestamp: Thu Jan 16 04:20:20 2025, metric_value: 86711.72
timestamp: Wed Jan 15 04:24:05 2025, metric_value: 85430.64
timestamp: Tue Jan 14 04:22:19 2025, metric_value: 87301.49
timestamp: Mon Jan 13 04:19:36 2025, metric_value: 83876.73
timestamp: Sun Jan 12 04:20:33 2025, metric_value: 83689.32
timestamp: Sat Jan 11 04:23:14 2025, metric_value: 85983.61 <---- Anomaly
timestamp: Fri Jan 10 04:20:42 2025, metric_value: 89605.05
timestamp: Thu Jan  9 04:22:23 2025, metric_value: 90195.36
timestamp: Wed Jan  8 04:22:21 2025, metric_value: 91196.37
timestamp: Tue Jan  7 04:25:50 2025, metric_value: 90533.32
timestamp: Mon Jan  6 04:20:11 2025, metric_value: 89260.64
timestamp: Sun Jan  5 04:21:54 2025, metric_value: 89676.07
timestamp: Sat Jan  4 04:19:29 2025, metric_value: 90762.79
timestamp: Fri Jan  3 04:21:13 2025, metric_value: 90446.23
timestamp: Thu Jan  2 04:21:48 2025, metric_value: 89222.93
timestamp: Wed Jan  1 04:23:23 2025, metric_value: 89887.06

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions