Reland minibench refactor

[kirklandsign]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10406

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 75 Cancelled Jobs

As of commit 9439372 with merge base dfd3dbe ():

NEW FAILURE - The following job has failed:

• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Lint / android-java-format / linux-job (gh)
  ##[error]The operation was canceled.
• Lint / lintrunner / linux-job (gh)
  ##[error]The operation was canceled.
• pull / android / build-llm-demo / linux-job (gh)
  ##[error]The operation was canceled.
• pull / android / run-emulator (gh)
• pull / test-binary-size-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-binary-size-linux-gcc / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-custom-ops-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
• pull / test-llama_runner_eager-linux / linux-job (gh)
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-linux-android / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-llava-runner-linux / linux-job (gh)
• pull / test-mediatek-models-linux / linux-job (gh)
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh)
• pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
• pull / test-models-linux-basic (mv3, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (vit, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-moshi-linux / linux-job (gh)
• pull / test-openvino-linux / linux-job (gh)
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-pybind-build-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-qnn-models-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-quantized-aot-lib-linux / linux-job (gh)
• pull / test-selective-build-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-setup-linux-gcc / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-static-llama-qnn-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / unittest / linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / unittest / macos / macos-job (gh)
  ##[error]The operation was canceled.
• pull / unittest-arm / linux-job (gh)
  ##[error]The operation was canceled.
• pull / unittest-buck / linux / linux-job (gh)
• pull / unittest-buck / macos / macos-job (gh)
  ##[error]The operation was canceled.
• pull / unittest-editable / linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / unittest-editable / macos / macos-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.