Skip to content

Commit 33bd456

Browse files
authored
[Backend Tester] Reduce log verbosity / spam (#13312)
When running tests, deprecation warnings for export_for_training and "internal consistency verification was requested but not available" create a large amount of spam in the CLI and drown out actual test info during the run. Neither warning is particularly useful. I've suppressed the the export_for_training warning in the backend tester only and switched the tester to not request internal consistency verification since it's not compiled into pybindings by default. This gives much cleaner CLI output. Before: ``` test_add_dtype_float32_xnnpack_static_int8_per_channel (test_add.Add.test_add_dtype_float32_xnnpack_static_int8_per_channel) ... /home/gregory/src/executorch/src/executorch/backends/test/harness/stages/quantize.py:50: FutureWarning: `torch.export.export_for_training` is deprecated and will be removed in PyTorch 2.10. Please use `torch.export.export` instead, which is functionally equivalent. captured_graph = export_for_training(artifact, inputs, strict=True).module() /home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torchao/quantization/pt2e/utils.py:818: FutureWarning: `torch.export.export_for_training` is deprecated and will be removed in PyTorch 2.10. Please use `torch.export.export` instead, which is functionally equivalent. aten_pattern = torch.export.export_for_training( /home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torchao/quantization/pt2e/utils.py:818: FutureWarning: `torch.export.export_for_training` is deprecated and will be removed in PyTorch 2.10. Please use `torch.export.export` instead, which is functionally equivalent. aten_pattern = torch.export.export_for_training( /home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torchao/quantization/pt2e/utils.py:818: FutureWarning: `torch.export.export_for_training` is deprecated and will be removed in PyTorch 2.10. Please use `torch.export.export` instead, which is functionally equivalent. aten_pattern = torch.export.export_for_training( /home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torchao/quantization/pt2e/utils.py:818: FutureWarning: `torch.export.export_for_training` is deprecated and will be removed in PyTorch 2.10. Please use `torch.export.export` instead, which is functionally equivalent. ... (it logs like 20x per test) ``` After: ``` test_add_dtype_float32_xnnpack (test_add.Add.test_add_dtype_float32_xnnpack) ... ok test_add_dtype_float32_xnnpack_static_int8_per_channel (test_add.Add.test_add_dtype_float32_xnnpack_static_int8_per_channel) ... ok test_add_f32_alpha_xnnpack (test_add.Add.test_add_f32_alpha_xnnpack) ... ok test_add_f32_alpha_xnnpack_static_int8_per_channel (test_add.Add.test_add_f32_alpha_xnnpack_static_int8_per_channel) ... ERROR ... ```
1 parent 7308528 commit 33bd456

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

backends/test/harness/stages/serialize.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
try:
1414
from executorch.extension.pybindings.portable_lib import ( # @manual
1515
_load_for_executorch_from_buffer,
16+
Verification,
1617
)
1718
except ImportError as e:
1819
logger.warning(f"{e=}")
@@ -39,7 +40,9 @@ def graph_module(self) -> None:
3940

4041
def run_artifact(self, inputs):
4142
inputs_flattened, _ = tree_flatten(inputs)
42-
executorch_module = _load_for_executorch_from_buffer(self.buffer)
43+
executorch_module = _load_for_executorch_from_buffer(
44+
self.buffer, program_verification=Verification.Minimal
45+
)
4346
executorch_output = copy.deepcopy(
4447
executorch_module.run_method("forward", tuple(inputs_flattened))
4548
)

backends/test/suite/runner.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import re
44
import time
55
import unittest
6+
import warnings
67

78
from datetime import timedelta
89
from typing import Any
@@ -249,6 +250,10 @@ def build_test_filter(args: argparse.Namespace) -> TestFilter:
249250
def runner_main():
250251
args = parse_args()
251252

253+
# Suppress deprecation warnings for export_for_training, as it generates a
254+
# lot of log spam. We don't really need the warning here.
255+
warnings.simplefilter("ignore", category=FutureWarning)
256+
252257
begin_test_session(args.report)
253258

254259
if len(args.suite) > 1:

0 commit comments

Comments
 (0)