Simplify cmake test invocation #7043

kirklandsign · 2024-11-23T00:23:11Z

Summary

We can build the test while building the lib, just in one pass.

After this, developers can run tests while they develop. Both can be built at the same time. For developers, when they write their code, it's easy for them to test out. Just add -DEXECUTORCH_BUILD_TESTS=ON to their cmake config. The test target will be built automatically in cmake-out and they can run it.

NOTE: examples/models/llama/tokenizer/test and extension/llm/tokenizer/test can't use this right now. They don't have a good triggering path, and can't be triggered from root either. Use the old trigger path for them.

Test plan

Method 1: script for all tests

sh test/run_oss_cpp_tests.sh

Method 2: go to cmake-out and test

cd cmake-out
ctest

Method 3: test specific target

For example, we want to test kernel_resolution_test in runtime/core

cmake -DEXECUTORCH_BUILD_TESTS=ON . -Bcmake-out
cmake --build cmake-out
cd cmake-out
ctest -R kernel_resolution_test

pytorch-bot · 2024-11-23T00:23:14Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7043

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 0dcd11c with merge base 047fd37 ():

NEW FAILURE - The following job has failed:

pull / unittest-arm / linux-job (gh)
RuntimeError: Command docker exec -t e71a6b20d9e88da59e1240fd9a45348febcf1131be56e635b81d8addde1a19d9 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

larryliu0820 · 2024-12-03T00:39:07Z

CMakeLists.txt

+if(BUILD_TESTING)
+  # This contains the list of tests which are always built
+  add_subdirectory(extension/evalue_util/test)
+  add_subdirectory(extension/kernel_util/test)
+  add_subdirectory(extension/memory_allocator/test)
+  add_subdirectory(extension/parallel/test)
+  add_subdirectory(extension/pytree/test)
+  add_subdirectory(kernels/portable/test)
+  add_subdirectory(kernels/portable/cpu/util/test)
+  add_subdirectory(kernels/prim_ops/test)
+  add_subdirectory(kernels/test)
+  add_subdirectory(runtime/core/exec_aten/testing_util/test)
+  add_subdirectory(runtime/core/exec_aten/util/test)
+  add_subdirectory(runtime/core/portable_type/test)
+  add_subdirectory(runtime/core/test)
+  add_subdirectory(runtime/executor/test)
+  add_subdirectory(runtime/kernel/test)
+  add_subdirectory(runtime/platform/test)
+  add_subdirectory(test/utils)


Can we move these into test.cmake

larryliu0820 · 2024-12-03T00:40:09Z

CMakeLists.txt


 # Add googletest if any test targets should be built
-if(EXECUTORCH_BUILD_GTESTS)
+if(BUILD_TESTING)


Why not EXECUTORCH_BUILD_TESTS?

BUILD_TESTING is from cmake system to recognize whether test config is on or not. It will be enabled when we include ctest (see https://github.com/pytorch/executorch/pull/7043/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aR318-R320)

facebook-github-bot · 2024-12-04T01:29:59Z