Add torchao kernels to xcframework #10963

hinriksnaer · 2025-05-19T02:20:15Z

Summary

In accordance with the following ticket, we want to enable building the torchao kernel through the main ExecuTorch CMakeLists.txt. This PR should cover all the necessary steps required to build low-bit kernels on the iOS app.

List of changes

Provide an optional flag EXECUTORCH_BUILD_KERNELS_TORCHAO that enables building torchao through the CMakeLists.txt.
Update the apple framework script to include the torchao build.
Updated the apple framework defaults to include torchao.

Test plan

Code has been successfully tested on the iPhone 16 simulator

pytorch-bot · 2025-05-19T02:20:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10963

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 935da89 with merge base da0c80a ():

NEW FAILURE - The following job has failed:

pull / test-mediatek-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t b1ea91a27da4e97f72e350ab2d53968a71a22ae691e70d42cac42393bb0ecde4 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-05-19T02:20:21Z

Hi @hinriksnaer!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

github-actions · 2025-05-19T02:20:45Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:.

If not, please add the release notes: none label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

metascroy · 2025-05-19T18:53:01Z

CMakeLists.txt

+    )
+    target_link_options_shared_lib(torchao_ops_executorch)
+    list(APPEND link_libraries torchao_ops_executorch)
+    if(EXECUTORCH_BUILD_MPS)


@hinriksnaer let's remove the section on EXECUTORCH_BUILD_MPS for now.

metascroy · 2025-05-19T19:09:45Z

CMakeLists.txt

+      ${CMAKE_CURRENT_BINARY_DIR}/third-party/ao/torchao/experimental
+    )
+    target_link_options_shared_lib(torchao_ops_executorch)
+    list(APPEND link_libraries torchao_ops_executorch)


I think link_libraries is not defined here. What we need to do to register the kernels is to link the kernel library torchao_ops_executorch against the ET library.

In the llama example, this is done by linking the op library against the llama runner executable here: https://github.com/pytorch/executorch/blob/main/examples/models/llama/CMakeLists.txt#L223.

But here, we don't have an executable, so we need to modify the logic slightly. We should link against executorch_core to register the kernels. See how kernels_custom does it. From https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L666-L669 it calls this: https://github.com/pytorch/executorch/blob/main/extension/llm/custom_ops/CMakeLists.txt#L78 for similar logic.

If it worked correctly, you should be able to follow the steps here: https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md#running-with-low-bit-kernels

But instead of building the kernels as part of the application in (by pasing -DEXECUTORCH_BUILD_TORCHAO=ON in step 2), we should be able to pass (-DEXECUTORCH_BUILD_KERNELS_TORCHAO=ON in step1 when we build executorch and not pass anything in step2).

cc @larryliu0820 if he has any other thoughts here

Yep let’s do what @metascroy said

metascroy · 2025-05-19T19:10:38Z

scripts/build_apple_frameworks.sh

 libquantized_ops_lib.a,\
 :"

+FRAMEWORK_KERNELS_TORCHAO="kernels_torchao:\


Before testing xcframework, let's verify your changes work on macOS desktop first.

TODO move some of these to the apple build framework

metascroy · 2025-06-30T21:19:43Z

Thanks @hinriksnaer! Left a few minor comments/questions and tagged @shoumikhin to look at xcframework changes.

I also kicked off the CI jobs for your PR.

shoumikhin

Thanks for adding AO kernels!

A sanity check, do they really deserve a separate deliverable, or could be a part of kernels_optimized framework?

Also, notice the Package.swift has to be updated, as well as .Package.swift dir, and the corresponding files on the "swiftpm" branch.

scripts/build_apple_frameworks.sh

shoumikhin · 2025-07-10T19:09:44Z

scripts/build_apple_frameworks.sh

-        echo -e "\033[31m[error] unknown option: ${arg}\033[0m"
-        exit 1
-      ;;
+  -h | --help) usage ;;


nit: need to change the indentation?

shoumikhin · 2025-07-10T21:25:20Z

scripts/build_apple_frameworks.sh

-          -DCMAKE_BUILD_TYPE="${mode}" \
-          ${CMAKE_OPTIONS_OVERRIDE[@]:-} \
-          --preset "${preset}"
+      -B "${preset_output_dir}" \


nit: restore indentation?

shoumikhin · 2025-07-10T21:25:29Z

scripts/build_apple_frameworks.sh

  //extension/module: \
-  //extension/tensor: \
-| rsync -av --files-from=- "$SOURCE_ROOT_DIR" "$HEADERS_ABSOLUTE_PATH/executorch"
+  //extension/tensor: |


shoumikhin · 2025-07-10T21:25:46Z

scripts/build_apple_frameworks.sh

 cp "$SOURCE_ROOT_DIR/extension/apple/ExecuTorch/Exported/"*.h "$HEADERS_ABSOLUTE_PATH/executorch"

-cat > "$HEADERS_ABSOLUTE_PATH/module.modulemap" << 'EOF'
+cat >"$HEADERS_ABSOLUTE_PATH/module.modulemap" <<'EOF'


nit: restore formatting?

shoumikhin · 2025-07-10T21:26:01Z

scripts/build_apple_frameworks.sh

-      name=$(echo "$framework" | cut -d: -f1)
-      spec=$(echo "$framework" | cut -d: -f2-)
-      framework="${name}_$(echo "$mode" | tr '[:upper:]' '[:lower:]'):${spec}"
+    local name spec


nit: restore indentation?

shoumikhin · 2025-07-10T21:28:01Z

examples/demo-apps/apple_ios/LLaMA/torchao_debug.xcconfig

@@ -0,0 +1,34 @@
+ET_PLATFORM[sdk=iphonesimulator*] = simulator


Yes, this is not needed.

metascroy · 2025-07-11T03:37:17Z

Thanks for adding AO kernels!

A sanity check, do they really deserve a separate deliverable, or could be a part of kernels_optimized framework?

Also, notice the Package.swift has to be updated, as well as .Package.swift dir, and the corresponding files on the "swiftpm" branch.

A sanity check, do they really deserve a separate deliverable, or could be a part of kernels_optimized framework?

Given their more experimental status, I think it makes sense for them to be separate, but maybe in future we fold into kernels_optimized?

metascroy · 2025-07-29T16:56:37Z

export_llama_torchao.sh

@@ -0,0 +1,23 @@
+# No quantization


@hinriksnaer this file should be removed. It looks like it is just for testing?

metascroy · 2025-07-29T16:57:44Z

CMakeLists.txt

+  set(EXECUTORCH_LIBRARIES executorch pthreadpool extension_threadpool cpuinfo)
+
+  add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/third-party/ao/torchao/experimental)
+  target_link_options_shared_lib(torchao_ops_executorch)


@hinriksnaer it looks like "target_link_options_shared_lib" was renamed to "executorch_target_link_options_shared_lib"?

metascroy · 2025-07-29T17:01:44Z

tools/cmake/executorch-config.cmake

  endif()
 endforeach()

+# TODO: investigate use of install(EXPORT) to cleanly handle


@hinriksnaer it looks like this file was cleaned up on main (see https://github.com/pytorch/executorch/blob/main/tools/cmake/executorch-config.cmake), and your PR is including the previous state.

I think the only changes needed now are lines 87/88.

metascroy · 2025-07-30T18:08:57Z

CMakeLists.txt

+  list(APPEND TORCHAO_INCLUDE_DIRS
+    ${EXECUTORCH_ROOT}/backends/xnnpack/third-party/pthreadpool/include
+    ${EXECUTORCH_ROOT}/backends/xnnpack/third-party/cpuinfo/include
+    ${EXECUTORCH_ROOT}/third-party/ao


What adding "${EXECUTORCH_ROOT}/third-party/ao" necessary?

Removing it caused the build process to fail.

What was the failure?

Here is the bottom of the terminal output

CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/quantize.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/quantization/quantize.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'torchao_kernels_aarch64' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wall -Werror -Wno-deprecated -Wno-shorten-64-to-32 @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/quantize.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/quantize.dia -c /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/quantization/quantize.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/quantize.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/quantization/quantize.cpp:9:10: fatal error: 'torchao/experimental/kernels/cpu/aarch64/quantization/quantize.h' file not found 9 | #include <torchao/experimental/kernels/cpu/aarch64/quantization/quantize.h> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/interleave.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/valpacking/interleave.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'torchao_kernels_aarch64' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wall -Werror -Wno-deprecated -Wno-shorten-64-to-32 @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/interleave.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/interleave.dia -c /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/valpacking/interleave.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/interleave.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/valpacking/interleave.cpp:7:10: fatal error: 'torchao/experimental/kernels/cpu/aarch64/valpacking/valpack.h' file not found 7 | #include <torchao/experimental/kernels/cpu/aarch64/valpacking/valpack.h> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/find_min_and_max.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/find_min_and_max.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'torchao_kernels_aarch64' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wall -Werror -Wno-deprecated -Wno-shorten-64-to-32 @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/find_min_and_max.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/find_min_and_max.dia -c /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/find_min_and_max.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/find_min_and_max.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/find_min_and_max.cpp:9:10: fatal error: 'torchao/experimental/kernels/cpu/aarch64/reduction/reduction.h' file not found 9 | #include <torchao/experimental/kernels/cpu/aarch64/reduction/reduction.h> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. Libtool /Users/softmax/Documents/projects/executorch/cmake-out/ios/Release/libabsl_utf8_for_code_point.a normal (in target 'utf8_for_code_point' from project 'executorch') cd /Users/softmax/Documents/projects/executorch /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static -arch_only arm64 -D -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS18.4.sdk -L/Users/softmax/Documents/projects/executorch/cmake-out/ios/Release -filelist /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/utf8_for_code_point.build/Release-iphoneos/Objects-normal/arm64/absl_utf8_for_code_point.LinkFileList -dependency_info /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/utf8_for_code_point.build/Release-iphoneos/Objects-normal/arm64/absl_utf8_for_code_point_libtool_dependency_info.dat -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/Release/libabsl_utf8_for_code_point.a CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/compute_sum.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/compute_sum.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'torchao_kernels_aarch64' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wall -Werror -Wno-deprecated -Wno-shorten-64-to-32 @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/compute_sum.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/compute_sum.dia -c /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/compute_sum.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/compute_sum.o CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode.o /Users/softmax/Documents/projects/executorch/extension/llm/tokenizers/third-party/llama.cpp-unicode/src/unicode.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'tokenizers' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wno-attributes @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode.dia -c /Users/softmax/Documents/projects/executorch/extension/llm/tokenizers/third-party/llama.cpp-unicode/src/unicode.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode.o CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode-data.o /Users/softmax/Documents/projects/executorch/extension/llm/tokenizers/third-party/llama.cpp-unicode/src/unicode-data.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'tokenizers' from project 'executorch') cd /Users/softmax/Documents/projects/executorch Using response file: /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c++ -ivfsstatcache /var/folders/cq/9bb9rdrs2jvcv_9rk42bk3h80000gn/C/com.apple.DeveloperTools/16.3-16E140/Xcode/SDKStatCaches.noindex/iphoneos18.4-22E235-a09501eccf75f892bc376f81961b27ba.sdkstatcache -fmessage-length\=214 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -Wno-trigraphs -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-c++11-extensions -Wno-implicit-fallthrough -fstrict-aliasing -Wdeprecated-declarations -Winvalid-offsetof -Wno-sign-conversion -Wno-infinite-recursion -Wno-move -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -Wno-attributes @/Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/82b82416624d2658e5098eb0a28c15c5-common-args.resp -MMD -MT dependencies -MF /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode-data.d --serialize-diagnostics /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode-data.dia -c /Users/softmax/Documents/projects/executorch/extension/llm/tokenizers/third-party/llama.cpp-unicode/src/unicode-data.cpp -o /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/tokenizers.build/Release-iphoneos/Objects-normal/arm64/unicode-data.o note: Run script build phase 'Generate third-party/CMakeFiles/flatcc_external_project' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'flatcc_external_project' from project 'executorch') note: Run script build phase 'Generate third-party/CMakeFiles/flatbuffers_external_project' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'flatbuffers_external_project' from project 'executorch') note: Run script build phase 'Generate CMakeFiles/ZERO_CHECK' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'ZERO_CHECK' from project 'executorch') note: Run script build phase 'Generate CMakeFiles/ALL_BUILD' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'ALL_BUILD' from project 'executorch') note: Run script build phase 'CMake PostBuild Rules' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'extension_apple' from project 'executorch') ** BUILD FAILED ** The following build commands failed: CompileC /Users/softmax/Documents/projects/executorch/cmake-out/ios/build/torchao_kernels_aarch64.build/Release-iphoneos/Objects-normal/arm64/find_min_and_max.o /Users/softmax/Documents/projects/executorch/third-party/ao/torchao/experimental/kernels/cpu/aarch64/reduction/find_min_and_max.cpp normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'torchao_kernels_aarch64' from project 'executorch') (1 failure)

### Summary In accordance with the following [ticket](pytorch#10694), we want to enable building the torchao kernel through the main ExecuTorch CMakeLists.txt. This PR should cover all the necessary steps required to build low-bit kernels on the iOS app. #### List of changes - Provide an optional flag `EXECUTORCH_BUILD_KERNELS_TORCHAO` that enables building torchao through the CMakeLists.txt. - Update the apple framework script to include the torchao build. - Updated the apple framework defaults to include torchao. ### Test plan - Code has been successfully tested on the iPhone 16 simulator --------- Co-authored-by: Scott Roy <[email protected]>

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 19, 2025

metascroy reviewed May 19, 2025

View reviewed changes

hinriksnaer force-pushed the torchao-build branch from 15fe940 to 882cfd5 Compare May 19, 2025 20:42

hinriksnaer force-pushed the torchao-build branch 2 times, most recently from 30b7176 to 50c149f Compare June 19, 2025 00:43

hinriksnaer mentioned this pull request Jun 26, 2025

Failed build of apple-ios example #11753

Closed

hinriksnaer force-pushed the torchao-build branch from 7dc3b4f to cb38a37 Compare June 26, 2025 20:58

hinriksnaer and others added 17 commits June 28, 2025 16:53

Rebase

03b306d

Updated CMakeLists for torchao

b4dd92b

Fixed parameter name

b6e0518

added missing flag

e0e6e98

updated build steps

f575cbf

up

9b9aa90

Work In Progress: changes to build_apple_framework.sh

9e913c4

cleaned up the cmake command

c16c779

updated cmake and framework build

c0c26aa

TODO move some of these to the apple build framework

up

70c6ae6

added torchao debug xcconfig

21bb1c1

added temporary build scripts

82659ec

aligned submodules

402a7ed

fixed accidental changes from rebase

e1f9b8a

added missing torchao param

3b11e4c

Rebase

06c2583

removed accidental code inclusion due to rebase

a8f3b35

hinriksnaer force-pushed the torchao-build branch from 963914e to a8f3b35 Compare June 28, 2025 20:54

Update comments in CMakeLists.txt

5dd6df8

metascroy added the ciflow/trunk label Jul 7, 2025

shoumikhin reviewed Jul 10, 2025

View reviewed changes

pytorch-bot bot removed the ciflow/trunk label Jul 10, 2025

metascroy reviewed Jul 29, 2025

View reviewed changes

hinriksnaer force-pushed the torchao-build branch from facf5e2 to 01e89aa Compare July 29, 2025 22:26

hinriksnaer added 3 commits July 29, 2025 18:37

updated rebase

5200d68

updated bash formatting

276232e

Removed dev scripts

58b24c2

hinriksnaer force-pushed the torchao-build branch from 01e89aa to 58b24c2 Compare July 29, 2025 22:39

hinriksnaer requested review from SS-JIA, digantdesai, jackzhxng, mcr229 and mergennachin as code owners July 29, 2025 22:39

hinriksnaer added 7 commits July 29, 2025 18:43

reverted to existing changes

6a00da3

removed dev script

435ffa1

fixed submodules

16b8984

Merge branch 'main' into torchao-build

3547232

updated link name

03f584a

changed dependency reference in torchao build

eab0f73

Merge branch 'main' into torchao-build

935da89

metascroy reviewed Jul 30, 2025

View reviewed changes

metascroy added the ciflow/trunk label Jul 30, 2025

metascroy merged commit d4c78ab into pytorch:main Jul 30, 2025
227 of 229 checks passed

		@@ -0,0 +1,34 @@
		ET_PLATFORM[sdk=iphonesimulator*] = simulator

Add torchao kernels to xcframework #10963

Add torchao kernels to xcframework #10963

Uh oh!

Conversation

hinriksnaer commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

List of changes

Test plan

Uh oh!

pytorch-bot bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10963

❌ 1 New Failure

Uh oh!

facebook-github-bot commented May 19, 2025

Action Required

Process

Uh oh!

github-actions bot commented May 19, 2025

This PR needs a release notes: label

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

metascroy commented Jun 30, 2025

Uh oh!

shoumikhin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

metascroy commented Jul 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

metascroy Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hinriksnaer Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hinriksnaer commented May 19, 2025 •

edited

Loading

pytorch-bot bot commented May 19, 2025 •

edited

Loading

This PR needs a `release notes:` label

metascroy Jul 29, 2025 •

edited

Loading

hinriksnaer Jul 30, 2025 •

edited

Loading