Integrate Hermes with the Xcode build process

Summary:
## Context
If React Native is built from *main* of any non-stable commit, then Hermes is built from source. The build is performed by `build-ios-framework.sh` and `build-mac-framework.sh` scripts in `hermes-engine.podspec` `prepare_command` stage. Since those scripts have no access build target information, they build all possible architectures and platforms just in case. That takes ages.
## Solution
The idea is to integrate build script into Xcode *run script* phase, and use build target information to build Hermes for active architecture only.
## Implementation
- Existing behaviour remains unchanged for local tarball and remote prebuild cases.
- `build-hermesc-xcode.sh` builds Hermesc as `hermes-engine.podspec` `prepare_command`. Default build location is `react-native/sdks/hermes-engine/build_host_hermesc`.
- `build-hermes-xcode.sh` builds Hermes in 'Build Hermes' Xcode script phase. It uses `$PLATFORM_NAME`, `$CONFIGURATION`, `$ARCHS`, `$IPHONEOS_DEPLOYMENT_TARGET` and `$MACOSX_DEPLOYMENT_TARGET` environment variables to configure cmake project so it builds only active architecture. The script also gets RN version, *cmake* path and *hermesc* path from the podspec.
- `copy-hermes-xcode.sh` copies Hermes.framework inside the app bundle. This script phase is added to the user app target in a `post_install` hook, after pods are integrated in a user project.
- `OTHER_LDFLAGS -framework "hermes"` added to the user project to enable linking against Hermes.framework.
- If `HERMES_OVERRIDE_HERMESC_PATH` is set, then Hermesc building is skipped, and `HERMES_OVERRIDE_HERMESC_PATH` is used for `build-hermes-xcode.sh`.
- `HERMES_CLI_PATH` is injected into user project config to enable Hermes source maps in `react-native-xcode.sh`.
## Things that didn't work
- *Running build-hermesc-xcode.sh in Xcode run script phase*. This doesn't work because Hermesc is supposed to be built for macos, and if build target is ios, then Xcode configures environment in such a way that Hermesc build fails.
- *Installing Hermesc into CocoaPods download folder*. So it then ends up in `Pods/hermes-engine/build_host_hermesc`, and all the housekeeping is handled by CocoaPods. This doesn't work because cmake uses absolute paths in a configured project. If configured project is moved to a different location, nothing builds.
- *Installing Hermesc directly into Pods/hermes-engine*. This doesn't work because CocoaPods runs prepare_command before Pods folder clean up, and everything gets wiped.
## Known issue
-  If `Pods/hermes-engine` is manually removed, then `sdks/hermes-engine/build_host_hermesc` must also be removed before running `pod install`. Otherwise cmake will complain about stale cache:
```
CMake Error: The source "<CocoaPodsCache>/hermes-engine/<hash2>/CMakeLists.txt" does not match the source
"<CocoaPodsCache>/hermes-engine/<has1>/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
```
## Benchmark
MacBook M1 2021 32 GB.
```
export REACT_NATIVE_PATH=~/fbsource/xplat/js/react-native-github
cd $REACT_NATIVE_PATH/packages/rn-tester
pod install
rm -rf $REACT_NATIVE_PATH/sdks/hermes-engine/build_host_hermesc
cd $REACT_NATIVE_PATH/packages/rn-tester/Pods/hermes-engine
echo 't1=$(date +%s); $@; t2=$(date +%s); diff=$(echo "$t2 - $t1" | bc); echo Operation took $diff seconds.' > /tmp/benchmark.sh
```
```
# Before
export BUILD_TYPE=Debug
export JSI_PATH=$REACT_NATIVE_PATH/ReactCommon/jsi
export RELEASE_VERSION=1000.0
export IOS_DEPLOYMENT_TARGET=iphonesimulator
export MAC_DEPLOYMENT_TARGET=12.6
cd $REACT_NATIVE_PATH/packages/rn-tester/Pods/hermes-engine
. /tmp/benchmark.sh $REACT_NATIVE_PATH/sdks/hermes-engine/utils/build-ios-framework.sh
# Operation took 252 seconds
. /tmp/benchmark.sh $REACT_NATIVE_PATH/sdks/hermes-engine/utils/build-mac-framework.sh
# Operation took 179 seconds
```
```
# After
. /tmp/benchmark.sh source $REACT_NATIVE_PATH/sdks/hermes-engine/utils/build-hermesc-xcode.sh $REACT_NATIVE_PATH/sdks/hermes-engine/build_host_hermesc
# Operation took 59 seconds.
. /tmp/benchmark.sh xcodebuild -workspace $REACT_NATIVE_PATH/packages/rn-tester/RNTesterPods.xcworkspace -scheme hermes-engine
# Operation took 106 seconds.
```
|Before|||After|||
|--|
|iOS framework (s)|Mac framework (s)|Total (s)|Hermesc (s)|Target-specific framework (s)|Total (s)|
|252|179|431|59|106|**165 (-266) (-61%)**|
The performance win is fixed, and does not depend on the project size and structure.
As an example, this is how these changes affect build time of RNTester.
|Before||||After|||
|--|
||Pod install (s)|Xcode build (s)|Total (s)|Pod install (s)|Xcode build (s)|Total (s)|
|Clean build|1219|132|1352|734 (-485)|249(+117)|**983 (-369)**|
|Incremental build|82|30|112|105 (+23)|**34 (+4)**|139 (+27)|
The most important values here are the total clean build time and the incremental Xcode build time. The first one went down by 369 seconds, the second one went up by 4 seconds. I consider it a reasonable tradeoff.
The extra 4 seconds in  the incremental Xcode build time can potentially be mitigated by setting up output file lists for the new script phases.

allow-large-files

Changelog:
[iOS][Changed] - Hermes is integrated into Xcode build.

Reviewed By: hramos

Differential Revision: D40063686

fbshipit-source-id: e6993d62225789377db769244bc07786cc978a27

Author: dmytrorykun

.gitignore

@@ -122,6 +122,7 @@ package-lock.json
 /sdks/download
 /sdks/hermes
 /sdks/hermesc
+/sdks/hermes-engine/build_host_hermesc
 
 # Visual studio
 .vscode

ReactCommon/hermes/React-hermes.podspec

@@ -40,7 +40,7 @@ Pod::Spec.new do |s|
   s.public_header_files    = "executor/HermesExecutorFactory.h"
   s.compiler_flags         = folly_compiler_flags + ' ' + boost_compiler_flags
   s.pod_target_xcconfig    = {
-                               "HEADER_SEARCH_PATHS" => "\"$(PODS_TARGET_SRCROOT)/..\" \"$(PODS_ROOT)/boost\" \"$(PODS_ROOT)/RCT-Folly\" \"$(PODS_ROOT)/DoubleConversion\" \"$(PODS_ROOT)/libevent/include\""
+                               "HEADER_SEARCH_PATHS" => "\"${PODS_ROOT}/hermes-engine/destroot/include\" \"$(PODS_TARGET_SRCROOT)/..\" \"$(PODS_ROOT)/boost\" \"$(PODS_ROOT)/RCT-Folly\" \"$(PODS_ROOT)/DoubleConversion\" \"$(PODS_ROOT)/libevent/include\""
                              }.merge!(build_type == :debug ? { "GCC_PREPROCESSOR_DEFINITIONS" => "HERMES_ENABLE_DEBUGGER=1" } : {})
   s.header_dir             = "reacthermes"
   s.dependency "React-cxxreact", version

packages/rn-tester/Podfile.lock

@@ -73,7 +73,13 @@ PODS:
     - FlipperKit/FlipperKitNetworkPlugin
   - fmt (6.2.1)
   - glog (0.3.5)
-  - hermes-engine (1000.0.0)
+  - hermes-engine (1000.0.0):
+    - hermes-engine/Hermes (= 1000.0.0)
+    - hermes-engine/JSI (= 1000.0.0)
+    - hermes-engine/Public (= 1000.0.0)
+  - hermes-engine/Hermes (1000.0.0)
+  - hermes-engine/JSI (1000.0.0)
+  - hermes-engine/Public (1000.0.0)
   - libevent (2.1.12)
   - OpenSSL-Universal (1.1.1100)
   - RCT-Folly (2021.07.22.00):
@@ -933,7 +939,7 @@ SPEC CHECKSUMS:
   CocoaAsyncSocket: 065fd1e645c7abab64f7a6a2007a48038fdc6a99
   DoubleConversion: 5189b271737e1565bdce30deb4a08d647e3f5f54
   FBLazyVector: 19e408e76fa9258dd32191a50d60c41444f52d29
-  FBReactNativeSpec: 27a89a8eea1b441a73a78f420dd18dad3ed13723
+  FBReactNativeSpec: 9761d52cf2d3727e2557fbf4014c514909d76b6b
   Flipper: 26fc4b7382499f1281eb8cb921e5c3ad6de91fe0
   Flipper-Boost-iOSX: fd1e2b8cbef7e662a122412d7ac5f5bea715403c
   Flipper-DoubleConversion: 2dc99b02f658daf147069aad9dbd29d8feb06d30
@@ -945,7 +951,7 @@ SPEC CHECKSUMS:
   FlipperKit: cbdee19bdd4e7f05472a66ce290f1b729ba3cb86
   fmt: ff9d55029c625d3757ed641535fd4a75fedc7ce9
   glog: 04b94705f318337d7ead9e6d17c019bd9b1f6b1b
-  hermes-engine: cd15ebd246edff3a995ec666e898dd1cbdcaa10d
+  hermes-engine: 445a2267b04cb39ca4a0b2d6758b5a0e5a58ccad
   libevent: 4049cae6c81cdb3654a443be001fb9bdceff7913
   OpenSSL-Universal: ebc357f1e6bc71fa463ccb2fe676756aff50e88c
   RCT-Folly: 424b8c9a7a0b9ab2886ffe9c3b041ef628fd4fb1
@@ -960,7 +966,7 @@ SPEC CHECKSUMS:
   React-cxxreact: ebed982230716c3515ab2f435cb13aec8a56af02
   React-Fabric: 141459e61c825acf02d26ece099acbd9cbd87b99
   React-graphics: 2dda97baebb0082bb85499c862c3f269a194f416
-  React-hermes: 0a5145bae4207edf0def8e28fbcb6a8fd6e806c2
+  React-hermes: 4912383b4f062173cb623e570ead70ab380f7bef
   React-jsi: c24dbcfdf7ea075138b73372387c7f17c0db56ef
   React-jsidynamic: 2b14ac1b6d3a1b7daa1e5a424b98de87da981698
   React-jsiexecutor: 14e899380e3fe9ca74c4e19727540a03e7574721

scripts/cocoapods/jsengine.rb

@@ -34,3 +34,23 @@ def setup_hermes!(react_native_path: "../node_modules/react-native", fabric_enab
     pod 'React-hermes', :path => "#{react_native_path}/ReactCommon/hermes"
     pod 'libevent', '~> 2.1.12'
 end
+
+def add_copy_hermes_framework_script_phase(installer, react_native_path)
+    utils_dir = File.join(react_native_path, "sdks", "hermes-engine", "utils")
+    phase_name = "[RN]Copy Hermes framework"
+    project = installer.generated_aggregate_targets.first.user_project
+    target = project.targets.first
+    if target.shell_script_build_phases.none? { |phase| phase.name == phase_name }
+        phase = target.new_shell_script_build_phase(phase_name)
+        phase.shell_script = ". #{utils_dir}/copy-hermes-xcode.sh"
+        project.save()
+    end
+end
+
+def remove_copy_hermes_framework_script_phase(installer, react_native_path)
+    utils_dir = File.join(react_native_path, "sdks", "hermes-engine", "utils")
+    phase_name = "[RN]Copy Hermes framework"
+    project = installer.generated_aggregate_targets.first.user_project
+    project.targets.first.shell_script_build_phases.delete_if { |phase| phase.name == phase_name }
+    project.save()
+end

scripts/react_native_pods.rb

@@ -203,6 +203,12 @@ def react_native_post_install(installer, react_native_path = "../node_modules/re
     flipper_post_install(installer)
   end
 
+  if ReactNativePodsUtils.has_pod(installer, 'hermes-engine') && ENV['HERMES_BUILD_FROM_SOURCE'] == "1"
+    add_copy_hermes_framework_script_phase(installer, react_native_path)
+  else
+    remove_copy_hermes_framework_script_phase(installer, react_native_path)
+  end
+
   ReactNativePodsUtils.exclude_i386_architecture_while_using_hermes(installer)
   ReactNativePodsUtils.fix_library_search_paths(installer)
   ReactNativePodsUtils.fix_react_bridging_header_search_paths(installer)

sdks/hermes-engine/hermes-engine.podspec

@@ -19,9 +19,6 @@ version = package['version']
 hermestag_file = File.join(react_native_path, "sdks", ".hermesversion")
 isInCI = ENV['CI'] == true
 
-# sdks/hermesc/osx-bin/ImportHermesc.cmake
-import_hermesc_file=File.join(react_native_path, "sdks", "hermesc", "osx-bin", "ImportHermesc.cmake")
-
 source = {}
 git = "https://github.com/facebook/hermes.git"
 
@@ -32,7 +29,7 @@ if ENV.has_key?('HERMES_ENGINE_TARBALL_PATH')
   Pod::UI.puts '[Hermes] Using pre-built Hermes binaries from local path.' if Object.const_defined?("Pod::UI")
   source[:http] = "file://#{ENV['HERMES_ENGINE_TARBALL_PATH']}"
 elsif isInMain
-  Pod::UI.puts '[Hermes] Installing hermes-engine may take a while, building Hermes from source...'.yellow if Object.const_defined?("Pod::UI")
+  Pod::UI.puts '[Hermes] Installing hermes-engine may take slightly longer, building Hermes compiler from source...'.yellow if Object.const_defined?("Pod::UI")
   source[:git] = git
   source[:commit] = `git ls-remote https://github.com/facebook/hermes main | cut -f 1`.strip
 elsif isNightly
@@ -62,42 +59,80 @@ Pod::Spec.new do |spec|
   spec.source      = source
   spec.platforms   = { :osx => "10.13", :ios => "12.4" }
 
-  spec.preserve_paths      = ["destroot/bin/*"].concat(build_type == :debug ? ["**/*.{h,c,cpp}"] : [])
-  spec.source_files        = "destroot/include/**/*.h"
-  spec.exclude_files       = [
-                               "destroot/include/jsi/jsi/JSIDynamic.{h,cpp}",
-                               "destroot/include/jsi/jsi/jsilib-*.{h,cpp}",
-                             ]
-  spec.header_mappings_dir = "destroot/include"
-
-  spec.ios.vendored_frameworks = "destroot/Library/Frameworks/universal/hermes.xcframework"
-  spec.osx.vendored_frameworks = "destroot/Library/Frameworks/macosx/hermes.framework"
+  spec.preserve_paths      = '**/*.*'
+  spec.source_files        = ''
 
   spec.xcconfig = {
                     "CLANG_CXX_LANGUAGE_STANDARD" => "c++17",
                     "CLANG_CXX_LIBRARY" => "compiler-default"
                   }.merge!(build_type == :debug ? { "GCC_PREPROCESSOR_DEFINITIONS" => "HERMES_ENABLE_DEBUGGER=1" } : {})
 
-  if source[:git] then
-    ENV['REACT_NATIVE_PATH'] = react_native_path
-    hermes_utils_path = "/sdks/hermes-engine/utils"
-
-    spec.prepare_command = <<-EOS
-    export BUILD_TYPE=#{build_type.to_s.capitalize}
-    export RELEASE_VERSION="#{version}"
-    export IOS_DEPLOYMENT_TARGET="#{spec.deployment_target('ios')}"
-    export MAC_DEPLOYMENT_TARGET="#{spec.deployment_target('osx')}"
-    export JSI_PATH="$REACT_NATIVE_PATH/ReactCommon/jsi"
-
-    # Set HERMES_OVERRIDE_HERMESC_PATH if pre-built HermesC is available
-    #{File.exist?(import_hermesc_file) ? "export HERMES_OVERRIDE_HERMESC_PATH=#{import_hermesc_file}" : ""}
-    #{File.exist?(import_hermesc_file) ? "echo \"Overriding HermesC path...\"" : ""}
-
-    # Build iOS framework
-    $REACT_NATIVE_PATH#{hermes_utils_path}/build-ios-framework.sh
-
-    # Build Mac framework
-    $REACT_NATIVE_PATH#{hermes_utils_path}/build-mac-framework.sh
-    EOS
+  if source[:http] then
+
+    spec.subspec 'Pre-built' do |ss|
+      ss.preserve_paths = ["destroot/bin/*"].concat(build_type == :debug ? ["**/*.{h,c,cpp}"] : [])
+      ss.source_files = "destroot/include/**/*.h"
+      ss.exclude_files = ["destroot/include/jsi/jsi/JSIDynamic.{h,cpp}", "destroot/include/jsi/jsi/jsilib-*.{h,cpp}"]
+      ss.header_mappings_dir = "destroot/include"
+      ss.ios.vendored_frameworks = "destroot/Library/Frameworks/universal/hermes.xcframework"
+      ss.osx.vendored_frameworks = "destroot/Library/Frameworks/macosx/hermes.framework"
+    end
+
+  elsif source[:git] then
+
+    ENV['HERMES_BUILD_FROM_SOURCE'] = "1"
+
+    spec.subspec 'Hermes' do |ss|
+      ss.source_files = ''
+      ss.public_header_files = 'API/hermes/*.h'
+      ss.header_dir = 'hermes'
+    end
+
+    spec.subspec 'JSI' do |ss|
+      ss.source_files = ''
+      ss.public_header_files = 'API/jsi/jsi/*.h'
+      ss.header_dir = 'jsi'
+    end
+
+    spec.subspec 'Public' do |ss|
+      ss.source_files = ''
+      ss.public_header_files = 'public/hermes/Public/*.h'
+      ss.header_dir = 'hermes/Public'
+    end
+
+    hermesc_path = ""
+
+    if ENV.has_key?('HERMES_OVERRIDE_HERMESC_PATH') && File.exist?(ENV['HERMES_OVERRIDE_HERMESC_PATH']) then
+      hermesc_path = ENV['HERMES_OVERRIDE_HERMESC_PATH']
+    else
+      # Keep hermesc_path synchronized with .gitignore entry.
+      ENV['REACT_NATIVE_PATH'] = react_native_path
+      hermesc_path = "${REACT_NATIVE_PATH}/sdks/hermes-engine/build_host_hermesc"
+      spec.prepare_command = ". #{react_native_path}/sdks/hermes-engine/utils/build-hermesc-xcode.sh #{hermesc_path}"
+    end
+
+    spec.user_target_xcconfig = {
+      'FRAMEWORK_SEARCH_PATHS' => '"$(PODS_ROOT)/hermes-engine/destroot/Library/Frameworks/iphoneos" ' +
+                                  '"$(PODS_ROOT)/hermes-engine/destroot/Library/Frameworks/iphonesimulator" ' +
+                                  '"$(PODS_ROOT)/hermes-engine/destroot/Library/Frameworks/macosx" ' +
+                                  '"$(PODS_ROOT)/hermes-engine/destroot/Library/Frameworks/catalyst"',
+      'OTHER_LDFLAGS' => '-framework "hermes"',
+      'HERMES_CLI_PATH' => "#{hermesc_path}/bin/hermesc"
+    }
+
+    spec.script_phases = [
+      {
+        :name => 'Build Hermes',
+        :script => <<-EOS
+        . ${PODS_ROOT}/../.xcode.env
+        export CMAKE_BINARY=${CMAKE_BINARY:-#{%x(command -v cmake | tr -d '\n')}}
+        . ${REACT_NATIVE_PATH}/sdks/hermes-engine/utils/build-hermes-xcode.sh #{version} #{hermesc_path}/ImportHermesc.cmake
+        EOS
+      },
+      {
+        :name => 'Copy Hermes Framework',
+        :script => ". ${REACT_NATIVE_PATH}/sdks/hermes-engine/utils/copy-hermes-xcode.sh"
+      }
+    ]
   end
 end

sdks/hermes-engine/utils/build-hermes-xcode.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+set -x
+
+release_version="$1"; shift
+hermesc_path="$1"; shift
+
+build_cli_tools="false"
+if [[ "$PLATFORM_NAME" == macosx ]]; then
+  build_cli_tools="true"
+fi
+
+enable_debugger="false"
+if [[ "$CONFIGURATION" == "Debug" ]]; then
+  enable_debugger="true"
+fi
+
+deployment_target=${IPHONEOS_DEPLOYMENT_TARGET}
+if [ -z "$deployment_target" ]; then
+  deployment_target=${MACOSX_DEPLOYMENT_TARGET}
+fi
+
+architectures=$( echo "$ARCHS" | tr  " " ";" )
+
+echo "Configure Apple framework"
+
+"$CMAKE_BINARY" \
+  -S "${PODS_ROOT}/hermes-engine" \
+  -B "${PODS_ROOT}/hermes-engine/build/${PLATFORM_NAME}" \
+  -DHERMES_APPLE_TARGET_PLATFORM:STRING="$PLATFORM_NAME" \
+  -DCMAKE_OSX_ARCHITECTURES:STRING="$architectures" \
+  -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING="$deployment_target" \
+  -DHERMES_ENABLE_DEBUGGER:BOOLEAN="$enable_debugger" \
+  -DHERMES_ENABLE_INTL:BOOLEAN=true \
+  -DHERMES_ENABLE_LIBFUZZER:BOOLEAN=false \
+  -DHERMES_ENABLE_FUZZILLI:BOOLEAN=false \
+  -DHERMES_ENABLE_TEST_SUITE:BOOLEAN=false \
+  -DHERMES_ENABLE_BITCODE:BOOLEAN=false \
+  -DHERMES_BUILD_APPLE_FRAMEWORK:BOOLEAN=true \
+  -DHERMES_BUILD_APPLE_DSYM:BOOLEAN=true \
+  -DHERMES_ENABLE_TOOLS:BOOLEAN="$build_cli_tools" \
+  -DIMPORT_HERMESC:PATH="${hermesc_path}" \
+  -DHERMES_RELEASE_VERSION="for RN $release_version" \
+  -DCMAKE_INSTALL_PREFIX:PATH="${PODS_ROOT}/hermes-engine/destroot" \
+  -DCMAKE_BUILD_TYPE="$CONFIGURATION"
+
+echo "Build Apple framework"
+
+"$CMAKE_BINARY" \
+  --build "${PODS_ROOT}/hermes-engine/build/${PLATFORM_NAME}" \
+  --target "install/strip" \
+  -j "$(sysctl -n hw.ncpu)"

sdks/hermes-engine/utils/build-hermesc-xcode.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+set -x
+
+hermesc_dir_path="$1"
+
+CMAKE_BINARY=${CMAKE_BINARY:-cmake}
+
+if ! "$CMAKE_BINARY" -S . -B "$hermesc_dir_path"
+then
+    echo "Failed to configure Hermesc cmake project."
+    exit 1
+fi
+if ! "$CMAKE_BINARY" --build "$hermesc_dir_path" --target hermesc -j "$(sysctl -n hw.ncpu)"
+then
+    echo "Failed to build Hermesc cmake project."
+    exit 1
+fi

sdks/hermes-engine/utils/copy-hermes-xcode.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+set -x
+
+source="${PODS_ROOT}/hermes-engine/destroot/Library/Frameworks/${PLATFORM_NAME}/hermes.framework"
+
+if [[ ! -f "$source" ]]; then
+    cp -R "$source" "${PODS_XCFRAMEWORKS_BUILD_DIR}/hermes/hermes.framework"
+    cp -R "$source" "${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}"
+fi