Skip to content
This repository was archived by the owner on Jul 1, 2023. It is now read-only.

Commit 9537148

Browse files
compnerddan-zhengBradLarson
authored
Documentation: update the developer documentation (#1157)
Documentation: update the developer documentation This adds documentation covering how to build X10 for Windows, Linux, and macOS as well as links to prebuilt binaries. Additionally, document the required additional flags for building Swift APIs. This is required in order to build with a toolchain which does not have X10 integrated into the image. It also paves the way towards building Swift APIs with the Swift.org toolchain. Add a note about SIP and quarantining. Co-authored-by: Dan Zheng <[email protected]> Co-authored-by: Brad Larson <[email protected]>
1 parent 1a7f355 commit 9537148

File tree

2 files changed

+229
-12
lines changed

2 files changed

+229
-12
lines changed

‎Documentation/Development.md

Lines changed: 229 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,29 +4,245 @@
44

55
### Requirements
66

7-
* [Swift for TensorFlow toolchain][toolchain].
8-
* An environment that can run the Swift for TensorFlow toolchains: Ubuntu 18.04, macOS with Xcode 10, or Windows 10.
7+
* The latest Swift toolchain snapshot from [swift.org](https://swift.org/download/#sanpshots).
8+
9+
### Building
10+
11+
### X10
12+
13+
X10 provides the underlying lazy tensor implementation that is used for the
14+
tensor computation in the TensorFlow Swift APIs. This library builds atop of
15+
the TensorFlow library, using the XLA compiler to perform optimizations over
16+
the tensor computation.
17+
18+
#### (Option 1) Use a prebuilt version of X10
19+
20+
You can use a prebuilt version of the X10 library for building the TensorFlow
21+
Swift APIs package if you do not need to make changes to the X10 library
22+
implementation.
23+
24+
There are prebuilt versions of the X10 library for certain platforms. If a
25+
prebuilt library is unavailable for your desired platform, you can build X10
26+
from source following the instructions below.
27+
28+
- [Windows 10 (x64)](https://artprodeus21.artifacts.visualstudio.com/A8fd008a0-56bc-482c-ba46-67f9425510be/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2NvbXBuZXJkL3Byb2plY3RJZC8zMTMzZDZhYi04MGE4LTQ5OTYtYWM0Zi0wM2RmMjVjZDMyMjQvYnVpbGRJZC80Mzc2OC9hcnRpZmFjdE5hbWUvdGVuc29yZmxvdy13aW5kb3dzLXg2NA2/content?format=zip)
29+
- [macOS (x64)](https://artprodeus21.artifacts.visualstudio.com/A8fd008a0-56bc-482c-ba46-67f9425510be/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2NvbXBuZXJkL3Byb2plY3RJZC8zMTMzZDZhYi04MGE4LTQ5OTYtYWM0Zi0wM2RmMjVjZDMyMjQvYnVpbGRJZC80Mzc2OC9hcnRpZmFjdE5hbWUvdGVuc29yZmxvdy1kYXJ3aW4teDY00/content?format=zip)
30+
31+
The location where you extract the prebuilt library is important to remember
32+
as it will be required in building the TensorFlow Swift APIs.
33+
34+
#### (Option 2) Building X10
35+
36+
Although using the prebuilt libraries is more convenient, there may be some
37+
situations where you may need to build X10 yourself. These include:
38+
39+
1. You are targeting a platform where we do not have prebuilt binaries for X10
40+
2. You are attempting to make changes to the X10 implementation itself, and must
41+
therefore build a new version.
42+
43+
#### Requirements
44+
945
* Bazel. This can be installed [manually][bazel] or with
1046
[Bazelisk][bazelisk]. You will need a version supported by TensorFlow
1147
(between `_TF_MIN_BAZEL_VERSION` and `_TF_MAX_BAZEL_VERSION` as specified in
1248
[tensorflow/configure.py][configure.py]).
1349
* Python3 with [numpy][numpy].
14-
* CMake. CMake 3.16 or newer is required to build with CMake.
1550

16-
### Building and testing
51+
#### Building
52+
53+
The X10 implementation is distributed as part of the Swift for TensorFlow APIs
54+
repository. It consists of two halves:
55+
56+
1. [XLA Client](Sources/x10/xla_client): provides an abstract interface for dispatching XLA computations on devices
57+
2. [X10](Sources/x10/xla_tensor): Lowering rules and tracing support for Tensors
58+
59+
The library is designed to be built as part of the
60+
[tensorflow](https://github.com/tensorflow/tensorflow) build. As such, in
61+
order to build X10, you must build tensorflow.
62+
63+
Currently X10 is developed against TensorFlow 2.4.0. The following build
64+
scripts provide commands to build on common platforms. They largely replicate
65+
the build instructions for TensorFlow. The instructions diverge in that we
66+
must copy the additional X10 library sources into the tensorflow repository.
67+
The following table identifies the copied locations:
68+
69+
| swift-apis source | tensorflow destination |
70+
|-------------------|------------------------|
71+
| Sources/CX10 | swift_bindings |
72+
| Sources/x10/xla_client | tensorflow/compiler/xla/xla_client |
73+
| Sources/x10/xla_tensor | tensorflow/compiler/tf2xla/xla_tensor |
74+
75+
We build two specific targets:
76+
1. `//tensorflow:tensorflow`
77+
2. `//tensorflow/compiler/tf2xla/xla_tensor:x10`
78+
79+
On Windows, we build the additional targets to allow us to link against the
80+
libraries:
81+
1. `//tensorflow:tensorflow_dll_import_lib`
82+
2. `//tensorflow/compiler/tf2xla/xla_tensor:x10_dll_import_lib `
83+
84+
We must pass the `--nocheck_visibility` flag to bazel to accomodate the new
85+
libraries.
86+
87+
<details>
88+
<summary>Windows Build Script</summary>
89+
90+
91+
*Note: This must be executed in the x64 Native Developer Command Prompt*
92+
93+
*Note: You will either need to be running with elevated privilleges, have rights to create symbolic links and junctions, or have enabled developer mode to successfully create the directory junctions. You may alternatively copy the sources instead of creating a junction.*
94+
95+
```cmd
96+
:: clone swift-apis
97+
git clone git://github.com/tensorflow/swift-apis
98+
:: checkout tensorflow
99+
git clone --depth 1 --no-tags git://github.com/tensorflow/tensorflow
100+
git -C tensorflow checkout refs/heads/r2.4
101+
102+
:: Link X10 into the source tree
103+
mklink /J %CD%\tensorflow\swift_bindings %CD%\swift-apis\Sources\CX10
104+
mklink /J %CD%\tensorflow\tensorflow\compiler\xla\xla_client %CD%\swift-apis\Sources\x10\xla_client
105+
mklink /J %CD%\tensorflow\tensorflow\compiler\tf2xla\xla_tensor %CD%\swift-apis\Sources\x10\xla_tensor
106+
107+
:: configure path - we need the Git tools in the path
108+
path %ProgramFiles%\Git\usr\bin;%PATH%
109+
:: ensure that python dependencies are available
110+
python -m pip install --user numpy six
111+
:: configure X10/TensorFlow
112+
set TF_ENABLE_XLA=1
113+
set TF_NEED_ROCM=0
114+
set TF_NEED_CUDA=0
115+
set TF_CUDA_COMPUTE_CAPABILITIES=7.5
116+
set CC_OPT_FLAGS="/arch:AVX /D_USE_MATH_DEFINES"
117+
set TF_OVERRIDE_EIGEN_STRONG_INLINE=1
118+
.\tensorflow\configure.py
119+
:: build
120+
set BAZEL_SH=%ProgramFiles%\Git\usr\bin\bash.exe
121+
set BAZEL_VC=%VCINSTALLDIR%
122+
bazel --output_user_root %CD%/caches/bazel/tensorflow build -c opt --copt /D_USE_MATH_DEFINES --define framework_shared_object=false --config short_logs --nocheck_visibility //tensorflow:tensorflow //tensorflow:tensorflow_dll_import_lib //tensorflow/compiler/tf2xla/xla_tensor:x10 //tensorflow/compiler/tf2xla/xla_tensor:x10_dll_import_lib
123+
:: terminate bazel daemon
124+
bazel --output_user_root %CD%/caches/bazel/tensorflow shutdown
125+
126+
:: package
127+
set DESTDIR=%CD%\Library\tensorflow-windows-%VSCMD_ARG_TGT_ARCH%\tensorflow-2.4.0
128+
129+
md %DESTDIR\usr\bin
130+
copy tensorflow\bazel-bin\tensorflow\tensorflow.dll %DESTDIR%\usr\bin\
131+
copy tensorflow\bazel-bin\tensorflow\compiler\tf2xla\xla_tensor\x10.dll %DESTDIR%\usr\bin\
132+
133+
md %DESTDIR%\usr\lib
134+
copy tensorflow\bazel-out\%VSCMD_ARG_TGT_ARCH%_windows-opt\bin\tensorflow\tensorflow.lib %DESTDIR%\usr\lib\
135+
copy tensorflow\bazel-out\%VSCMD_ARG_TGT_ARCH%_windows-opt\bin\tensorflow\compiler\tf2xla\xla_tensor\x10.lib %DESTDIR%\usr\lib\
136+
137+
md %DESTDIR%\usr\include\tensorflow\c
138+
copy tensorflow\tensorflow\c\c_api.h %DESTDIR%\usr\include\tensorflow\c\
139+
copy tensorflow\tensorflow\c\c_api_experimental.h %DESTDIR%\usr\include\tensorflow\c\
140+
copy tensorflow\tensorflow\c\tf_attrtype.h %DESTDIR%\usr\include\tensorflow\c\
141+
copy tensorflow\tensorflow\c\tf_datatype.h %DESTDIR%\usr\include\tensorflow\c\
142+
copy tensorflow\tensorflow\c\tf_file_statistics.h %DESTDIR%\usr\include\tensorflow\c\
143+
copy tensorflow\tensorflow\c\tf_status.h %DESTDIR%\usr\include\tensorflow\c\
144+
copy tensorflow\tensorflow\c\tf_tensor.h %DESTDIR%\usr\include\tensorflow\c\
145+
146+
md %DESTDIR%\usr\include\tensorflow\c\eager
147+
cp tensorflow\tensorflow\c\eager\c_api.h %DESTDIR%\usr\include\tensorflow\c\eager\
148+
149+
md %DESTDIR%\usr\include\x10
150+
copy swift-apis\Sources\x10\swift_bindings\device_wrapper.h %DESTDIR%\usr\include\x10\
151+
copy swift-apis\Sources\x10\swift_bindings\xla_tensor_tf_ops.h %DESTDIR%\usr\include\x10\
152+
copy swift-apis\Sources\x10\swift_bindings\xla_tensor_wrapper.h %DESTDIR%\usr\include\x10\
153+
154+
md %DESTDIR%\usr\share
155+
copy tensorflow\bazel-out\%VSCMD_ARG_TGT_ARCH%_windows-opt\bin\tensorflow\tensorflow_filtered_def_file.def %DESTDIR%\usr\share
156+
```
157+
</details>
158+
159+
<details>
160+
<summary>macOS/Linux Build Script</summary>
161+
162+
*Note: If you are unable to run bazel on macOS due to an error about an unverified developer due to System Integrity Protection (SIP), you can use `xattr -dr com.apple.quarantine bazel`*
163+
164+
```bash
165+
# clone swift-apis
166+
git clone git://github.com/tensorflow/swift-apis
167+
# checkout tensorflow
168+
git clone --depth 1 --no-tags git://github.com/tensorflow/tensorflow
169+
git -C tensorflow checkout refs/heads/r2.4
170+
171+
# Link X10 into the source tree
172+
ln -sf ${PWD}/swift-apis/Sources/CX10 ${PWD}/tensorflow/swift_bindings
173+
ln -sf ${PWD}/swift-apis/Sources/x10/xla_client ${PWD}/tensorflow/tensorflow/compiler/xla/xla_client
174+
ln -sf ${PWD}/swift-apis/Sources/x10/xla_tensor ${PWD}/tensorflow/tensorflow/compiler/tf2xla/xla_tensor
175+
176+
# ensure that python dependencies are available
177+
python3 -m pip install --user numpy six
178+
# configure X10/TensorFlow
179+
export PYTHON_BIN_PATH=$(which python3)
180+
export USE_DEFAULT_PYTHON_LIB_PATH=1
181+
export TF_NEED_OPENCL_SYCL=0
182+
export TF_DOWNLOAD_CLANG=0
183+
export TF_SET_ANDROID_WORKSPACE=0
184+
export TF_CONFIGURE_IOS=0
185+
export TF_ENABLE_XLA=1
186+
export TF_NEED_ROCM=0
187+
export TF_NEED_CUDA=0
188+
export TF_CUDA_COMPUTE_CAPABILITIES=7.5
189+
export CC_OPT_FLAGS="-march=native"
190+
python3 ./tensorflow/configure.py
191+
bazel --output_user_root ${PWD}/caches/bazel/tensorflow build -c opt --define framework_shared_object=false --config short_logs --nocheck_visibility //tensorflow:tensorflow //tensorflow/compiler/tf2xla/xla_tensor:x10
192+
# terminate bazel daemon
193+
bazel --output_user_root ${PWD}/caches/bazel/tensorflow shutdown
194+
195+
# package
196+
DESTDIR=${PWD}/Library/tensorflow-$(echo $(uname -s) | tr 'A-Z' 'a-z')-$(uname -m)/tensorflow-2.4.0
197+
198+
mkdir -p ${DESTDIR}/usr/lib
199+
cp tensorflow/bazel-bin/tensorflow/libtensorflow-2.4.0.(dylib|so) ${DESTDIR}/usr/lib/
200+
cp tensorflow/bazel-bin/tensorflow/compiler/tf2xla/xla_tensor/libx10.(dylib|so) ${DESTDIR}/usr/lib/
201+
202+
mkdir -p ${DESTDIR}/usr/include/tensorflow/c
203+
cp tensorflow/tensorflow/c/c_api.h ${DESTDIR}/usr/include/tensorflow/c/
204+
cp tensorflow/tensorflow/c/c_api_experimental.h ${DESTDIR}/usr/include/tensorflow/c/
205+
cp tensorflow/tensorflow/c/tf_attrtype.h ${DESTDIR}/usr/include/tensorflow/c/
206+
cp tensorflow/tensorflow/c/tf_datatype.h ${DESTDIR}/usr/include/tensorflow/c/
207+
cp tensorflow/tensorflow/c/tf_file_statistics.h ${DESTDIR}/usr/include/tensorflow/c/
208+
cp tensorflow/tensorflow/c/tf_status.h ${DESTDIR}/usr/include/tensorflow/c/
209+
cp tensorflow/tensorflow/c/tf_tensor.h ${DESTDIR}/usr/include/tensorflow/c/
210+
211+
mkdir -p ${DESTDIR}/usr/include/tensorflow/c/eager
212+
cp tensorflow/tensorflow/c/eager/c_api.h ${DESTDIR}/usr/include/tensorflow/c/eager/
213+
214+
mkdir -p ${DESTDIR}/usr/include/x10
215+
cp swift-apis/Sources/x10/swift_bindings/device_wrapper.h ${DESTDIR}/usr/include/x10/
216+
cp swift-apis/Sources/x10/swift_bindings/xla_tensor_tf_ops.h ${DESTDIR}/usr/include/x10/
217+
cp swift-apis/Sources/x10/swift_bindings/xla_tensor_wrapper.h ${DESTDIR}/usr/include/x10/
218+
```
219+
</details>
220+
221+
### (TensorFlow) Swift APIs
17222

18223
#### SwiftPM
19224

20225
*Note: Building with SwiftPM does not include changes to X10 modules.*
21226

227+
Because X10 is a required component for building the TensorFlow Swift APIs,
228+
and the project does not build with Swift Package Manager, we must pass along
229+
the necessary information to the build. We need to pass two items to the
230+
build:
231+
232+
1. the location of the X10 & TensorFlow headers
233+
2. the location of the X10 & TensorFlow libraries
234+
235+
The path to these libraries is not fixed and depends on your machine setup.
236+
You should substitute the paths with the appropriate values. In the example
237+
commands below, we assume that the library is packaged in a traditional Unix
238+
style layout and placed in `/Library/tensorflow-2.4.0`.
239+
22240
```shell
23-
$ swift build
241+
$ swift build -Xcc -I/Library/tensorflow-2.4.0/usr/include -Xlinker -L/Library/tensorflow-2.4.0/usr/lib
24242
```
25243

26-
*Note: Testing with SwiftPM does not run X10 tests.*
27-
28244
```shell
29-
$ swift test
245+
$ swift test -Xcc -I/Library/tensorflow-2.4.0/usr/include -Xlinker -L/Library/tensorflow-2.4.0/usr/lib
30246
```
31247

32248
#### CMake
@@ -76,3 +292,8 @@ tests. This avoids an error: `cannot load underlying module for 'XCTest'`.
76292
cmake -B out -D USE_BUNDLED_CTENSORFLOW=YES -D USE_BUNDLED_X10=YES -D BUILD_TESTING=NO -G Ninja -S swift-apis
77293
cmake --build out
78294
```
295+
296+
[bazel]: https://docs.bazel.build/versions/master/install.html
297+
[bazelisk]: https://github.com/bazelbuild/bazelisk
298+
[configure.py]: https://github.com/tensorflow/tensorflow/blob/master/configure.py
299+
[numpy]: https://numpy.org/

‎README.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -125,8 +125,4 @@ participating.
125125
[gradient]: https://www.tensorflow.org/swift/api_docs/Functions#/s:10TensorFlow8gradient2at2in13TangentVectorQzx_AA0A0Vyq_GxXEtAA14DifferentiableRzAA0aB13FloatingPointR_r0_lF
126126
[swift-models]: https://github.com/tensorflow/swift-models
127127
[toolchain]: https://github.com/tensorflow/swift/blob/main/Installation.md
128-
[bazel]: https://docs.bazel.build/versions/master/install.html
129-
[bazelisk]: https://github.com/bazelbuild/bazelisk
130-
[configure.py]: https://github.com/tensorflow/tensorflow/blob/master/configure.py
131-
[numpy]: https://numpy.org/
132128
[forum]: https://groups.google.com/a/tensorflow.org/d/forum/swift

0 commit comments

Comments
 (0)