|
1 | | -### MacDeploy ### |
| 1 | +# MacOS Deployment |
2 | 2 |
|
3 | | -For Snow Leopard (which uses [Python 2.6](http://www.python.org/download/releases/2.6/)), you will need the param_parser package: |
| 3 | +The `macdeployqtplus` script should not be run manually. Instead, after building as usual: |
4 | 4 |
|
5 | | - sudo easy_install argparse |
| 5 | +```bash |
| 6 | +make deploy |
| 7 | +``` |
6 | 8 |
|
7 | | -This script should not be run manually, instead, after building as usual: |
| 9 | +During the deployment process, the disk image window will pop up briefly |
| 10 | +when the fancy settings are applied. This is normal, please do not interfere, |
| 11 | +the process will unmount the DMG and cleanup before finishing. |
8 | 12 |
|
9 | | - make deploy |
| 13 | +When complete, it will have produced `Bitcoin-Qt.dmg`. |
10 | 14 |
|
11 | | -During the process, the disk image window will pop up briefly where the fancy |
12 | | -settings are applied. This is normal, please do not interfere. |
| 15 | +## SDK Extraction |
13 | 16 |
|
14 | | -When finished, it will produce `Bitcoin-Qt.dmg`. |
| 17 | +`Xcode.app` is packaged in a `.xip` archive. |
| 18 | +This makes the SDK less-trivial to extract on non-macOS machines. |
| 19 | +One approach (tested on Debian Buster) is outlined below: |
15 | 20 |
|
| 21 | +```bash |
| 22 | + |
| 23 | +apt install clang cpio git liblzma-dev libxml2-dev libssl-dev make |
| 24 | + |
| 25 | +git clone https://github.com/tpoechtrager/xar |
| 26 | +pushd xar/xar |
| 27 | +./configure |
| 28 | +make |
| 29 | +make install |
| 30 | +popd |
| 31 | + |
| 32 | +git clone https://github.com/NiklasRosenstein/pbzx |
| 33 | +pushd pbzx |
| 34 | +clang -llzma -lxar pbzx.c -o pbzx -Wl,-rpath=/usr/local/lib |
| 35 | +popd |
| 36 | + |
| 37 | +xar -xf Xcode_10.2.1.xip -C . |
| 38 | + |
| 39 | +./pbzx/pbzx -n Content | cpio -i |
| 40 | + |
| 41 | +find Xcode.app -type d -name MacOSX.sdk -execdir sh -c 'tar -c MacOSX.sdk/ | gzip -9n > /MacOSX10.14.sdk.tar.gz' \; |
| 42 | +``` |
| 43 | + |
| 44 | +on macOS the process is more straightforward: |
| 45 | + |
| 46 | +```bash |
| 47 | +xip -x Xcode_10.2.1.xip |
| 48 | +tar -C Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.14.sdk.tar.gz MacOSX.sdk |
| 49 | +``` |
| 50 | + |
| 51 | +Our previously used macOS SDK (`MacOSX10.11.sdk`) can be extracted from |
| 52 | +[Xcode 7.3.1 dmg](https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/Xcode_7.3.1/Xcode_7.3.1.dmg). |
| 53 | +The script [`extract-osx-sdk.sh`](./extract-osx-sdk.sh) automates this. First |
| 54 | +ensure the DMG file is in the current directory, and then run the script. You |
| 55 | +may wish to delete the `intermediate 5.hfs` file and `MacOSX10.11.sdk` (the |
| 56 | +directory) when you've confirmed the extraction succeeded. |
| 57 | + |
| 58 | +```bash |
| 59 | +apt-get install p7zip-full sleuthkit |
| 60 | +contrib/macdeploy/extract-osx-sdk.sh |
| 61 | +rm -rf 5.hfs MacOSX10.11.sdk |
| 62 | +``` |
| 63 | + |
| 64 | +## Deterministic macOS DMG Notes |
| 65 | +Working macOS DMGs are created in Linux by combining a recent `clang`, the Apple |
| 66 | +`binutils` (`ld`, `ar`, etc) and DMG authoring tools. |
| 67 | + |
| 68 | +Apple uses `clang` extensively for development and has upstreamed the necessary |
| 69 | +functionality so that a vanilla clang can take advantage. It supports the use of `-F`, |
| 70 | +`-target`, `-mmacosx-version-min`, and `--sysroot`, which are all necessary when |
| 71 | +building for macOS. |
| 72 | + |
| 73 | +Apple's version of `binutils` (called `cctools`) contains lots of functionality missing in the |
| 74 | +FSF's `binutils`. In addition to extra linker options for frameworks and sysroots, several |
| 75 | +other tools are needed as well such as `install_name_tool`, `lipo`, and `nmedit`. These |
| 76 | +do not build under Linux, so they have been patched to do so. The work here was used as |
| 77 | +a starting point: [mingwandroid/toolchain4](https://github.com/mingwandroid/toolchain4). |
| 78 | + |
| 79 | +In order to build a working toolchain, the following source packages are needed from |
| 80 | +Apple: `cctools`, `dyld`, and `ld64`. |
| 81 | + |
| 82 | +These tools inject timestamps by default, which produce non-deterministic binaries. The |
| 83 | +`ZERO_AR_DATE` environment variable is used to disable that. |
| 84 | + |
| 85 | +This version of `cctools` has been patched to use the current version of `clang`'s headers |
| 86 | +and its `libLTO.so` rather than those from `llvmgcc`, as it was originally done in `toolchain4`. |
| 87 | + |
| 88 | +To complicate things further, all builds must target an Apple SDK. These SDKs are free to |
| 89 | +download, but not redistributable. To obtain it, register for an Apple Developer Account, |
| 90 | +then download [Xcode 10.2.1](https://download.developer.apple.com/Developer_Tools/Xcode_10.2.1/Xcode_10.2.1.xip). |
| 91 | + |
| 92 | +This file is many gigabytes in size, but most (but not all) of what we need is |
| 93 | +contained only in a single directory: |
| 94 | + |
| 95 | +```bash |
| 96 | +Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk |
| 97 | +``` |
| 98 | + |
| 99 | +See the SDK Extraction notes above for how to obtain it. |
| 100 | + |
| 101 | +The Gitian descriptors build 2 sets of files: Linux tools, then Apple binaries which are |
| 102 | +created using these tools. The build process has been designed to avoid including the |
| 103 | +SDK's files in Gitian's outputs. All interim tarballs are fully deterministic and may be freely |
| 104 | +redistributed. |
| 105 | + |
| 106 | +`genisoimage` is used to create the initial DMG. It is not deterministic as-is, so it has been |
| 107 | +patched. A system `genisoimage` will work fine, but it will not be deterministic because |
| 108 | +the file-order will change between invocations. The patch can be seen here: [cdrkit-deterministic.patch](https://github.com/bitcoin/bitcoin/blob/master/depends/patches/native_cdrkit/cdrkit-deterministic.patch). |
| 109 | +No effort was made to fix this cleanly, so it likely leaks memory badly, however it's only used for |
| 110 | +a single invocation, so that's no real concern. |
| 111 | + |
| 112 | +`genisoimage` cannot compress DMGs, so afterwards, the DMG tool from the |
| 113 | +`libdmg-hfsplus` project is used to compress it. There are several bugs in this tool and its |
| 114 | +maintainer has seemingly abandoned the project. |
| 115 | + |
| 116 | +The DMG tool has the ability to create DMGs from scratch as well, but this functionality is |
| 117 | +broken. Only the compression feature is currently used. Ideally, the creation could be fixed |
| 118 | +and `genisoimage` would no longer be necessary. |
| 119 | + |
| 120 | +Background images and other features can be added to DMG files by inserting a |
| 121 | +`.DS_Store` before creation. This is generated by the script `contrib/macdeploy/custom_dsstore.py`. |
| 122 | + |
| 123 | +As of OS X 10.9 Mavericks, using an Apple-blessed key to sign binaries is a requirement in |
| 124 | +order to satisfy the new Gatekeeper requirements. Because this private key cannot be |
| 125 | +shared, we'll have to be a bit creative in order for the build process to remain somewhat |
| 126 | +deterministic. Here's how it works: |
| 127 | + |
| 128 | +- Builders use Gitian to create an unsigned release. This outputs an unsigned DMG which |
| 129 | + users may choose to bless and run. It also outputs an unsigned app structure in the form |
| 130 | + of a tarball, which also contains all of the tools that have been previously (deterministically) |
| 131 | + built in order to create a final DMG. |
| 132 | +- The Apple keyholder uses this unsigned app to create a detached signature, using the |
| 133 | + script that is also included there. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs). |
| 134 | +- Builders feed the unsigned app + detached signature back into Gitian. It uses the |
| 135 | + pre-built tools to recombine the pieces into a deterministic DMG. |
0 commit comments