diff --git a/README.md b/README.md index 08da9e1..cb2b961 100644 --- a/README.md +++ b/README.md @@ -6,23 +6,9 @@ This repository contains infrastructure for providing this support. [ap]: https://azure.microsoft.com/en-us/services/devops/pipelines/ - -## Custom `cross` program - -In order to do all sorts of tricky cross-compilation tests, we need to use a -customized version of the [`cross`] program. The `custom-cross/` directory -contains the files needed to create it. We do a creative/ingenious/weird/dumb -thing where we distribute the customized binary as a Docker container that just -copies the binary out of the container to the host. - -[`cross`]: https://github.com/cross-rs/cross - -See the `README.md` in the `custom-cross` subdirectory for more information. - - ## Custom `cross` images -As a less weird thing, we need to create custom images used by [`cross`] to +We need to create custom images used by [`cross`] to build Tectonic. Scripts to create them are in the `cross-images/` directory. See the `README.md` in the `cross-images` subdirectory for more information. @@ -38,5 +24,6 @@ Check the Git history for: - Updated `libharfbuzz` package for old Ubuntus - Old CI helper tool `ttcitool`, superseded by [Cranko] - Custom PPC build chroot for testing Tectonic on bigendian systems +- Custom cross build for custom docker behavior on old cross versions [Cranko]: https://pkgw.github.io/cranko/ diff --git a/cross-images/README.md b/cross-images/README.md index b7cbdd4..8a9889e 100644 --- a/cross-images/README.md +++ b/cross-images/README.md @@ -24,9 +24,9 @@ to compile more of the tooling yourself: analogous if you specify a different target; all appearances of the target name below will likewise track your choice). 1. Obtain a copy of the [tectonic] source code. -1. Within the Tectonic source tree, unpack the [custom Tectonic `cross` program][1]: +1. Install cross: ``` - docker run --rm -v $(pwd):/work:rw,Z tectonictypesetting/ttcross:latest + cargo install cross ``` 1. It is often necessary to delete your entire `target` tree before a new cross build. If you don't, you may see an error like: @@ -36,13 +36,13 @@ to compile more of the tooling yourself: This is because the containerized cross build is not smart enough to differentiate between build scripts built for your true host OS, and the host OS of the cross Docker container. -1. Use that program to execute the cross build: +1. Execute the cross build: ``` - ./cross build --target=x86_64-unknown-linux-musl --release + cross build --target=x86_64-unknown-linux-musl --release ``` 1. The final output will be found at `target/x86_64-unknown-linux-musl/release/tectonic`. -1. The aspiration is that `./cross test ...` will work, although this is often +1. The aspiration is that `cross test ...` will work, although this is often dicey in cross-compilation scenarios. In order to run any cross-compiled binaries, you may need to have QEmu handlers registered with your kernel, which can be achieved conveniently with: @@ -50,8 +50,6 @@ to compile more of the tooling yourself: docker run --rm --privileged multiarch/qemu-user-static:register --reset --credential yes ``` -[1]: ../custom-cross/README.md - ## Adding New Targets @@ -97,7 +95,7 @@ docker run \ To set up an environment that can run an actual build, many more settings are needed. The most reasonable way to determine them is to run a program like `ps -xawww` while a `./cross build` is running. +xawww` while a `cross build` is running. ## Static Build Implementation diff --git a/custom-cross/001-privileges.patch b/custom-cross/001-privileges.patch deleted file mode 100644 index e74a4a9..0000000 --- a/custom-cross/001-privileges.patch +++ /dev/null @@ -1,15 +0,0 @@ -diff --git a/src/docker.rs b/src/docker.rs -index 4ac2733..2ebf59a 100644 ---- a/src/docker.rs -+++ b/src/docker.rs -@@ -111,7 +111,9 @@ pub fn run(target: &Target, - - docker - .arg("--rm") -- .args(&["--user", &format!("{}:{}", id::user(), id::group())]) -+ .arg("--privileged") -+ .args(&["-e", &format!("HOST_UID={}", id::user())]) -+ .args(&["-e", &format!("HOST_GID={}", id::group())]) - .args(&["-e", "XARGO_HOME=/xargo"]) - .args(&["-e", "CARGO_HOME=/cargo"]) - .args(&["-e", "CARGO_TARGET_DIR=/target"]) diff --git a/custom-cross/002-rw-project.patch b/custom-cross/002-rw-project.patch deleted file mode 100644 index 715cf9f..0000000 --- a/custom-cross/002-rw-project.patch +++ /dev/null @@ -1,13 +0,0 @@ -diff --git a/src/docker.rs b/src/docker.rs -index 4ac2733..9c8ab88 100644 ---- a/src/docker.rs -+++ b/src/docker.rs -@@ -155,7 +157,7 @@ pub fn run(target: &Target, - .args(&["-v", &format!("{}:/xargo:Z", xargo_dir.display())]) - .args(&["-v", &format!("{}:/cargo:Z", cargo_dir.display())]) - .args(&["-v", "/cargo/bin"]) // Prevent `bin` from being mounted inside the Docker container. -- .args(&["-v", &format!("{}:/project:Z,ro", root.display())]) -+ .args(&["-v", &format!("{}:/project:Z,rw", root.display())]) - .args(&["-v", &format!("{}:/rust:Z,ro", sysroot.display())]) - .args(&["-v", &format!("{}:/target:Z", target_dir.display())]) - .args(&["-w", "/project"]); diff --git a/custom-cross/Dockerfile b/custom-cross/Dockerfile deleted file mode 100644 index 216647d..0000000 --- a/custom-cross/Dockerfile +++ /dev/null @@ -1,35 +0,0 @@ -# Copyright 2019 the Tectonic Project -# Licensed under the MIT License. - -FROM ubuntu:18.04 AS builder -ARG tag=v0.1.16 - -RUN TERM=dumb \ - apt-get update \ - && apt-get install -y \ - build-essential \ - curl \ - git \ - patch \ - && apt-get clean \ - && rm -rf /var/lib/apt/lists/* - -RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs |sh -s -- -y - -ADD 001-privileges.patch / -ADD 002-rw-project.patch / -RUN git clone https://github.com/rust-embedded/cross.git \ - && cd cross \ - && git checkout $tag \ - && patch -p1 -i /001-privileges.patch \ - && patch -p1 -i /002-rw-project.patch - -RUN cd cross && PATH=$HOME/.cargo/bin:$PATH cargo install --force --path . - -# Use Docker's fancy multiple-FROM / COPY --from functionality to make a super -# minimal container that just copies the binary out into the host system. - -FROM alpine:3.10 -COPY --from=builder /root/.cargo/bin/cross / -VOLUME /work -CMD ["cp", "/cross", "/work/"] diff --git a/custom-cross/README.md b/custom-cross/README.md deleted file mode 100644 index 71459c0..0000000 --- a/custom-cross/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# Tectonic CI support: custom `cross` program - -This section of the repository creates a Docker container that hosts a -customized build of the [cross](https://github.com/cross-rs/cross) command. - - -## Installation - -To install the custom `cross` program, run: - -``` -docker run --rm -v $(pwd):/work:rw,Z tectonictypesetting/ttcross:latest -``` - -This will copy a file named `cross` into the current directory (or whatever you -mount to `/work` within the Docker container). The file is an x86-64 Linux -executable with minimal dependencies. - -This is basically a silly way to distribute a pre-built binary, and it could be -superseded with any of a variety of other mechanisms. But it's what we have -right now. - - -## Development - -The files in this directory can be used to build the -`tectonictypesetting/ttcross` Docker image referenced above. Run: - -``` -docker build -t tectonictypesetting/ttcross:latest . -``` - -This framework is currently based on a very old version of the `cross` program, -and its build infrastructure could use a refresh. Fortunately (sort of), the -existing program suffices for Tectonic’s CI. - - -## Modifications - -We patch `cross` to run the build containers with the `--privileged` option, -which is necessary to enable the chroot magic that happens inside the builders. -Because of this we have to enter the container as root and have it switch to the -correct (host) UID. - -We also patch it to mount the `/project` directory as read-write, since the -Tectonic test suite wants to create a few files there. The test suite should -be fixed to emit these into `/target`, but I'm being lazy.