Skip to content

Commit 5d51d32

Browse files
committed
tc_build: Add basic support for building Rust
There a lot of options missing and it does not do any fancy kind of build. But it is a start, and it is already useful, e.g. Peter could have used it to test the new KCFI arity flag that requires LLVM 21 but upstream Rust still uses LLVM 20. I took the approach that the new script only takes care of building Rust provided an existing LLVM, which seemed simple and clear. Thus add the basic infrastructure, plus a bit of documentation. Add it to the CI, too. I successfully built it in a clean Debian 12 and Fedora 41. Cc: Peter Zijlstra <[email protected]> Signed-off-by: Miguel Ojeda <[email protected]>
1 parent 9d62567 commit 5d51d32

File tree

5 files changed

+358
-1
lines changed

5 files changed

+358
-1
lines changed

.github/workflows/build.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ jobs:
1212
run: bash ci.sh deps
1313
- name: Build LLVM
1414
run: bash ci.sh llvm
15+
- name: Build Rust
16+
run: bash ci.sh rust
1517
- name: Build binutils
1618
run: bash ci.sh binutils
1719
- name: Build kernel

README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ These scripts have been tested in a Docker image of the following distributions
3737
lld \
3838
make \
3939
ninja-build \
40+
pkg-config \
4041
python3-dev \
4142
texinfo \
4243
u-boot-tools \
@@ -145,6 +146,19 @@ bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute
145146

146147
Having a standalone copy of binutils (ideally in the same folder at the LLVM toolchain so that only one `PATH` modification is needed) works around this without any adverse side effects. Another workaround is bind mounting the new `LLVMgold.so` to `/usr/lib/LLVMgold.so`.
147148

149+
## build-rust.py
150+
151+
By default, `./build-rust.py` will clone Rust and build it using an LLVM previously built by `./build-llvm.py`, e.g.:
152+
153+
```sh
154+
./build-llvm.py
155+
./build-rust.py
156+
```
157+
158+
This script does not apply any Rust-specific patches to LLVM.
159+
160+
Run `./build-rust.py -h` for more options and information.
161+
148162
## Contributing
149163

150164
This repository openly welcomes pull requests! There are a few presubmit checks that run to make sure the code stays consistently formatted and free of bugs.

build-rust.py

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
#!/usr/bin/env python3
2+
# pylint: disable=invalid-name
3+
4+
from argparse import ArgumentParser, RawTextHelpFormatter
5+
from pathlib import Path
6+
import textwrap
7+
import time
8+
9+
import tc_build.utils
10+
11+
from tc_build.rust import RustBuilder, RustSourceManager
12+
13+
# This is a known good revision of Rust for building the kernel
14+
GOOD_REVISION = '69b3959afec9b5468d5de15133b199553f6e55d2'
15+
16+
parser = ArgumentParser(formatter_class=RawTextHelpFormatter)
17+
clone_options = parser.add_mutually_exclusive_group()
18+
19+
parser.add_argument('--debug',
20+
help=textwrap.dedent('''\
21+
Build a debug compiler and standard library. This enables debug assertions,
22+
debug logging, overflow checks and debug info. The debug assertions and overflow
23+
checks can help catch issues when compiling.
24+
25+
'''),
26+
action='store_true')
27+
parser.add_argument('-b',
28+
'--build-folder',
29+
help=textwrap.dedent('''\
30+
By default, the script will create a "build/rust" folder in the same folder as this
31+
script and build each requested stage within that containing folder. To change the
32+
location of the containing build folder, pass it to this parameter. This can be either
33+
an absolute or relative path.
34+
35+
'''),
36+
type=str)
37+
parser.add_argument('-i',
38+
'--install-folder',
39+
help=textwrap.dedent('''\
40+
By default, the script will leave the toolchain in its build folder. To install it
41+
outside the build folder for persistent use, pass the installation location that you
42+
desire to this parameter. This can be either an absolute or relative path.
43+
44+
'''),
45+
type=str)
46+
parser.add_argument('-l',
47+
'--llvm-install-folder',
48+
help=textwrap.dedent('''\
49+
By default, the script will try to use a built LLVM by './build-llvm.py'. To use
50+
another LLVM installation (perhaps from './build-llvm.py --install-folder'), pass
51+
it to this parameter.
52+
53+
'''),
54+
type=str)
55+
parser.add_argument('-R',
56+
'--rust-folder',
57+
help=textwrap.dedent('''\
58+
By default, the script will clone the Rust project into the tc-build repo. If you have
59+
another Rust checkout that you would like to work out of, pass it to this parameter.
60+
This can either be an absolute or relative path. Implies '--no-update'. When this
61+
option is supplied, '--ref' and '--use-good-revision' do nothing, as the script does
62+
not manipulate a repository it does not own.
63+
64+
'''),
65+
type=str)
66+
parser.add_argument('-n',
67+
'--no-update',
68+
help=textwrap.dedent('''\
69+
By default, the script always updates the Rust repo before building. This prevents
70+
that, which can be helpful during something like bisecting or manually managing the
71+
repo to pin it to a particular revision.
72+
73+
'''),
74+
action='store_true')
75+
parser.add_argument('-r',
76+
'--ref',
77+
help=textwrap.dedent('''\
78+
By default, the script builds the main branch (tip of tree) of Rust. If you would
79+
like to build an older branch, use this parameter. This may be helpful in tracking
80+
down an older bug to properly bisect. This value is just passed along to 'git checkout'
81+
so it can be a branch name, tag name, or hash. This will have no effect if
82+
'--rust-folder' is provided, as the script does not manipulate a repository that it
83+
does not own.
84+
85+
'''),
86+
default='master',
87+
type=str)
88+
parser.add_argument('--show-build-commands',
89+
help=textwrap.dedent('''\
90+
By default, the script only shows the output of the comands it is running. When this option
91+
is enabled, the invocations of the build tools will be shown to help with reproducing
92+
issues outside of the script.
93+
94+
'''),
95+
action='store_true')
96+
clone_options.add_argument('--use-good-revision',
97+
help=textwrap.dedent('''\
98+
By default, the script updates Rust to the latest tip of tree revision, which may at times be
99+
broken or not work right. With this option, it will checkout a known good revision of Rust
100+
that builds and works properly. If you use this option often, please remember to update the
101+
script as the known good revision will change. This option may work best with a matching good
102+
revision used to build LLVM by './build-llvm.py'.
103+
104+
'''),
105+
action='store_const',
106+
const=GOOD_REVISION,
107+
dest='ref')
108+
parser.add_argument('--vendor-string',
109+
help=textwrap.dedent('''\
110+
Add this value to the Rust version string (like "rustc ... (ClangBuiltLinux)"). Useful when
111+
reverting or applying patches on top of upstream Rust to differentiate a toolchain built
112+
with this script from upstream Rust or to distinguish a toolchain built with this script
113+
from the system's Rust. Defaults to ClangBuiltLinux, can be set to an empty string to
114+
override this and have no vendor in the version string.
115+
116+
'''),
117+
type=str,
118+
default='ClangBuiltLinux')
119+
args = parser.parse_args()
120+
121+
# Start tracking time that the script takes
122+
script_start = time.time()
123+
124+
# Folder validation
125+
tc_build_folder = Path(__file__).resolve().parent
126+
src_folder = Path(tc_build_folder, 'src')
127+
128+
if args.build_folder:
129+
build_folder = Path(args.build_folder).resolve()
130+
else:
131+
build_folder = Path(tc_build_folder, 'build/rust')
132+
133+
if args.llvm_install_folder:
134+
llvm_install_folder = Path(args.llvm_install_folder).resolve()
135+
else:
136+
llvm_install_folder = Path(tc_build_folder, 'build/llvm/final')
137+
138+
# Validate and configure Rust source
139+
if args.rust_folder:
140+
if not (rust_folder := Path(args.rust_folder).resolve()).exists():
141+
raise RuntimeError(f"Provided Rust folder ('{args.rust_folder}') does not exist?")
142+
else:
143+
rust_folder = Path(src_folder, 'rust')
144+
rust_source = RustSourceManager(rust_folder)
145+
rust_source.download(args.ref)
146+
if not (args.rust_folder or args.no_update):
147+
rust_source.update(args.ref)
148+
149+
# Build Rust
150+
tc_build.utils.print_header('Building Rust')
151+
152+
# Final build
153+
final = RustBuilder()
154+
final.folders.source = rust_folder
155+
final.folders.build = Path(build_folder, 'final')
156+
final.folders.install = Path(args.install_folder).resolve() if args.install_folder else None
157+
final.llvm_install_folder = llvm_install_folder
158+
final.debug = args.debug
159+
final.vendor_string = args.vendor_string
160+
final.show_commands = args.show_build_commands
161+
162+
final.configure()
163+
final.build()
164+
final.show_install_info()
165+
166+
print(f"Script duration: {tc_build.utils.get_duration(script_start)}")

ci.sh

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ set -eu
99
function parse_parameters() {
1010
while (($#)); do
1111
case $1 in
12-
all | binutils | deps | kernel | llvm) action=$1 ;;
12+
all | binutils | deps | kernel | llvm | rust) action=$1 ;;
1313
*) exit 33 ;;
1414
esac
1515
shift
@@ -19,6 +19,7 @@ function parse_parameters() {
1919
function do_all() {
2020
do_deps
2121
do_llvm
22+
do_rust
2223
do_binutils
2324
do_kernel
2425
}
@@ -54,6 +55,7 @@ function do_deps() {
5455
lld \
5556
make \
5657
ninja-build \
58+
pkg-config \
5759
python3 \
5860
texinfo \
5961
xz-utils \
@@ -111,5 +113,13 @@ function do_llvm() {
111113
"${extra_args[@]}"
112114
}
113115

116+
function do_rust() {
117+
"$base"/build-rust.py \
118+
--debug \
119+
--llvm-install-folder "$install" \
120+
--install-folder "$install" \
121+
--show-build-commands
122+
}
123+
114124
parse_parameters "$@"
115125
do_"${action:=all}"

0 commit comments

Comments
 (0)