AOSP RPi Builder VM (Embedded Android Summer School)

This repository contains the builder VM scripts for a Android AOSP builder VM, mainly based on LabVM Framework (automated using qemu and Packer).

The VM container a series of scripts and workarounds for facilitating a multi-user AOSP build environment optimized for disk usage:

• AOSP code is only downloaded once and shared among all users (using overlay fs);
• Script to create/enter specific mount namespaces for each user inside an unchanged /build mountpoint to prevent AOSP build cache invalidation;
• Automated build scripts for the raspberry-vanilla project;
• TMUX / ZSH / NVim customizations for a friendly terminal experience;
• Script to establish Wireguard tunnel to a frontend;

Requirements:

• a modern Linux system;
• basic build tools (make);
• Hashicorp's Packer;
• qemu+kvm;
• hosting with >= 16 CPUs, 64GB RAM, 300GB disk space (yep, this is the mininum requirement for building AOSP!

Preparation

Download and save a Ubuntu 22.04 Live Server install iso image.

Optionally, create config.local.mk, copy the variables from config.sample.mk and/or framework/config.default.mk and edit them to your liking.

You might also want to ensure that packer and qemu are properly installed and configured.

Building the VM

The following Makefile goals are available (the build process is usually in this order):

• base: builds a base Ubuntu 22.04 install (required for the VM image);
• main: builds the maine VM with all required scripts and config;
• cloud: builds (from main VM) the cloud VM, cleaned up and ready for cloud usage (e.g., AWS, OpenStack).
• main_edit: easily edit an already build Lab VM (uses the previous image as backing snapshot);
• main_commit: commits the edited VM back to its backing base;
• [*]_clean: removes the generated image(s);
• ssh: SSH-es into a running Packer VM;

If packer complains about the output file existing, you must either manually delete the generated VM from inside TMP_DIR, or set the DELETE=1 makefile variable (but be careful):

make DELETE=1 main

If you want to keep the install scripts at the end of the provisioning phase, set the DEBUG variable. Also check out PAUSE (it pauses packer, letting you inspect the VM inside qemu):

make PAUSE=1 DEBUG=1 main

Read [https://github.com/cs-pub-ro/labvm-framework|LabVM Framework's] documentation for more lower-level targets and options.

VM: OverlayFS Usage

Once booted up, the VM has a clean environment.

If you wish, mount a large (>= 300GB) volume at /home/_aosp_build and do your first build! During the AOSP development process, the overlay filesystem will be mounted on /build (using a separate mount namespace for each user so they do not conflict with eachother).

Base Layer (as admin)

To build the base AOSP images (main + car), simply run this script (inside tmux recommended):

whoami  # admin, right ???
sudo builder-init-all.sh

Advanced usage: to manually create a mount namespace for admin and mount the base layer at /build:

sudo builder-mkmountns.sh
# work with the base layer, NOT an upper overlay:
sudo builder-mkoverlay.sh --root
# enter the new mount namespace:
sudo builder-enter.sh

Overlay for each student user

After the base build is over, here's how to make an user-specific overlay (e.g. for student):

The easy way, autostarted with VM (replace with actual username):

STUDENT_USER=student
systemctl enable build-env-for@$STUDENT_USER
systemctl restart build-env-for@$STUDENT_USER
# this should do the required setup

Or the manual way:

sudo builder-mkmountns.sh $STUDENT_USER
# work with the base layer, NOT an upper overlay:
sudo builder-mkoverlay.sh $STUDENT_USER

As student, enter the new mount namespace and build AOSP:

# first, must enter the namespace (with tmux, recommended)
sudo builder-enter.sh tmux
# android should be already built from previous step!
ls -l /build
# e.g., further change the build
. build/envsetup.sh
lunch aosp_rpi5-bp2a-userdebug
# change something
vim device/brcm/rpi5/BoardConfig.mk
# rebuild
make bootimage systemimage vendorimage -j$(nproc --ignore=2)