Merge pull request #528 from fosslinux/mirror-docs

Document mirrors a bit better

Author: stikonas

.github/workflows/bwrap.yml

@@ -98,7 +98,7 @@ jobs:
       - name: Copy distfiles
         run: ./download-distfiles.sh file:///${PWD}/mirror
       - name: Run bootstrap
-        run: ./rootfs.py --bwrap --external-sources --build-kernels --cores 2 --internal-ci pass2
+        run: ./rootfs.py --bwrap --external-sources --build-kernels --cores 2 --internal-ci pass2 --mirror file://${PWD}/mirror
       - name: Archive created packages
         if: failure() # archive failed builds progress
         uses: actions/upload-artifact@v4

README.rst

@@ -17,6 +17,10 @@ How do I use this?
 
 Quick start:
 
+Choose a mirror from https://github.com/fosslinux/live-bootstrap/wiki/Mirrors,
+or create a private/public mirror yourself (see further below). You should
+provide the mirror as ``--mirror`` to ``rootfs.py``.
+
 See ``./rootfs.py --help`` and follow the instructions given there.
 This uses a variety of userland tools to prepare the bootstrap.
 
@@ -25,6 +29,8 @@ preparations! This is a currently unsolved problem.*)
 
 Without using Python:
 
+0. Choose a mirror as detailed above. (You will input this later, instead of
+   passing it to ``rootfs.py```).
 1. ``git clone https://github.com/fosslinux/live-bootstrap``
 2. ``git submodule update --init --recursive``
 3. Consider whether you are going to run this in a chroot, in QEMU, or on bare
@@ -68,6 +74,31 @@ Without using Python:
    c. **Bare metal:** Follow the same steps as QEMU, but the disks need to be
       two different *physical* disks, and boot from the first disk.
 
+Mirrors
+-------
+
+It has been decided that repackaging distfiles for live-bootstrap is generally
+permissible, particularly from git repositories. We do this primarily because
+
+a. currently live-bootstrap only supports tarballs/raw files as input, not git
+   repositories
+b. to reduce load on servers
+
+You may choose to use an existing mirror from 
+https://github.com/fosslinux/live-bootstrap/wiki/Mirrors, however you may be
+(to some varied extent) trusting the operator of the mirror.
+
+Alternatively, you can create your own local mirror - one such implementation
+is in ``./mirror.sh``. You can invoke it with
+``./mirror.sh path/to/mirror/dir path/to/mirror/state``.
+You would then pass ``--mirror path/to/mirror/dir`` to rootfs.py.
+(If not using rootfs.py, you need to copy files around manually into distfiles.)
+
+Most helpfully to the project, you could create your own public mirror, by
+running ``./mirror.sh`` or writing your own script that does something similar
+on a timer (systemd timer or cron job, for example), where the mirror directory
+is publicly accessible on the Internet (ideally, via HTTP and HTTPS).
+
 Background
 ----------
 

rootfs.py

@@ -178,6 +178,10 @@ def check_types():
     else:
         args.swap = 0
 
+    # Validate mirrors
+    if not args.mirrors:
+        raise ValueError("At least one mirror must be provided.")
+
     # Set constant umask
     os.umask(0o022)
 

steps/configurator

@@ -40,6 +40,19 @@ this correctly, otherwise you risk overwriting an existing disk on your
 system. LIVE-BOOTSTRAP TAKES NO LIABILITY FOR ANY DATA LOSS RESULTING FROM
 ITS USE.
 
+o /general/mirrors MIRRORS "" _ Mirrors
+live-bootstrap needs a mirror to download repackaged distfiles from,
+as long as you do not already have all the distfiles on an attached disk.
+Please see the README for your options of selecting one.
+This should be a URI of the form http(s)://[...], or if it is locally accessible
+on a filesystem (perhaps because you are running this in a sandboxing tool,
+instead of a VM or hardware), file://[...].
+Multiple mirrors can be provided, space separated.
+ENSURE YOU UPDATE MIRRORS_COUNT.
+
+o /general/mirrors_count MIRRORS_LEN int _ Number of mirrors configured
+The number of mirrors you put into Mirrors. Should be an integer >= 0.
+
 m /sysinfo _ _ _ System information
 Details about your specific system and the environment live-bootstrap is
 running in.