Skip to content

Commit cf46e6f

Browse files
pirat89Monstrofil
authored andcommitted
overlay lib: Redesign creation of the source overlay composition
The in-place upgrade itself requires to do some changes on the system to be able to perform the in-place upgrade itself - or even to be able to evaluate if the system is possible to upgrade. However, we do not want to (and must not) change the original system until we pass beyond the point of not return. For that purposes we have to create a layer above the real host file system, where we can safely perform all operations without affecting the system setup, rpm database, etc. Currently overlay (OVL) technology showed it is capable to handle our requirements good enough - with some limitations. However, the original design we used to compose overlay layer above the host system had number of problems: * buggy calculation of the required free space for the upgrade RPM transaction * consumed too much space to handle partitions formatted with XFS without ftype attributes (even tens GBs) * bad UX as people had to manually adjust size of OVL disk images * .. and couple of additional issues derivated from problems listed above The new solution prepares a disk image (represented by sparse-file) and an overlay image for each mountpoint configured in /etc/fstab, excluding those with FS types noted in the `OVERLAY_DO_NOT_MOUNT` set. Such prepared OVL images are then composed together to reflect the real host filesystem. In the end everything is cleaned. The composition could look like this: orig mountpoint -> disk img -> overlay img -> new mountpoint ------------------------------------------------------------- / -> root_ -> root_/ovl -> root_/ovl/ /boot -> root_boot -> root_boot/ovl -> root_/ovl/boot /var -> root_var -> root_var/ovl -> root_/ovl/var /var/lib -> root_var_lib -> root_var_lib/ovl -> root_/ovl/var/lib ... The new solution can be now problematic for system with too many partitions and loop devices, as each disk image is loop mounted (that's same as before, but number of disk images will be bigger in total number). For such systems we keep for now the possibility of the fallback to an old solution, which has number of issues mentioned above, but it's a trade of. To fallback to the old solution, set envar: LEAPP_OVL_LEGACY=1 Disk images created for OVL are formatted with XFS by default. In case of problems, it's possible to switch to Ext4 FS using: LEAPP_OVL_IMG_FS_EXT4=1 XFS is better optimized for our use cases (faster initialisation consuming less space). However we have reported several issues related to overlay images, that happened so far only on XFS filesystems. We are not sure about root causes, but having the possibility to switch to Ext4 seems to be wise. In case of issues, we can simple ask users to try the switch and see if the problem is fixed or still present. Some additional technical details about other changes * Added simple/naive checks whether the system has enough space on the partition hosting /var/lib/leapp (usually /var). Consuming the all space on the partition could lead to unwanted behaviour - in the worst case if we speak about /var partition it could mean problems also for other applications running on the system * In case the container is larger than the expected min default or the calculation of the required free space is lower than the minimal protected size, return the protected size constant (200 MiB). * Work just with mountpoints (paths) in the _prepare_required_mounts() instead of with list of MountPoint named tuple. I think about the removal of the named tuple, but let's keep it for now. * Make apparent size of created disk images 5% smaller to protect failed upgrades during the transaction execution due to really small amount of free space. * Cleanup the scratch directory at the end to free the consumed space. Disks are kept after the run of leapp when LEAPP_DEVEL_KEEP_DISK_IMGS=1 (cherry picked from commit d074926)
1 parent d6dc991 commit cf46e6f

File tree

3 files changed

+448
-6
lines changed

3 files changed

+448
-6
lines changed

repos/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -739,11 +739,13 @@ def perform():
739739

740740
indata = _InputData()
741741
prod_cert_path = _get_product_certificate_path()
742+
reserve_space = overlaygen.get_recommended_leapp_free_space(_get_target_userspace())
742743
with overlaygen.create_source_overlay(
743744
mounts_dir=constants.MOUNTS_DIR,
744745
scratch_dir=constants.SCRATCH_DIR,
745746
storage_info=indata.storage_info,
746-
xfs_info=indata.xfs_info) as overlay:
747+
xfs_info=indata.xfs_info,
748+
scratch_reserve=reserve_space) as overlay:
747749
with overlay.nspawn() as context:
748750
target_repoids = _gather_target_repositories(context, indata, prod_cert_path)
749751
_create_target_userspace(context, indata.packages, indata.files, target_repoids)

repos/system_upgrade/common/libraries/dnfplugin.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -360,14 +360,17 @@ def perform_transaction_install(target_userspace_info, storage_info, used_repos,
360360

361361

362362
@contextlib.contextmanager
363-
def _prepare_perform(used_repos, target_userspace_info, xfs_info, storage_info):
363+
def _prepare_perform(used_repos, target_userspace_info, xfs_info, storage_info, target_iso=None):
364+
reserve_space = overlaygen.get_recommended_leapp_free_space(target_userspace_info.path)
364365
with _prepare_transaction(used_repos=used_repos,
365366
target_userspace_info=target_userspace_info
366367
) as (context, target_repoids, userspace_info):
367368
with overlaygen.create_source_overlay(mounts_dir=userspace_info.mounts, scratch_dir=userspace_info.scratch,
368369
xfs_info=xfs_info, storage_info=storage_info,
369-
mount_target=os.path.join(context.base_dir, 'installroot')) as overlay:
370-
yield context, overlay, target_repoids
370+
mount_target=os.path.join(context.base_dir, 'installroot'),
371+
scratch_reserve=reserve_space) as overlay:
372+
with mounting.mount_upgrade_iso_to_root_dir(target_userspace_info.path, target_iso):
373+
yield context, overlay, target_repoids
371374

372375

373376
def perform_transaction_check(target_userspace_info, used_repos, tasks, xfs_info, storage_info, plugin_info):

0 commit comments

Comments
 (0)