BTRFS made easier to setup during installation

[slim-salm]

Good day,
Forgive me if there is already a thread open for this idea.
I know there is a way to install Linux mint on a BTRFS filesystem, but it is a bit advanced for a newcomer.
I have been running BTRFS with zram for my swap now for nearly 2 years on my laptop and the seamless restore is really nice to have, not even to mention the COW functionality.
Although EXT4 is the default, would it not be nice to have a default pre-configured option for a BTRFS that will work out of the box without going in knee deep into the setup doing things like creating subvolumes @ and @home?

Maybe also add a additional feature to support more subvolumes for timeshift, should advanced users desire to have more subvolumes for whatever reason?

I can really see that business would easily be able to make use of linux mint systems as additional workstations with additional toolsets like versioning with BTRFS

Edit:
I have discovered Antynea/grub-btrfs, and it is awesome

[jakjasie1]

I agree, this sinplifies e.g snapshots in Timeshift. Could even enable them by default when installed with BTRFS, since they don't occupy a lot of space in BTRFS. I believe a good way to implement this is to simply add a switch in the installer, like some other programs do.

[Heavygrass]

this sinplifies e.g snapshots in Timeshift. Could even enable them by default when installed with BTRFS, since they don't occupy a lot of space in BTRFS.

Timeshift doesn't support splitting of root into subvolumes. Many people don't use Timeshift and there're other (better) alternatives, so I wouldn't enable it by default.
Snapshots CAN occupy space with time.

[slim-salm]

"Many people don't use Timeshift and there're other (better) alternatives, so I wouldn't enable it by default." Although there is definitely better options, I do not agree by not using it, especially if you are a newcomer, having a snapshot for your system and home sub-volume can at times be a life saver.

"Snapshots CAN occupy space with time." That is true, but it the snapshot manager is set up sanely and correctly, it should not be a problem.

Using a additional tool such as Antynea/grub-btrfs that make snapshots of your boot, you can intentionally break your os, and still recover from that disaster.

I think adding the latter as a default for - linux mint and lmde - btrfs setup, that might set mint on its own bar compared to other distros.

[Heavygrass]

"I do not agree by not using it" - When did I say one shouldn't use Timeshift or whatever?
I'm just saying we don't want bashing here about which snapshoting tool to include by default. You want Timeshift, other people want other snapshoting tools.
For many setups Timeshift would be useless due to split root.
"but it the snapshot manager is set up sanely and correctly, it should not be a problem." - some people may know nothing about BTRFS snapshots and will complain about it taking "too much space".
Someday BTRFS might become default FS.

Also there are even modern laptops with 256GB drive (even less in GiB), there's simply no space for long-term snapshots.

[Fantu]

This tutorial can help with the use of btrfs on mint in some cases: https://github.com/orgs/linuxmint/discussions/549

[slim-salm]

I know there are many tutorials to set it up. The point I try to make is to make it seamless like it is set up for EXT4 filesystem.

[mtwebster]

We wouldn't recommend btrfs for new users at this point - if someone thinks they need it it's easy to use the 'Something else' option to make and assign the partitions themselves.

My personal experiences have all started great - I love the snapshot capabilities, but have ended with 'where's my disk space and how do I get it back?' - just google 'btrfs lost disk space'. That's not something a typical user should ever have to worry about running into.

[Heavygrass]

I don't believe significant loss of space can happen unless one uses snapshots. And if one does use snapshots, there's clearly needs to be some experience first.
I'd just keep things like timeshift disabled by default.

[mortim]

That is why I use zram-tools for swap

But if you want to enable the hibernation you need a Swapfile/swap partition

[slim-salm]

zram-tools creates the swapfile/partition you need if I was not clear enough, and I am happy to report that I can suspend my laptop with ease

[slim-salm]

...because the special features (especially CoW) of this filesystem does not allow swap by default.

@mortim
Can you maybe explain yourself on this matter? For what I know it is something you need to setup with disk via GParted, and link the swap to your fstab, or alternatively, you install zram-tools and setup the configuration, do a reboot and your swap should show

[mortim]

...because the special features (especially CoW) of this filesystem does not allow swap by default.

@mortim Can you maybe explain yourself on this matter? For what I know it is something you need to setup with disk via GParted, and link the swap to your fstab, or alternatively, you install zram-tools and setup the configuration, do a reboot and your swap should show

@slim-salm I suggested that Mint team should set up swap memory when the user install Linux Mint on a BTRFS partition.

[slim-salm]

If it cannot be done, that is fine, if you don't want to do it that is fine as well.
But just because of issues such as "lost disk space on BTRFS" shouldn't repel you from using it. The probability is when someone does choose to make use of BTRFS, they already made up their mind that they want to try it. But again, it is a idea, I am no developer, but I do like cool technologies like ZFS an BTRFS.

[Fantu]

I have been using btrfs for about ten years now at work on servers and for a few years also on some disks in personal use.
It has both pros and cons and is not optimal for any use (I still use ext4 in most cases).
It seems stable and reliable to me, apart from the well-known problems with raids with parity (where there is a risk of data loss and is better avoid them), in fact I only use raid 1 and in some cases DUP profile (for double copy even on the same disk).
raid 1 or DUP together with data checksums and self-repair guarantee an excellent advantage for integrity, in some cases more than that of hardware raid and software raid with mdadm, with DUP on single hdd also to limit problems such as those given by possible defective sectors.
It requires some occasional maintenance operations more than ext4, the space problems I have seen are mainly due to the management of snapshots and not the fault of btrfs itself (there can easily be similar problems even using zfs and lvm snapshots) even if in part it keeps a little too much space reserved and can require the "balance" to solve.
There is also less performance (especially on hdd), and mainly because of the cow (which can be deactivated, but you would lose most of the features/advantages), so in certain cases I think it is better not to use it.
I think having it better integrated even if not predefined would be very useful.

[Fantu]

Making maintenance easier and automating some things can help, but unfortunately it cannot make up for the shortcomings of the users.

Most of the serious tests on btrfs, even intentionally causing problems or testing the edge situations (obviously only in test environments) I did at the beginning about ten years ago. Once I had the information I needed and having used btrfs on dozens of volumes for many years without particular problems and only in certain uses, it did not allow me to acquire enough knowledge on some things and in fact recently I am getting better informed and doing more tests because I intend to use btrfs more and in new fields, for example on my next workstation I will use btrfs entirely on the root (and not just a data partition and external backups).

I have read about many btrfs problem situations, but many of them are clearly caused by errors and lack of knowledge of users.
For example:

• Consider snapshots as backups: snapshots on the same volume are NOT backups, having snapshots on a btrfs volume (but it could be for something else) does not provide protection from hardware damage but only to recover previous versions of data due to user or software errors. Yet I have seen people who thought they had complete backup-level protection given by snapshots even on a single disk and single profile (some even on metadata).
  Generally speaking, there are too many people who do not do backups at all or neglect them, regardless of the fs used, and I think this is the main cause of data loss.
  I think people should be better informed about what backups are (basically copies of data on other disks and/or devices) and their importance. In my experience I have seen too many people put important data, without making copies, on external disks or even pen drives thinking that they are safer when in fact it is often worse, and I'm not just talking about my early years of experience in computer support (started 26 years ago) but these are unfortunately frequent things even in recent years.
  I think that users should be informed more about the importance of backups, what they are and more help in managing them.
• Unknowingly using defragmentation: firstly defragmentation should not be used on SSD disks, this on any filesystem, it increases wear unnecessarily, but unfortunately I have seen several cases of use on SSD (for example autodefrag in the mount options).
  It is important to know that in btrfs defragmentation, modifying the data, causes the loss of the deduplication that is automatically achieved with snapshots and reflink copies and therefore takes up more space, much more space if thanks to snapshots and reflinks the snapshots and any other copies took up very little space, do not take this into account, use it when it is not needed or use it when you do not have enough free space can easily lead to having full space on the filesystem. If it is necessary, make sure you have enough free space (it could even be 90% or more of the data space of the subvolume that is being defragmented if it has very high deduplication), in many cases it can be useful (or necessary) to delete all the snapshots or almost and also wait for them to be deleted effectively freeing up the space (on large hdds it can take a long time)
• do not consider or underestimate hardware problems: it's a problem on any filesystem, I don't know if on btrfs some users expect that thanks to the checksums it works miracles, but I've seen several times people continue to use and/or try to repair the filesystem on disks with certain hardware problems, in some cases even evidently serious ones.
  When there are certain problems on the disk, unless you are sure that they are only single and few defective sectors you must absolutely stop using it immediately, and this with any filesystem. If you have backups and there is no need for any recovery, create on a new disk and restore from backup. If instead you need to recover the data you must first clone on a good disk (of equal or greater size), I found ddrescue excellent that allows you to better manage the thing, resume the procedure make several attempts etc... only after cloning on the healthy disk then you try to repair the filesystem and/or recover data.
  On ext4 it seems to me that the users have more chances of seeing problems because in many cases of unexpected events/errors it mounts read-only, on btrfs instead thanks to the checksums on both data and metadata it manages to detect and if it has raids or duplicate profiles correct automatically and continue without the user noticing. In this regard, it could be useful to make sure to have more controls and notifications to users in case of problems (is useful also in case of any other filesystem more check and notification on issues).
  Another thing that is useful to know about btrfs is that contrary to what is used on ext4, fsck should be used rarely and with great care, once you have detected problems, and you have excluded that there are serious hardware problems, instead you should first do a scrub to detect the damages thanks to the checksums (unless they are disabled in some parts) and repair thanks to copies from raid or dup. profiles (usually there is dup. metadata even in the case of a single disk).
• poor space consideration, mainly with snapshots: when using snapshots you have to carefully consider the space they can occupy, and often you can risk underestimating the thing arriving at full space errors (this can obviously also happen in other snapshot systems, such as for example lvm and zfs).
  In the case of btrfs I have seen people complaining about having full space errors even though the data occupied by the main working subvolume was much less. Taking an extreme example, the root subvolume occupied only about a third of the actual space but the btrfs volume was full, going to look at the snapshots there are a large quantity, they even kept the first one made after the initial installation and then over time many have been made for system updates (including even a distro version upgrade), in addition to the work on the data. Obviously if you have many snapshots including one made since the beginning you will also have all the changes made since then on the data, even if you delete data it is as if they were not deleted.
  So you must always take into account the space occupied by snapshots, never underestimate it and always manage snapshots by doing any manual or automatic cleaning in order to limit possible problems.
  As for space management even without snapshots it is always good to keep a good margin, it is also useful in other fs even if you could have fewer problems working at the limit with ext4 rather than btrfs.
  Keeping a large margin is also very useful for other reasons, for example on SSD disks to maintain good performance and longer life (this I think would be a good practice to keep with any filesystem on SSD).
  Of the possible useful checks and notifications for a system I think could be for example to bring to the attention of the user if the free space goes below 10%, if there are more than 20 snapshots and/or there are snapshots older than 6 months. Maybe also make the alerts configurable and have more of them, I don't think I've seen any notification software of the kind for desktop, even if I've never looked seriously and for the servers I've made custom scripts (very limited and specific to what we need) for some parts of management and monitoring (with reports and alert by email).

There would be other things to say, but it would be too long and would take too much time. I'm also not good at explaining or in English.

---

About the disk configuration part in the installer, I think it would be a great thing to improve, but I should do more tests before recommending what would be better to add and/or improve. Recently I tried especially the server part, and I noticed for example the proxmox installer much more limited (not only with btrfs) in disk management, before doing the 2 new installations with btrfs on the root on my next workstation if I have enough time I will try to do some tests with the latest versions of different distributions to try to understand what a good disk configuration support could be (for desktop and workstation, for linux mint), and in particular of btrfs, in the installer.

[Heavygrass]

@AtomicRobotMan0101 Could you please share your BTRFS maintenance monthly cron scrub and trim commands/scripts?
Why do you need monthly scrub and trim?

[AtomicRobotMan0101]

@AtomicRobotMan0101 Could you please share your BTRFS maintenance monthly cron scrub and trim commands/scripts? Why do you need monthly scrub and trim?

# @monthly sudo fstrim --all --verbose #FStrim
# @monthly btrfs balance start /mnt/10TB_rusty # Btrfs balance every 3 months
# @monthly btrfs scrub start /mnt/10TB_rusty # Btrfs scrub on the 1st of every month

0 1 5 */3 * btrfs balance start -dusage=80 /mnt/10TB_rusty
0 1 2 * * btrfs scrub start -Bd /mnt/10TB_rusty

The drive in question is a Seagate model: ST10000NM0046 9.1 TiB .... its a noisy beast, but its ultra reliable as its a server disk. The disk is running in DUP, which halves the disk size but doubles up the data written to it. If a sector is problematic the DUP system may assist... (I hope!)

The first 3 entries are disabled as I found the disk simply didn't need so much management. Its a bit of a garbage dump as well as a store for movies. Having it rattle away for the sake of a bit of maintenance was irritating - so I pushed it back to 3 monthly.

On what each does....
Scrub

Verifies data and metadata checksums for all blocks.
Detects and corrects errors using redundancy (RAID1, DUP).
Does not defragment or rebalance space.
Think: integrity check.

Balance

Rewrites block groups to reclaim unused space and redistribute chunks.
Can convert chunk profiles (e.g., RAID0 → RAID1).
Useful after device removal or to resolve unbalanced space usage.
Think: space reorganisation.

For my own sanity, there's a second old machine with some old hard disks I use as a secondary and ternary backups. For those I simply use SyncThing and FreeFileSync. FFS is for the backups and ST ensures all my machines have exact real-time copies of Useful Things kept locally.

The use of ST on my mobile is especially useful.

Apologies for wandering off your question! :-)

[Heavygrass]

Does it make sense to do scrub on a single SSD (no RAID)?

[AtomicRobotMan0101]

oh,absolutely. I see it as a safeguard against bit-rot.
It also validates the data against the checksums.

[Heavygrass]

Also installer should NOT be trying to format existing BTRFS partition (at least by default).
BTRFS is already basically a partition manager, and can span over the whole drive (or several drives).
BTRFS doesn't even require a partition table.
Its like trying to delete the GPT and repartition it. One of course can if really wants too.

[fosslover306]

With the prevalence of SSDs and USB drives the flash-friendly-file-system would also be nice to have as an install option, since it mainly focuses on data integrity and massively reduces write cycles without causing any problems for the user to deal with in my personal experience, and since their are now laptops out their with soldered SSDs that will be very important in the future.

[Heavygrass]

without going in knee deep into the setup doing things like creating subvolumes @ and @home?

I'd give them more meaningful names, e.g. "@root_mint_version" "@home_mint_version".
One can have any number of different OS on BTRFS, this would avoid a name conflict.
GRUB can boot from almost any subvolume name.

[Fantu]

One small thing I think help to improve btrfs support is the mount options, for example the ones proposed for Debian (unfortunately too late for Trixie) and additional 2 or more compress options: https://salsa.debian.org/installer-team/partman-btrfs/-/merge_requests/3
I saw unfortunately that the Mint 22.1 installer does not support mount options in partitioning at all, not only for btrfs :(

[SpeeQz1]

@mtwebster Any updates on the statement about adding BTRFS to the Linux Mint installer as an optional choice for users?

[slim-salm]

It is already there in the advanced section, but you need to set it up yourself, if that is what you were asking. But as for a option in the normal install, I dont think it will be there anytime soon.