BTRFS

A subvolume is a part of filesystem with its own independent file/directory hierarchy. Subvolumes can share file extents.

A subvolume looks and behaves like a normal directory, but can be mounted like a separate filesystem using options subvol or subvolid. Once a subvolume is mounted, the parent directory or mountpoint is not visible and accessible, similar to a bind mount.

A freshly created filesystem is also a subvolume, called top-level, and internally it has an id 5. This is also the subvolume that will be mounted by default and hence cannot be be removed., unless the default subvolume has been changed using the subcommand set-default.

Any subvolume cannot be deleted, if it is a parent volume with at least one child in it i.e if there are nested subvolume in it.

Since subvolumes can be created anywhere in the filesystem, there are two common layouts depending on where it is created.

When subvolumes are created at top-level subvolume, this is known as "flat" layout. These subvolumes can be mounted to their respective mountpoints using subvol=path in /etc/fstab.

prabu@homepc2 ~> tree -L 1 /mnt/btrfs/
/mnt/btrfs/
├── @
├── @arescue
├── @avanilla
├── @debian
├── @docs
├── @home
├── _apk_snap
└── _btrbk_snap

Creating a subvolume inside another subvolume is known as "nested" layout. Nested subvolumes are useful for user specific folders like ".cache" etc. Be aware that such nested subvolumes are excluded by snapshot based backup management tools as explained below.

Flat layout however causes issues with Snapper as it prefers and uses "nested" layout. Workaround is provided in the linked page.

A snapshot is also subvolume, but with a given initial content of the original subvolume. By default, snapshots are created read-write. File modifications in a snapshot do not affect the files in the original subvolume.

Flat layout of subvolumes is mandatory for snapshot based backup tools like btrbk, Timeshift etc and also helpful during OS recovery.

When a snapshot is taken for a subvolume, it will be without the parent or child subvolumes. When taking backups using snapshot aware tools like btrbk, to exclude a directory from snapshot, it must be made a subvolume. Empty folder names will appear for subvolumes in snapshots.

Snapshots cannot be created on a different device/partition. The error message "ERROR: cannot snapshot 'home': Invalid cross-device link" will appear when attempting this. The btrfs send/receive option allows to transfer a read only subvolume to a btrfs filesystem on a different drive or network.

To temporarily convert a readwrite subvolume to readonly for an ad-hoc transfer, use the below command:

doas btrfs property set -ts /path/to/subvolume ro true

Note: Once the transfer is complete, flip it back to read-write by changing true to false.

growpart is a Linux command-line tool used to extend a partition in a partition table to fill available space. This command is provided by cloud utils package in debian and cloud-utils-growpart package in Alpine Linux.

In this example, we’re extending partition 2 in disk /dev/sdb. Replace the partition number and name with appropriate values.

$sudo growpart /dev/sdb 2

growpart cannot be used if a partition to be extended is not the end partition. fdisk is used in below example to resize root partition in btrfs as MSDOS partition table was used. Use gdisk if gpt partition table is used.

prabu@onepc-lm:~$ sudo fdisk -l /dev/sda
Disklabel type: dos
Disk identifier: 0xf3f3f3f3
Device  Boot    Start       End   Sectors  Size Id Type
/dev/sda1  *        2048  78125055  78123008 37.3G 83 Linux
/dev/sda3       105551872 312580095 207028224 98.7G 83 Linux

Here's the partition table after resizing and redoing the partition table.

Device  Boot Start      End   Sectors   Size Id Type
/dev/sda1  *    2048 312581807 312579760 149.1G 83 Linux

Resize btrfs filesystem as shown below.

$sudo btrfs device usage /data
$sudo btrfs filesystem resize +256M /data
$sudo btrfs filesystem resize max /data
$sudo btrfs filesystem resize -1G /data

In the above examples, the Btrfs filesystem removed 1 GB or added 256MB of disk space from/to the filesystem pool. You can use the slack space (Device slack) to grow/expand the Btrfs filesystem later.

prabu@homepc-lm:~$ sudo btrfs device usage /mnt/backup/
/dev/sdb1, ID: 1
   Device size:         232.88GiB
   Device slack:        71.87GiB
   Data,single:         115.00GiB
   Metadata,DUP:            6.00GiB
   System,DUP:          16.00MiB
   Unallocated:         40.00GiB

In homepc-lm, the data from /home folder was moved to btrfs filesystem on /dev/sda3 before enabling compression on it through /etc/fstab. So to compress all existing data on it, the below command was used. It took almost 10 minutes to compress 13GB of data.

Subvolume before compression and after compression

doas df -h /home

Filesystem  	Size  Used Avail Use% Mounted on
/dev/sda3    	30G   14G   17G  46% /home

Command to compress the filesystems:

doas btrfs filesystem defragment -r -c /home

To specifically use a specific algorithm(i.e zstd) with verbose option, the above command can be run as follows:

doas btrfs filesystem defragment -r -v -czstd /home

doas df -h /home

Filesystem  	Size  Used Avail Use% Mounted on
/dev/sda3    	30G  8.6G   21G  30% /home

Commands to show details about btrfs filesystems:

btrfs filesystem df /home
doas btrfs filesystem show

To accurately view the compression information, use the compsize utility as shown before compression:

doas compsize /mnt/btrfs/@myhome

Processed 96643 files, 51047 regular extents (51328 refs), 50701 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       95%       17G          18G          18G
none       100%       17G          17G          17G
zstd        30%      339M         1.0G         1.0G
prealloc   100%      1.2M         1.2M         9.0M

After compression:

doas compsize /mnt/btrfs/@myhome

Processed 96643 files, 100534 regular extents (100610 refs), 53052 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       88%       16G          18G          18G
none       100%       13G          13G          13G
zstd        58%      2.9G         5.0G         5.0G
prealloc   100%       19M          19M         9.0M

The following command removes all subvolumes in a directory:

#ls /mnt/snapshots | xargs btrfs subvolume delete

In case there are files or directories that are not btrfs subvolumes present, an error will be printed, but they are not removed.

A directory must be converted to a subvolume, if it needs its own snapshot based backup policy management. Converting a directoy to a nested subvolume is explained below:

• Rename the directory to a temp dir to make the name available

  doas mv dir_name dir_name_temp
  

• Create a subvolume using the command

  doas btrfs subvolume create dir_name
  

• Copy the contents to the newly created subvolume

  cp -a --reflink=always dir_name_temp/. dir_name/
  

• Verify that 100% copy has occurred. There should be no output for the below command, if there is no difference.

  diff -r dir_name_temp dir_name
  

• After verifying that everything is copied, delete the temp folder

  rm -rf dir_name_temp
  

Warning: Snapshots must be created as read only to use btrfs send.

Day1: First time backup using btrfs send is called initial bootstrapping. It corresponds to a full backup. This task will take some time, depending on the size of the snapshot .

$sudo btrfs subvolume snapshot -r /home /.snapshots/home-day1
$sudo btrfs send /.snapshots/home-day1 | sudo btrfs receive /run/media/user/mydisk/bk

Day2: Subsequent incremental sends will take a shorter time, as only differences are sent.

$sudo btrfs send -p /.snapshot/home-day1 /.snapshot/home-day2 | sudo btrfs receive /run/media/user/mydisk/bk

Day3: The steps get repeated again and the day after day3

$sudo btrfs subvolume snapshot -r /home /.snapshots/home-day3
$sudo btrfs send -p /.snapshot/home-day2 /.snapshot/home-day3 | sudo btrfs receive /run/media/user/mydisk/bk

Note: Don’t delete the last snapshot on source and destination. If lost, one needs to do initial bootstrapping again.

Command to create a ro snapshot with current timestamp

$sudo btrfs subv snaps -r @ /.snapshot/onepc_@_$(date -d "today" +"%Y%m%d%H%M")

Create a Btrfs Snapshot before updating system. Create a file backup.sh with below information and run sudo backup.sh before you update

#!/bin/sh
mount /dev/sda1 /mnt
cd /mnt
 #Checks if backup exists and deletes it
[ -d @-BACKUP ] && sudo btrfs sub del @-BACKUP
[ -d @home-BACKUP ] && sudo btrfs sub del @home-BACKUP
btrfs sub snap @ @-BACKUP
btrfs sub snap @home @home-BACKUP
cd /
umount /mnt

When using linux mint, i was using Timeshift for this purpose. Packages like apt-btrfs-snapshot or apk-snap allows package manager integration with snapshot management using Snapper.

Recover individual files:

$cp /run/media/user/mydisk/bk/myproject-day1/filename.odt $HOME/Documents/myproject

A snapshot can be restored from external hdd or within system.

The following example shows getting a snapshot from an external hdd.

$ sudo btrfs send /run/media/user/mydisk/bk/myproject-day1 | sudo btrfs receive $HOME/Documents/
$ mv Documents/myproject-day1 Documents/myproject
$ btrfs property set Documents/myproject ro false

Restore the previous system state in case of any failure during the system upgrade and you want to do a rollback.

To do so let’s mount the btrfs filesystem to a separate location, fol example /mnt .

mount /dev/sda1 /mnt

List the ‘/mnt’ directory and you will see subvolumes in its output.

ls -l /mnt/
total 0
drwxr-xr-x 1 root root 178 May 24 02:40 @
drwxr-xr-x 1 root root 166 May 24 00:54 @apt-snapshot-2016-05-24_02:18:31
drwxr-xr-x 1 root root 8 May 24 01:07 @home

Where ‘@apt-snapshot-2016-05-24_02:18:31’ is a snapshot of our working root filesystem (@) before the apt operation. In order to make the system boot from that working snapshot instead of from the current subvolume, we rename @ to something else and then @apt-snapshot-2012-11-22_11:50:38 to @ . After renaming reboot your system and you will be glad to see that your system is at the previous state of before system upgrade.

You can confirm by repeating the ‘apt-get upgrade’ command which will show the same packages to be upgrade.

The above command creates a new read write subvolume based off the read only snapshot taken before the upgrade in the place of the old root subvolume. No change in /etc/fstab required as the root subvolume is mounted via "subvol=@", i.e. path and not numeric id.

Let's assume that the / is on /dev/sda1 and the OS is installed with / on @ subvolume and /home is on @home subvolume. I created a swap file on the btrfs filesystem earlier using the below steps.

Since kernel 6.1+, below command is the canonical way to do this.

sudo btrfs filesystem mkswapfile --size 4G /path/to/swapfile

chattr +C disables Copy-on-Write (CoW) before creating the file.

root@onepc-lm:swapoff -a
root@onepc-lm:sudo mount /dev/sda1 /mnt
root@onepc-lm:sudo btrfs sub create /mnt/var/@swap
root@onepc-lm:sudo umount /mnt
root@onepc-lm:sudo mkdir /var/swap
root@onepc-lm:sudo mount -o subvol=@swap /dev/sda1 /var/swap
root@onepc-lm:/var/swap# cd /var/swap/
root@onepc-lm:/var/swap# sudo touch swapfile
root@onepc-lm:/var/swap# chmod 600 swapfile
root@onepc-lm:/var/swap# chattr +C swapfile
root@onepc-lm:/var/swap# ls -al swapfile
-rw------- 1 root root 0 Jan  3 18:58 swapfile
root@onepc-lm:/var/swap# fallocate swapfile -l3g
root@onepc-lm:/var/swap# ls -al swapfile
-rw------- 1 root root 3221225472 Jan  3 18:58 swapfile
root@onepc-lm:/var/swap# mkswap swapfile
Setting up swapspace version 1, size = 3 GiB (3221221376 bytes)
no label, UUID=21c2c9d6-662d-4ede-b76c-5e0fa7fb3825
root@onepc-lm:/var/swap# swapon /var/swap/swapfile
root@onepc-lm:/var/swap# free -m
Swap:       3071        0       3071
root@onepc-lm:/var/swap# lsblk /dev/sda1 -f
NAME FSTYPE LABEL UUID                              FSAVAIL FSUSE% MOUNTPOINT
sda1 btrfs      1670c8ed-101b-4a58-9091-ea9ed16b6d3d   21.4G    41% /run/timeshift
You also need to update /etc/fstab to mount the swap file on boot. Add these two lines:
UUID=XXXXXXXXXXXXXXX /swap btrfs subvol=@swap 0 0
/swap/swapfile none swap sw 0 0

The above UUID is the one of your /dev/sda1.

root@onepc-lm:/var/swap# cat /etc/fstab
UUID=1670c8ed-101b-4a58-9091-ea9ed16b6d3d /var/swap  btrfs  subvol=@swap  0  0
/var/swap/swapfile none swap sw 0 0

For a quick and accurate calculation of disk usage by a subvolume use the below command. This command does not require quotas to be enabled.

prabu@homepc2 ~> doas compsize /mnt/btrfs/@
doas (prabu@homepc2) password:
Processed 56582 files, 39484 regular extents (39720 refs), 27545 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       60%      1.8G         3.0G         3.0G
none       100%      1.2G         1.2G         1.2G
zlib        37%       18M          49M          40M
zstd        34%      633M         1.7G         1.8G
prealloc   100%      2.5M         2.5M         2.1M

If quotas are enabled on the btrfs volume, then a python script btrfs-subvolumes.py available at https://gist.github.com/stecman/3fd04a36111874f67c484c74e15ef311/6690edbd6a88380a1712024bb4115969b2545509 can be used to view detailed usage for all Subvolumes.

To enable quota on the volume, use the command:

$ doas btrfs quota enable /

The quota option can be kept disabled if not used. I have not enabled it.

I run utility fstrim on weekly basis and scrub on a monthly basis using cron to monitor the health of the filesystem. The results are logged at /var/log/btrfs_maintenance.log

Here's the current version of the script:

prabu@homepc2 ~> cat /usr/local/bin/btrfs_maint
#!/bin/ash
# BTRFS Maintenance Script for ash/BusyBox
LOG_FILE="/var/log/btrfs_maintenance.log"
MOUNT_POINT="/"

# Simple timestamping function for ash
log_msg() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') $1" >> "$LOG_FILE"
}

case "$1" in
    trim)
        log_msg "[TRIM] Starting weekly fstrim on $MOUNT_POINT"
        /sbin/fstrim -v "$MOUNT_POINT" >> "$LOG_FILE" 2>&1
        ;;
    scrub)
        log_msg "[SCRUB] Starting monthly scrub on $MOUNT_POINT"
        # -B makes scrub stay in foreground so we capture the result
        /sbin/btrfs scrub start -B "$MOUNT_POINT" >> "$LOG_FILE" 2>&1
        log_msg "[SCRUB] Finished."
        ;;
    *)
        echo "Usage: $0 {trim|scrub}"
        exit 1
        ;;
esac

The script is called as follows:

prabu@homepc2 ~> doas crontab -l |grep btrfs
doas (prabu@homepc2) password:
@weekly                 ID=weekly-fstrim    /usr/local/bin/btrfs_maint trim
@monthly                ID=monthly-scrub    /usr/local/bin/btrfs_maint scrub

btrfs-scan service from btrfs-progs-openrc package of Alpine Linux runs at boot runlevel.

To manually check filesystem status, the following command can be used:

$ doas btrfs device stats /

I use Refind, a boot manager with btrfs driver inbuilt to maintain a multi-boot environment on supported systems. On my desktop pc, i maintain a Alpine Linux based rescue OS, with necessary tools and ESR version of Firefox with swaywm and a non-shared home folder. This allows me to boot into a graphical rescue enivornment to troubleshoot, read about and solve, if i face any issue.

Refind page has necessary configuration information to boot the rescue OS. This arrangment also allows me to quickly spin up a bare metal Alpine Linux vanilla installation and test various scenarios or changes before updating the Alpine Wiki pages and delete the subvolume once the testing is over.

Using Refind+BTRFS subvolumes allows me to use my low spec desktops to perform reasonably well as virtualization options like qemu is either impossible or not performant enough.