====== BTRFS ======

Btrfs (B-tree file system) is a filesystem based on technology Copy-On-Write with advanced functionnality:
  * maximum capacity per filesystem 2^64 octets == 16 EB
  * Allow sub-volumes creation
  * Ability to add/remove block devices without interruption
  * Ability to balance the filesystem without interruption
  * RAID 0, RAID 1, RAID 5, RAID 6 and RAID 10
  * Snapshots and file cloning
  * Compression (gzip / LZO)
  * quotas
  * defragmentation online
  * checksum, Online data scrubbing for finding errors and automatically fixing them for files with redundant copies

Optional, dedup is partialy supported using an additionnal package **duperemove** or **dduper**

ZFS is a direct concurrent to BTRFS, with more functionality, but as it's not integrated to Linux kernel due to CDDL license which is incompatible with GPL license, it's not recommended to use it in production.

===== Usage =====

==== Create a BTRFS filesystem ====

Create a filesystem on a patition, disk or logical volume (can be used with LVM)
<cli prompt='#'>
manu-opensuse:~ # mkfs.btrfs /dev/sdb3
manu-opensuse:~ # cat /etc/fstab
...
/dev/sdb3 /backup btrfs defaults 0 0

manu-opensuse:~ # mkdir /backup
manu-opensuse:~ # mount -t btrfs /dev/sdb3  /backup
</cli>

List your filesystem:
<cli prompt='#'>
manu-opensuse:~ # btrfs filesystem show
Label: none  uuid: a3da64e9-f198-4cb2-adcd-01ec0541cba9
        Total devices 1 FS bytes used 11.58GiB
        devid    1 size 40.00GiB used 13.80GiB path /dev/sdb3
</cli>

First, let's see how much free space we have:
<cli prompt='#'>
manu-opensuse:/ # btrfs filesystem df -h /backup
Data, single: total=38.97GiB, used=35.73GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=1.00GiB, used=764.86MiB
GlobalReserve, single: total=73.78MiB, used=0.00B
</cli>

Or
<cli prompt='#'>
# btrfs filesystem usage /backup
Overall:
    Device size:                  20.02GiB
    Device allocated:             13.78GiB
    Device unallocated:            6.24GiB
    Device missing:                  0.00B
    Used:                         10.02GiB
    Free (estimated):              9.63GiB      (min: 9.63GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:              144.00MiB      (used: 0.00B)

             Data     Metadata  System
Id Path      single   single    single   Unallocated
-- --------- -------- --------- -------- -----------
 1 /dev/sda3 13.00GiB 768.00MiB 32.00MiB     6.24GiB
-- --------- -------- --------- -------- -----------
   Total     13.00GiB 768.00MiB 32.00MiB     6.24GiB
   Used       9.61GiB 421.36MiB 16.00KiB
</cli>

Increase or reduce the partition size online:
<cli prompt='#'>
# btrfs filesystem resize -42g /backup
</cli>

To prevent device from changing name, you can also use the UUID, which is a persistent name
<cli prompt='#'>
manu-opensuse:/ # lsblk -a -o UUID,NAME,FSUSED,FSTYPE
UUID                                   NAME                    FSUSED FSTYPE
                                       sda                            
F926-FC70                              ├─sda1                    8.3M vfat
a2e132cf-d69c-448c-bc59-9e833bebb95c   ├─sda2                         ext3
a3da64e9-f198-4cb2-adcd-01ec0541cba9   ├─sda3                   36.6G btrfs
4c91d4e3-89f7-4b17-9507-84b17c69d777   ├─sda4                   15.2G xfs
859cf5c6-a1ba-4711-98b2-387b8c2bd860   ├─sda5                         swap
1z8s8i-WQL7-yutq-VKYE-cbBD-p1ZB-3dCuwo └─sda6                         LVM2_member
35bfc2a9-a3c0-4eee-82a1-1f62ca52aad7     ├─libraryvg-vmlv      107.9G ext4

manu-opensuse:/ # cat /etc/fstab
UUID=a3da64e9-f198-4cb2-adcd-01ec0541cba9  /             btrfs  defaults               0  0
UUID=a3da64e9-f198-4cb2-adcd-01ec0541cba9  /var          btrfs  subvol=/@/var          0  0
UUID=07e198ed-18a3-41ed-9e48-bde82ead65fc  /backup       btrfs  defaults,compress      0  1
</cli>
==== Convert an ext4 partition to BTRFS ====

Make a backup before doing this operation (supported on ext2, 3, 4 and reiserfs)
<cli prompt='#'>
manu-opensuse:~ # btrfs-convert /dev/<device>
</cli>

===== Avanced functions =====

==== Sub-volumes ====

A subvolume is a part of filesystem with its own independent file/directory hierarchy. Subvolumes can share file extents. A snapshot is also subvolume, but with a given initial content of the original subvolume.

=== subvolumes et snapshots ===

Un subvolume est comparable à un simple répertoire (il peut contenir des fichiers et d’autres répertoires). Lorsque que l’on utilise Btrfs, il existe au moins un subvolume, le subvolume racine.
<cli prompt='#'>
# btrfs subvolume create /backup/aaa
# btrfs subvolume create /backup/bbb
</cli>

List available subvolumes on /backup:
<cli prompt='#'>
# btrfs subvolume list /backup/
</cli>

Note : A snapshot is a copy-on-write of Btrfs. It shares the same blocks as the filesystem. It doesn't preserve in case of corruption.

Create a snapshot :
<cli prompt='#'>
# btrfs subvolume snapshot /backup/aaa /backup/bbb/snapshot1
</cli>

Suppress a subvolume (and those snapshots) :
<cli prompt='#'>
# btrfs subvolume delete /bbb/bbb/snapshot1
</cli>

List properties of subvolume :
<cli prompt='#'>
# btrfs property list -ts /path/to/subvolume
</cli>

Put a subvolume in RW:
<cli prompt='#'>
# btrfs property set -ts /path/to/subvolume ro false
</cli>

Put a subvolume in RO:
<cli prompt='#'>
# btrfs property set -ts /path/to/subvolume ro true
</cli>

=== Operations on subvolume ===

Pour déplacer un subvolume, il faut créer un snapshot en read-only du subvolume que l’on souhaite déplacer/renommer puis supprimer l’original.
<cli prompt='#'>
# btrfs sub snap -r /path/to/subvolume /path/to/snapshot
# btrfs subvolume delete /path/to/subvolume
</cli>

On passera ensuite le volume en RO/RW :
<cli prompt='#'>
# btrfs property set -ts /path/to/snapshot ro false
</cli>

Envoyer un subvolume

Pour transférer un subvolume vers un autre serveur, il faut créer un snapshot en RO du subvolume en question.
<cli prompt='#'>
# btrfs sub snap -r /path/to/subvolume /path/to/snapshot-RO
</cli>

You can synchronize a BTRFS snapshot with a remote server:
<cli prompt='#'>
# btrfs send /path/to/snapshot-RO | ssh root@192.0.2.1 "btrfs receive /path/to/remote-snapshot"
</cli>

Compression

On peut monter avec l’option compress=zlib ou compress=lzo pour activer une compression à la volée des fichiers. Si le volume avant n’avait pas l’option compress on pourra tout recompresser avec btrfs filesystem defragment -czlib ou -clzo.
Maintenance

==== Quota for Subvolumes ====

The Btrfs root file system subvolumes /var/log, /var/crash and /var/cache can use all of the available disk space during normal operation, and cause a system malfunction. To help avoid this situation, SUSE Linux Enterprise Server now offers Btrfs quota support for subvolumes. If you set up the root file system by using the respective YaST proposal, it is prepared accordingly: quota groups (qgroup) for all subvolumes are already set up. To set a quota for a subvolume in the root file system, proceed as follows:

Enable quota support:
<cli prompt='#'>
# btrfs quota enable /
</cli>

Get a list of subvolumes:
<cli prompt='#'>
# btrfs subvolume list /
</cli>

Quotas can only be set for existing subvolumes.

Set a quota for one of the subvolumes that was listed in the previous step. A subvolume can either be identified by path (for example /var/tmp) or by 0/SUBVOLUME ID (for example 0/272). The following example sets a quota of 5 GB for /var/tmp.
<cli prompt='#'>
# btrfs qgroup limit 5G /var/tmp
</cli>

The size can either be specified in bytes (5000000000), kilobytes (5000000K), megabytes (5000M), or gigabytes (5G). The resulting values in bytes slightly differ, since 1024 Bytes = 1 KiB, 1024 KiB = 1 MiB, etc.

To list the existing quotas, use the following command. The column max_rfer shows the quota in bytes.
<cli prompt='#'>
# btrfs qgroup show -r /
</cli>

TipTip: Nullifying a Quota

In case you want to nullify an existing quota, set a quota size of none:
<cli prompt='#'>
# btrfs qgroup limit none /var/tmp
</cli>

To disable quota support for a partition and all its subvolumes, use btrfs quota disable:
<cli prompt='#'>
# btrfs quota disable /
</cli>





==== RAID protection ====

Defrag, relocate chunks with less than 5% of usage:
<cli prompt='#'>
manu-opensuse:~ # btrfs fi balance start -dusage=5 /
Done, had to relocate 0 out of 19 chunks
</cli>

Add a new disk and balance the data
<cli prompt='#'>
manu-opensuse:~ # btrfs device add /dev/sdc /
manu-opensuse:~ # btrfs filesystem balance /
</cli>

<cli prompt='#'>
# btrfs device replace
</cli>

Create a subvolume with compression
<cli prompt='#'>
manu-opensuse:~ # btrfs subvolume create /mnt/btrfs/home
manu-opensuse:~ # mount -o compress=lzo subvol=home /dev/sdb2 /mnt/btrfs/home
</cli>

Enabling quota and then printing out the qgroup information:
<cli prompt='#'>
manu-opensuse:~ # btrfs quota enable /btrfs/
manu-opensuse:~ # btrfs qgroup show  /btrfs/
0/5 4698025984 8192
0/257 52432896 4096
0/263 4405821440 12288
0/264 4698025984 8192
</cli>

A snapshot is a subvolume (Copy on Write CoW), you can create one in read-only (-r)
<cli prompt='#'>
manu-opensuse:~ # mkdir /snapshots
manu-opensuse:~ # btrfs subvolume snapshot -r / /snapshots/root.$(date +%Y%m%d-%H%M)
Create a readonly snapshot of '/' in '/snapshots/root.20160919-0954'
</cli>

Snapshots can also be automated with **snapper**
<cli prompt='#'>
manu-opensuse:~ # snapper -c home create-config /mnt/btrfs/home
</cli>

A config file will be created in /etc/snapper/configs/ and you can start the services
  systemctl start snapper-timeline.timer snapper-cleanup.timer
  systemctl enable snapper-timeline.timer snapper-cleanup.timer

List snapshots  
<cli prompt='#'>
manu-opensuse:~ # snapper list
Type   | #   | Pre # | Date                     | User | Cleanup | Description           | Userdata     
-------+-----+-------+--------------------------+------+---------+-----------------------+--------------
single | 0   |       |                          | root |         | current               |              
single | 1   |       | Thu Jun 21 12:41:01 2018 | root |         | first root filesystem |              
single | 2   |       | Thu Jun 21 12:53:40 2018 | root | number  | after installation    | important=yes
pre    | 3   |       | Thu Jun 21 12:59:06 2018 | root | number  | zypp(zypper)          | important=yes
post   | 4   | 3     | Thu Jun 21 13:00:21 2018 | root | number  |                       | important=yes
pre    | 172 |       | Wed Jun 27 09:50:23 2018 | root | number  | zypp(packagekitd)     | important=no 
post   | 173 | 172   | Wed Jun 27 09:50:31 2018 | root | number  |                       | important=no 
pre    | 180 |       | Wed Jun 27 11:54:39 2018 | root | number  | zypp(zypper)          | important=no 
</cli>

Delete snapshot
<cli prompt='#'>
manu-opensuse:~ # snapper delete 172
</cli>

You can reduce it using the following command:
<cli prompt='#'>
manu-opensuse:~ # snapper set-config SPACE_LIMIT=0.2 NUMBER_LIMIT=2-6 NUMBER_LIMIT_IMPORTANT=4
</cli>

===== BTRFS corruption =====

Check data integrity on block and metadata level with repair if needed (using -B in background if needed)
<cli prompt='#'>
# btrfs scrub start /backup
</cli>

<cli prompt='#'>
# btrfs scrub status /backup
UUID:             76fac721-2294-4f89-a1af-620cde7a1980
Scrub started:    Wed Apr 10 12:34:56 2023
Status:           running
Duration:         0:00:05
Time left:        0:00:05
ETA:              Wed Apr 10 12:35:01 2023
Total to scrub:   28.32GiB
Bytes scrubbed:   13.76GiB  (48.59%)
Rate:             2.75GiB/s
Error summary:    no errors found
</cli>

With some errors found:
<code>
Error summary:    csum=72
  Corrected:      2
  Uncorrectable:  72
  Unverified:     0
</code>

**btrfs check** seems to verify and optionally attempt repair of the structure of the filesystem. **btrfs scrub** verifies (and possibly repairs) the checksums of every data and metadata block

Advanced check on a partition:
<cli prompt='#'>
# btrfs check -p /dev/sda9
</cli>

Check if the filesystem is in error, and indicate a progress bar:
<cli prompt='#'>
# btrfs device stats /dev/sda3
[/dev/sda3].write_io_errs   0
[/dev/sda3].read_io_errs    0
[/dev/sda3].flush_io_errs   0
[/dev/sda3].corruption_errs 0
[/dev/sda3].generation_errs 0
</cli>

===== script btrfs_size.sh =====

<cli prompt='#'>
manu-opensuse:~ # cat ./btrfs_size.sh
</cli>
<code>
#!/bin/bash
#Author Kyle Agronick <agronick@gmail.com>
#Usage: Invoke this script to get the size of your subvolumes and snapshots
#Make sure to run "sudo btrfs quota enable /" first


LOCATION='/'
if [ $1 ]; then
LOCATION=$1
fi

OUTPUT="" 

COL1=`sudo btrfs subvolume list "$LOCATION"`

if [ $? -ne 0 ]; then
        echo "Failed to the volume data! BTRFS volume is required on the target location!"
        exit 1
fi

COL1=$(echo "$COL1" | cut -d ' ' -f 2,9) # Only taking the ID and the Snapshot name

COL2=`sudo btrfs qgroup show "$LOCATION" --raw 2>&1`
CONTINUE=false
if [[ $COL2 == *"unrecognized option"* ]]; then
    COL2=`sudo btrfs qgroup show "$LOCATION" `
fi    

COL2=$(echo "$COL2" | cut -c 2-)


function convert()
{ 
        OUT=`echo "$i" | awk '{ sum=$1 ; hum[1024^4]="TB";hum[1024^3]="GB";hum[1024^2]="MB";hum[1024]="KB"; for (x=1024^4; x>=1024; x/=1024){ if (sum>=x) { printf "%.2f%s\n",sum/x,hum[x];break } }}'`
        OUTPUT=$(printf "%-9s" $OUT) 
        echo "$OUTPUT"
}

i=0
ECL_TOTAL=0
INDEX=0
LC_ALL=C
for i in $COL2; do  
    if [[ $i == *"groupid"* ]] || [[ $i == *"----"* ]]; then
        continue;
    fi 
    if [[ ! $i =~ ^[A-Za-z-]+$ ]]; then  
      if [[ "$i" == *\/* ]]; then 
        INDEX=0
        ROWID=$(echo "$i" | cut -c 2-)   
        OUTPUT+="
$ROWID  " 
      else
        ((INDEX++))   
        if [ -z `echo $i | tr -d "[:alpha:]"` ]; then
            echo $i" letters\n"
            OUTPUT="$OUTPUT"$(printf "%-9s" $i)
        else
            if [ $INDEX -eq 2 ]; then
                ECL_TOTAL=$(($i + $ECL_TOTAL)) 
            fi
            OUTPUT="$OUTPUT$(convert $i)"
        fi
       fi
    fi
done


# Determine terminal width
if hash tput 2>/dev/null; then
        COLCOUNT=`tput cols`
elif hash stty 2>/dev/null; then
        COLCOUNT=`stty size | cut -d' ' -f2`
else
        COLCOUNT=80 # Default
fi

declare -a COLUMNWIDHTS=(-$(($COLCOUNT-30)) 20 6)

function printRow
{
        DATA=("$@")

        # The offset is calculated to help aligning the next column properly,
        # if the preveious one was too long
        local offset=0
        for ((i=0;i < $#;i++))
        {
                local modifier=""
                local width=${COLUMNWIDHTS[$i]}
                if [ $width -lt 0 ]; then
                        width=$((0-$width)) # Gettings abs value
                        modifier="-." # Left-padded and truncating if too long
                fi
                local pattern="%$modifier*s"
                local column # The current column with padding
                printf -v column $pattern $(($width + $offset)) "${DATA[$i]}"
                printf "$column"
                offset=$(($offset + $width - ${#column}))
        }
        printf "\n"
}

function printHorizontalLine
{
        printf '%*s\n' $COLCOUNT '' | tr ' ' '='
}

# Header start
printHorizontalLine
printRow "Snapshot / Subvolume" "Total Exclusive Data" "ID"
printHorizontalLine
# Header end

IFS=$'\n'

# Table body start
for item in  $COL1; do
    ID=$(echo $item | cut -d' ' -f1)
    name=$(echo $item | cut -d' ' -f2)
    for item2 in $OUTPUT; do
        ID2=$(echo $item2 | grep -o '^[0-9.]\+' )
        if [ "$ID" = "$ID2" ]; then
                        eval ROWDATA=($(echo $name ${item2[@]} | awk -F' ' '{print $1, $3, $2}'))
                        printRow "${ROWDATA[@]}"
                        break;
        fi
    done
done
# Table body end

if [ $ECL_TOTAL -gt "1" ]; then
    printHorizontalLine
    i=$ECL_TOTAL
    printf "%-64s" " "  
    printf "Exclusive Total: $(convert $i) \n"
fi
</code>
<cli prompt='#'>
manu-opensuse:~ # ./btrfs_size.sh /
==========================================================================================================================================================================================================================================================================================
Snapshot / Subvolume                                                                                                                                                                                                                                        Total Exclusive Data    ID
===================================================================================
@                                                                   16.00KB   257
@/var                                                              498.04MB   258
@/usr/local                                                         31.34MB   259
@/tmp                                                                1.98GB   260
@/srv                                                              150.35MB   261
@/root                                                              10.76MB   262
@/opt                                                              871.97MB   263
@/boot/grub2/x86_64-efi                                              3.38MB   264
@/boot/grub2/i386-pc                                                16.00KB   265
@/.snapshots                                                       488.00KB   266
@/.snapshots/1/snapshot                                              7.18GB   267
...
@/.snapshots/173/snapshot                                            7.18GB   459
@/.snapshots/180/snapshot                                            7.18GB   466
@/.snapshots/181/snapshot                                            7.18GB   467
</cli>