This is an old revision of the document!
Table of Contents
BTRFS
Btrfs (B-tree file system) is a filesystem based on technology Copy-On-Write with advanced functionnality:
- maximum capacity per filesystem 2^64 octets == 16 EB
- Allow sub-volumes creation
- Ability to add/remove block devices without interruption
- Ability to balance the filesystem without interruption
- RAID 0, RAID 1, RAID 5, RAID 6 and RAID 10
- Snapshots and file cloning
- Compression (gzip / LZO)
- quotas
- defragmentation online
- checksum, Online data scrubbing for finding errors and automatically fixing them for files with redundant copies
Optional, dedup is partialy supported using an additionnal package duperemove or dduper
ZFS is a direct concurrent to BTRFS, with more functionality, but as it's not integrated to Linux kernel due to CDDL license which is incompatible with GPL license, it's not recommended to use it in production.
Usage
Create a BTRFS filesystem
Create a filesystem on a patition, disk or logical volume (can be used with LVM)
manu-opensuse:~ # mkfs.btrfs /dev/sdb3 manu-opensuse:~ # cat /etc/fstab ... /dev/sdb3 /backup btrfs defaults 0 0 manu-opensuse:~ # mkdir /backup manu-opensuse:~ # mount -t btrfs /dev/sdb3 /backup
List your filesystem:
manu-opensuse:~ # btrfs filesystem show Label: none uuid: a3da64e9-f198-4cb2-adcd-01ec0541cba9 Total devices 1 FS bytes used 11.58GiB devid 1 size 40.00GiB used 13.80GiB path /dev/sdb3
First, let's see how much free space we have:
manu-opensuse:/ # btrfs fi df -h /backup Data, single: total=38.97GiB, used=35.73GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=1.00GiB, used=764.86MiB GlobalReserve, single: total=73.78MiB, used=0.00B
Or
# btrfs filesystem usage /backup Overall: Device size: 20.02GiB Device allocated: 13.78GiB Device unallocated: 6.24GiB Device missing: 0.00B Used: 10.02GiB Free (estimated): 9.63GiB (min: 9.63GiB) Data ratio: 1.00 Metadata ratio: 1.00 Global reserve: 144.00MiB (used: 0.00B) Data Metadata System Id Path single single single Unallocated -- --------- -------- --------- -------- ----------- 1 /dev/sda3 13.00GiB 768.00MiB 32.00MiB 6.24GiB -- --------- -------- --------- -------- ----------- Total 13.00GiB 768.00MiB 32.00MiB 6.24GiB Used 9.61GiB 421.36MiB 16.00KiB
Increase or reduce the partition size online:
# btrfs filesystem resize -42g /backup
To prevent device from changing name, you can also use the UUID, which is a persistent name
manu-opensuse:/ # lsblk -a -o UUID,NAME,FSUSED,FSTYPE UUID NAME FSUSED FSTYPE sda F926-FC70 ├─sda1 8.3M vfat a2e132cf-d69c-448c-bc59-9e833bebb95c ├─sda2 ext3 a3da64e9-f198-4cb2-adcd-01ec0541cba9 ├─sda3 36.6G btrfs 4c91d4e3-89f7-4b17-9507-84b17c69d777 ├─sda4 15.2G xfs 859cf5c6-a1ba-4711-98b2-387b8c2bd860 ├─sda5 swap 1z8s8i-WQL7-yutq-VKYE-cbBD-p1ZB-3dCuwo └─sda6 LVM2_member 35bfc2a9-a3c0-4eee-82a1-1f62ca52aad7 ├─libraryvg-vmlv 107.9G ext4 manu-opensuse:/ # cat /etc/fstab UUID=a3da64e9-f198-4cb2-adcd-01ec0541cba9 / btrfs defaults 0 0 UUID=a3da64e9-f198-4cb2-adcd-01ec0541cba9 /var btrfs subvol=/@/var 0 0 UUID=07e198ed-18a3-41ed-9e48-bde82ead65fc /backup btrfs defaults,compress 0 1
Convert an ext4 partition to BTRFS
Make a backup before doing this operation (supported on ext2, 3, 4 and reiserfs)
manu-opensuse:~ # btrfs-convert /dev/<device>
Avanced functions
Sub-volumes
A subvolume is a part of filesystem with its own independent file/directory hierarchy. Subvolumes can share file extents. A snapshot is also subvolume, but with a given initial content of the original subvolume.
subvolumes et snapshots
Un subvolume est comparable à un simple répertoire (il peut contenir des fichiers et d’autres répertoires). Lorsque que l’on utilise Btrfs, il existe au moins un subvolume, le subvolume racine.
# btrfs subvolume create /backup/aaa # btrfs subvolume create /backup/bbb
Liste des subvolumes disponibles :
# btrfs subvolume list /backup/
Un snapshot est un intantané figé de toutes les données contenues dans un subvolume. Si par exemple vous disposez deux fichiers (“foo” et “bar”) dans un subvolume, un snapshot avant la suppression d’un de ces deux vous permettra de le récupérer dans ce snapshot.
Note : Un snapshot n’est pas une sauvegarde, il s’appuie sur le mécanisme de copy-on-write de Btrfs. Il partage donc les mêmes blocs de données. Ainsi, si les données sont endommagées, elles le seront aussi bien sur le snapshot que sur le subvolume. Cette fonctionnalité est utile pour conserver une ou plusieurs copies locales qui peuvent enuite être utilisées pour effectuer un rollback ou une sauvegarde à partir de l’état figé d’un subvolume.
Créer un snapshot :
# btrfs subvolume snapshot /backup/aaa /backup/bbb/snapshot1
Supprimer un subvolume (et snapshot) :
# btrfs subvolume delete /bbb/bbb/snapshot1
Lister les propriétés d’un subvolume :
# btrfs property list -ts /path/to/subvolume
Passer un subvolume en RW :
# btrfs property set -ts /path/to/subvolume ro false
Passer un subvolume en RO :
# btrfs property set -ts /path/to/subvolume ro true
Manipulations d’un subvolume
Pour déplacer un subvolume, il faut créer un snapshot en read-only du subvolume que l’on souhaite déplacer/renommer puis supprimer l’original.
# btrfs sub snap -r /path/to/subvolume /path/to/snapshot # btrfs subvolume delete /path/to/subvolume
On passera ensuite le volume en RO/RW :
# btrfs property set -ts /path/to/snapshot ro false
Envoyer un subvolume
Pour transférer un subvolume vers un autre serveur, il faut créer un snapshot en RO du subvolume en question.
# btrfs sub snap -r /path/to/subvolume /path/to/snapshot-RO
On peut ensuite envoyer le volume via SSH :
# btrfs send /path/to/snapshot-RO | ssh root@192.0.2.1 “btrfs receive /path/to/remote-snapshot”
Compression
On peut monter avec l’option compress=zlib ou compress=lzo pour activer une compression à la volée des fichiers. Si le volume avant n’avait pas l’option compress on pourra tout recompresser avec btrfs filesystem defragment -czlib ou -clzo. Maintenance
Vérifier l’intégrité d’un subvolume avec l’opération scrub qui lance une lecture de l’ensemble des données et métadonnées du système de fichiers et utilise les sommes de contrôle pour identifier et réparer les données corrompues éventuelles :
# btrfs scrub start /backup/ # btrfs scrub status /backup/
Vérification plus poussée sur une partition non montée :
# btrfs check -p /dev/sda9
Voir si la partition a présenté des erreurs :
# btrfs dev stats /backup
Snapshoting
Compression
RAID protection
Defrag, relocate chunks with less than 5% of usage:
manu-opensuse:~ # btrfs fi balance start -dusage=5 / Done, had to relocate 0 out of 19 chunks
Add a new disk and balance the data
manu-opensuse:~ # btrfs device add /dev/sdc / manu-opensuse:~ # btrfs filesystem balance /
btrfs device replace
Create a subvolume with compression
manu-opensuse:~ # btrfs subvolume create /mnt/btrfs/home manu-opensuse:~ # mount -o compress=lzo subvol=home /dev/sdb2 /mnt/btrfs/home
Enabling quota and then printing out the qgroup information:
manu-opensuse:~ # btrfs quota enable /btrfs/ manu-opensuse:~ # btrfs qgroup show /btrfs/ 0/5 4698025984 8192 0/257 52432896 4096 0/263 4405821440 12288 0/264 4698025984 8192
A snapshot is a subvolume (Copy on Write CoW), you can create one in read-only (-r)
manu-opensuse:~ # mkdir /snapshots manu-opensuse:~ # btrfs subvolume snapshot -r / /snapshots/root.$(date +%Y%m%d-%H%M) Create a readonly snapshot of '/' in '/snapshots/root.20160919-0954'
Snapshots can also be automated with snapper
manu-opensuse:~ # snapper -c home create-config /mnt/btrfs/home
A config file will be created in /etc/snapper/configs/ and you can start the services
systemctl start snapper-timeline.timer snapper-cleanup.timer systemctl enable snapper-timeline.timer snapper-cleanup.timer
List snapshots
manu-opensuse:~ # snapper list Type | # | Pre # | Date | User | Cleanup | Description | Userdata -------+-----+-------+--------------------------+------+---------+-----------------------+-------------- single | 0 | | | root | | current | single | 1 | | Thu Jun 21 12:41:01 2018 | root | | first root filesystem | single | 2 | | Thu Jun 21 12:53:40 2018 | root | number | after installation | important=yes pre | 3 | | Thu Jun 21 12:59:06 2018 | root | number | zypp(zypper) | important=yes post | 4 | 3 | Thu Jun 21 13:00:21 2018 | root | number | | important=yes pre | 172 | | Wed Jun 27 09:50:23 2018 | root | number | zypp(packagekitd) | important=no post | 173 | 172 | Wed Jun 27 09:50:31 2018 | root | number | | important=no pre | 180 | | Wed Jun 27 11:54:39 2018 | root | number | zypp(zypper) | important=no
manu-opensuse:~ # cat ./btrfs_size.sh
#!/bin/bash
#Author Kyle Agronick <agronick@gmail.com>
#Usage: Invoke this script to get the size of your subvolumes and snapshots
#Make sure to run "sudo btrfs quota enable /" first
LOCATION='/'
if [ $1 ]; then
LOCATION=$1
fi
OUTPUT=""
COL1=`sudo btrfs subvolume list "$LOCATION"`
if [ $? -ne 0 ]; then
echo "Failed to the volume data! BTRFS volume is required on the target location!"
exit 1
fi
COL1=$(echo "$COL1" | cut -d ' ' -f 2,9) # Only taking the ID and the Snapshot name
COL2=`sudo btrfs qgroup show "$LOCATION" --raw 2>&1`
CONTINUE=false
if [[ $COL2 == *"unrecognized option"* ]]; then
COL2=`sudo btrfs qgroup show "$LOCATION" `
fi
COL2=$(echo "$COL2" | cut -c 2-)
function convert()
{
OUT=`echo "$i" | awk '{ sum=$1 ; hum[1024^4]="TB";hum[1024^3]="GB";hum[1024^2]="MB";hum[1024]="KB"; for (x=1024^4; x>=1024; x/=1024){ if (sum>=x) { printf "%.2f%s\n",sum/x,hum[x];break } }}'`
OUTPUT=$(printf "%-9s" $OUT)
echo "$OUTPUT"
}
i=0
ECL_TOTAL=0
INDEX=0
LC_ALL=C
for i in $COL2; do
if [[ $i == *"groupid"* ]] || [[ $i == *"----"* ]]; then
continue;
fi
if [[ ! $i =~ ^[A-Za-z-]+$ ]]; then
if [[ "$i" == *\/* ]]; then
INDEX=0
ROWID=$(echo "$i" | cut -c 2-)
OUTPUT+="
$ROWID "
else
((INDEX++))
if [ -z `echo $i | tr -d "[:alpha:]"` ]; then
echo $i" letters\n"
OUTPUT="$OUTPUT"$(printf "%-9s" $i)
else
if [ $INDEX -eq 2 ]; then
ECL_TOTAL=$(($i + $ECL_TOTAL))
fi
OUTPUT="$OUTPUT$(convert $i)"
fi
fi
fi
done
# Determine terminal width
if hash tput 2>/dev/null; then
COLCOUNT=`tput cols`
elif hash stty 2>/dev/null; then
COLCOUNT=`stty size | cut -d' ' -f2`
else
COLCOUNT=80 # Default
fi
declare -a COLUMNWIDHTS=(-$(($COLCOUNT-30)) 20 6)
function printRow
{
DATA=("$@")
# The offset is calculated to help aligning the next column properly,
# if the preveious one was too long
local offset=0
for ((i=0;i < $#;i++))
{
local modifier=""
local width=${COLUMNWIDHTS[$i]}
if [ $width -lt 0 ]; then
width=$((0-$width)) # Gettings abs value
modifier="-." # Left-padded and truncating if too long
fi
local pattern="%$modifier*s"
local column # The current column with padding
printf -v column $pattern $(($width + $offset)) "${DATA[$i]}"
printf "$column"
offset=$(($offset + $width - ${#column}))
}
printf "\n"
}
function printHorizontalLine
{
printf '%*s\n' $COLCOUNT '' | tr ' ' '='
}
# Header start
printHorizontalLine
printRow "Snapshot / Subvolume" "Total Exclusive Data" "ID"
printHorizontalLine
# Header end
IFS=$'\n'
# Table body start
for item in $COL1; do
ID=$(echo $item | cut -d' ' -f1)
name=$(echo $item | cut -d' ' -f2)
for item2 in $OUTPUT; do
ID2=$(echo $item2 | grep -o '^[0-9.]\+' )
if [ "$ID" = "$ID2" ]; then
eval ROWDATA=($(echo $name ${item2[@]} | awk -F' ' '{print $1, $3, $2}'))
printRow "${ROWDATA[@]}"
break;
fi
done
done
# Table body end
if [ $ECL_TOTAL -gt "1" ]; then
printHorizontalLine
i=$ECL_TOTAL
printf "%-64s" " "
printf "Exclusive Total: $(convert $i) \n"
fi
manu-opensuse:~ # ./btrfs_size.sh / ========================================================================================================================================================================================================================================================================================== Snapshot / Subvolume Total Exclusive Data ID =================================================================================== @ 16.00KB 257 @/var 498.04MB 258 @/usr/local 31.34MB 259 @/tmp 1.98GB 260 @/srv 150.35MB 261 @/root 10.76MB 262 @/opt 871.97MB 263 @/boot/grub2/x86_64-efi 3.38MB 264 @/boot/grub2/i386-pc 16.00KB 265 @/.snapshots 488.00KB 266 @/.snapshots/1/snapshot 7.18GB 267 ... @/.snapshots/173/snapshot 7.18GB 459 @/.snapshots/180/snapshot 7.18GB 466 @/.snapshots/181/snapshot 7.18GB 467
