[Guide] Incremental backup using the official backup system

Original forum link
https://forum.qubes-os.org/t/25792
Original poster
Solène R
Created at
2024-04-12 09:56:14
Last wiki edit
2024-06-15 12:09:04
Revisions
6 revisions
Posts count
33
Likes count
60

Hi,

I got really bored about backups, so I took a different approach. I like the official backup system because it works, it's easy to use and can restore everything, however it sucks because it takes too much disk space.

Instead of fighting it, I made a simple script to replace scrypt binary in dom0 that just neuters the encryption, this allows us to pipe the backup file to a tool handling deduplication and encryption.

Here is the result of a qube backup done 3 times, with a qube restart between each:

> du -shc *
1,6G    qubes-backup-2024-04-12T112336
1,6G    qubes-backup-2024-04-12T112420
1,6G    qubes-backup-2024-04-12T112557
754M    restic-repository

The repository didn't grow after each backup 🙂

I'd be happy to receive feedback about it, I didn't try it much yet. Update: I've been using it for 2 months with success, the backup repository doesn't grow much at each new backup.

Setup

In dom0, rename /usr/bin/scrypt to /usr/bin/scrypt.bak.

Create /usr/bin/scrypt with the following content, and make it executable:

#!/bin/sh

# $1 = enc / dec
# $2 = infile
# $3 = outfile

# remove parameter -f used when restoring
for arg do
    shift
    [ "$arg" = "-f" ] && continue
    set -- "$@" "$arg"
done

# stub prompt for backup
if [ "$1" = "enc" ]
then
    printf "Please enter passphrase: " >/dev/stderr
    printf "Please confirm passphrase: " >/dev/stderr
fi

# stub prompt for restore
if [ "$1" = "dec" ]
then
    printf "Please enter passphrase: " >/dev/stderr
fi 

# handle all combinations possible
# input = a file or stdin
# output = a file or stdout
if [ "$2" = "-" ]
then
    if [ -z "$3" ]
    then
        cat
    else
        cat > "$3"
    fi
else
    if [ -z "$3" ]
    then
        cat "$2"
    else
        cat "$2" > "$3"
    fi
fi

Usage

After this change, the backup tool won't encrypt the backup anymore, you can send it to a script to handle it in borg / restic / whatyou want, I explain the method in a blog post

When making backups, just type anything in the password field, it's not used but required for the GUI to be happy.

Make sure to disable compression, this doesn't help deduplication, and the backup tool will handle compression anyway (and better than gzip).

Cons

You can't use backups made prior to this change, you would have to rename /usr/bin/scrypt.old back to /usr/bin/scrypt to make them work again.

A dom0 update or upgrade may overwrite scrypt binary.

As stated in the second post of this topic, backups could be tampered with through the qube handling the backup tool. This is quite unlikely if you use packages from a template on a qube that isn't exposed to non trusted binaries, but be sure it fits into your threat model!

Misc

I verified and dom0 should only use the scrypt command for backups and restore, so it shouldn't break anything 🙂

Performance

I made a benchmark to compare. The protocol was to use the command qvm-backup --no-compress --yes with both restic (defaults settings) and borg (compression set to zstd instead of lz4).

The default backup was done with scrypt disabled, and was using --compress, plain good old qvm-backup.

With restic and borg, I did two backups of all my qubes (except the one with the backup) in a row, then I rebooteed and did a new backup, all temporary qubes disks got merged after that, increasing the size.

I did not do the qvm-backup multiple times because the size and time would be almost identical to the first run.

Type                            size (bytes / GB)   time
----
default backup with compression 94670100 (91GB)     137 minutes
restic                          76439920 (73GB)     22 minutes
restic 2nd backup               77499468 (74GB)     15 minutes
restic 3rd backup post reboot   81685884 (78GB)     15 minutes
borg                            78847352 (76GB)     23 minutes
borg 2nd backup                 82468580 (79GB)     17 minutes
borg 3rd backup post reboot     90346032 (87GB)     19 minutes