FreeBSD Network Backup Thing from Scrap

This document shows how to setup a dedicated backup machine for LAN use, made from bit and pieces most people don't care about these days: a Raspberry Pi 1 (it seemed such a great idea at the time... ahem...), a 32MB SD Card, and a 2GB USB key. For the software, we will use FreeBSD and BorgBackup.

Prepare the OS
Why waste a 8GB or 16GB fancy SD Card when you probably have some old SD and USB sticks lying around? Also reading performances on the RPi1 is much better on USB than with the crappy SD reader. Finally, FreeBSD on the RPi1 works great, and very light on resources which is a plus given the limitation of the RPi CPU and RAM.

What needs to be done in a nutshell is to have the mandatory MS-DOS  partition on the 32MB SD card and have the rest of the filesystem on the USB stick.

wget https://download.freebsd.org/ftp/releases/arm/armv6/ISO-IMAGES/11.2/FreeBSD-11.2-RELEASE-arm-armv6-RPI-B.img.xz xzcat FreeBSD-11.2-RELEASE-arm-armv6-RPI-B.img.xz | sudo dd bs=1M of=/dev/sdX status=progress sudo mkfs.vfat /dev/sdY1 sudo mount /dev/sdX1 /media/usb-key sudo mount /dev/sdY1 /media/sd-card sudo cp /media/usb-key/* /media/sd-card/ umount /dev/sdX1 umount /dev/sdY1
 * Get the FreeBSD RPi1 image (update to latest RPI-B image)
 * Plug USB stick and copy the FreeBSD image to it (adjust )
 * Plug the 32MB sdcard and create a W95 FAT32 partition with  or something (Hex code b) then format it:
 * Mount both the USB key's first partition and the SD card partition and copy the boot content from the USB key to the SD card:

Booting and ssh'ing
nmap -sP 192.168.1.0/24 ssh freebsd@192.168.1.XXX
 * Insert the SD card and USB key on the RPi1
 * Optionally: connect the composite out to something for easier troubleshooting
 * Power the RPi1, the booting process will automagically find the USB key and the FreeBSD system partition (actually this is a little known feature and it's super nice).
 * Optionally: if you can't figure out the IP of the RPi1, you can sweep your LAN with:
 * ssh in the RPi1 (default user is freebsd:freebsd):

Basic system config
Note: The first time FreeBSD starts, it will expand the filesystem to the full available space of the USB key.

passwd passwd pkg update pkg install vim-tiny echo "set nocompatible" > /root/.vimrc ntpd_enable="YES" ntpd_sync_on_start="YES" hostname="thevault" tzsetup reboot
 * Change default password for  user:
 * Become root with  (password is root)
 * Change default password for  user:
 * update the FreeBSD repos catalogue (will take some time to process):
 * Install
 * Enable NTP for setting up time at every startup (the RPi1 has no RTC to keep actual time), edit  and add the following:
 * While you're here, feel free to rename the RPi:
 * Configure your time zone:
 * reboot to celebrate your escape from Groundhog Day

Prepare the USB hard-drive for backup
Note: I'm using a 4TB USB disk here, do whatever you want.

ssh freebsd@192.168.1.XXX su dmesg | grep da1 gpart create -s GPT da1 gpart add -t freebsd-ufs -a 1M da1 gpart show da1 newfs -U /dev/da1p1 /dev/da1p1     /sendhelp       ufs     rw      2       2 mkdir /sendhelp mount /sendhelp chown freebsd:freebsd /sendhelp
 * ssh back in the RPi1, and become root
 * Plug the USB drive
 * If all goes well, it will show up as da1 (da0 is the USB key)
 * Create a partition
 * Format the new partition
 * Edit /etc/fstab and add an entry for the disk's mount point:
 * Create and mount the mount point

On the server side
ssh freebsd@192.168.1.XXX su pkg install py36-borgbackup
 * ssh in the RPi1, and become root
 * Install BorgBackup

On each client
ssh-copy-id freebsd@192.168.1.XXX ssh freebsd@192.168.1.XXX mkdir /sendhelp/${HOST}
 * Install BorgBackup (check https://borgbackup.readthedocs.io/en/stable/installation.html#distribution-package)
 * Install your ssh key(s) to the RPi1's main user account
 * Make a directory for your machine's backup on the RPi1

Introduction
Borg is deduplicating backup program, which means that each file is split into chunks and only chunks that have never been seen before are added to the repository. Borg uses repositories to manage such information, and inside a repository several archives can co-exist, all sharing the same deduplicating database. The nice thing about borg is that it does not assume a particular workflow, it's really up to the user to make archives that are following an incremental model, or snapshots of specific disks, one time folder copies, and of course mix and match of all these.

Note: In theory it means that the RPi1 could be used as one big central repository for different machines to maximise the deduplicatings mechanism. However in practice, each machine needs to maintain a cache of the repos index, so every time a machine creates a new archive, the other machine will need to update their local copy, which will add some time to do to the backup process. Most importantly, because the borg process locks the repos when being used by a machine, it means that no concurrent backup can be made on the same repos. For these reasons each machine were given their own directory, for their own repos, in the client setup above.

Examples
Below some basic commands to get started, read the docs though, this is only scratching the surface. Also see next section for some scripts to make life easier.

Repository
borg init --encryption=repokey-blake2 freebsd@192.168.1.XXX:/sendhelp/${HOST} borg delete freebsd@192.168.1.XXX:/sendhelp/${HOST}
 * Initialize backup repository for current machine (this needs to be done only once)
 * Delete a repository for current machine (and all the archives it contains!)

Archives
borg create --compression none freebsd@192.168.1.XXX:/sendhelp/${HOST}::{now} /something/to/backup borg create freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel /something/to/backup borg create --progress freebsd@192.168.1.XXX:/sendhelp/${HOST}::yolo-{now} /something/to/backup /something/else borg list freebsd@192.168.1.XXX:/sendhelp/${HOST} borg list freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel borg delete freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel
 * Create an archive of  with a timestamp as automatic name and no compression
 * Create a new archive of the same  with default compression and a top lel name
 * Create a new archive of the same  and   with progress
 * List all the archives in the repository
 * List the content of the top lel archive
 * Delete the top lel archive

Scripts
Some shell scripts and aliases to automate things further

Aliases
alias borg-list='borg list freebsd@192.168.1.XXX:/sendhelp/${HOST}' alias borg-space='ssh freebsd@192.168.1.XXX df -h | grep -e "sendhelp\|File" --color=no'

Outgoing email
Useful for various alerts and whatnot. We will just need to send emails to an existing external mail server, no need to run a full MTA on the RPI1. The following instructions make the assumption that you already have a working email server that you can use (from your ISP, a relay, etc).

sendmail_enable="NO" pkg install ssmtp sendmail       /usr/local/sbin/ssmtp send-mail      /usr/local/sbin/ssmtp mailq          /usr/local/sbin/ssmtp newaliases     /usr/local/sbin/ssmtp hoststat       /usr/bin/true purgestat      /usr/bin/true cp /usr/local/etc/ssmtp/revaliases.sample /usr/local/etc/ssmtp/revaliases cp /usr/local/etc/ssmtp/ssmtp.conf.sample /usr/local/etc/ssmtp/ssmtp.conf root=username@example.com mailhub=mail.example.com:port rewriteDomain=example.com hostname=thevault.local UseTLS=YES UseSTARTTLS=YES AuthUser=username AuthPass=password chpass echo "Subject: this is a test from the vault" | /usr/sbin/sendmail -v some@where.nice
 * Make sure FreeBSD's default MTA is not used, in
 * Install ssmtp
 * Edit  so that:
 * Copy sample config files
 * Edit  so that:
 * Note: you may need to adjust the settings above depending on your server config
 * Change root's default  to "root from the Vault", or whatever, with:
 * Test that you can send emails

Monitor the health of backup disk
We do this by getting weekly SMART reports from the drive (Self-Monitoring, Analysis and Reporting Technology).

pkg install smartmontools echo 'smartd_enable="YES"' >> /etc/rc.conf cp /usr/local/etc/smartd.conf.sample /usr/local/etc/smartd.conf
 * Install smartmontools
 * Copy sample config file

Monitor disk space on the backup disk
BorgBackup will not be happy (and as a consequence you too) if space gets tight on the backup disk. This daily check will let you know by email if more than 90% of the disk is used (requires the email output recipe above to work).

ssh freebsd@192.168.1.XXX mkdir bin CURRENT=$(df /sendhelp | grep /sendhelp | awk '{print $5}' | sed 's/%//g') THRESHOLD=90 if [ "${CURRENT}" -gt "${THRESHOLD}" ] then /usr/sbin/sendmail some@where.nice <<EOF Subject: The Vault is running out of space From: no-reply@vault.local /sendhelp is full at ${CURRENT}%. THIS IS FINE. EOF fi 0 13 * * * ~/bin/disk-space-alert.sh
 * as the regular user
 * If not done already make a small dir for your scripts and binaries
 * tweak the following script and put it in
 * 1) !/bin/sh
 * Turn that into a daily cron job with  and add the following line:

Swap file for the rainy days
The RPi1 does not have much RAM. That's enough for BorgBackup, but can be tight if it's used simultaneously by different clients. To be on the safe side and avoid the kernel to start killing processes, we can create a small 512MB swap file for emergencies.

dd if=/dev/zero of=/sendhelp/swap bs=1m count=512 chmod 0600 /sendhelp/swap md99	none	swap	sw,file=/sendhelp/swap,late	0	0 swapon -aL
 * as root
 * Edit  so that:
 * If you dont' want to reboot and start swap away: