FreeBSD Network Backup Thing from Scrap

From Run Your Own
Jump to navigation Jump to search

This document shows how to setup a dedicated backup machine for LAN use, made from bit and pieces most people don't care about these days: a Raspberry Pi 1 (it seemed such a great idea at the time... ahem...), a 32MB SD Card, and a 2GB USB key. For the software, we will use FreeBSD and BorgBackup.

Prepare the OS

Why waste a 8GB or 16GB fancy SD Card when you probably have some old SD and USB sticks lying around? Also reading performances on the RPi1 is much better on USB than with the crappy SD reader. Finally, FreeBSD on the RPi1 works great, and very light on resources which is a plus given the limitation of the RPi CPU and RAM.

What needs to be done in a nutshell is to have the mandatory MS-DOS /boot partition on the 32MB SD card and have the rest of the filesystem on the USB stick.

  • Get the FreeBSD RPi1 image (update to latest RPI-B image)
wget https://download.freebsd.org/ftp/releases/arm/armv6/ISO-IMAGES/11.2/FreeBSD-11.2-RELEASE-arm-armv6-RPI-B.img.xz
  • Plug USB stick and copy the FreeBSD image to it (adjust /dev/sdX)
xzcat FreeBSD-11.2-RELEASE-arm-armv6-RPI-B.img.xz | sudo dd bs=1M of=/dev/sdX status=progress
  • Plug the 32MB sdcard and create a W95 FAT32 partition with fdisk or something (Hex code b) then format it:
sudo mkfs.vfat /dev/sdY1
  • Mount both the USB key's first partition and the SD card partition and copy the boot content from the USB key to the SD card:
sudo mount /dev/sdX1 /media/usb-key
sudo mount /dev/sdY1 /media/sd-card
sudo cp /media/usb-key/* /media/sd-card/
umount /dev/sdX1
umount /dev/sdY1

Boot FreeBSD and basic system configuration

Booting and ssh'ing

  • Insert the SD card and USB key on the RPi1
  • Optionally: connect the composite out to something for easier troubleshooting
  • Power the RPi1, the booting process will automagically find the USB key and the FreeBSD system partition (actually this is a little known feature and it's super nice).
  • Optionally: if you can't figure out the IP of the RPi1, you can sweep your LAN with:
nmap -sP 192.168.1.0/24
  • ssh in the RPi1 (default user is freebsd:freebsd):
ssh freebsd@192.168.1.XXX

Basic system config

Note: The first time FreeBSD starts, it will expand the filesystem to the full available space of the USB key.

  • Change default password for freebsd user:
passwd
  • Become root with su (password is root)
  • Change default password for root user:
passwd
  • update the FreeBSD repos catalogue (will take some time to process):
pkg update
  • Install $YOUR_FAV_TEXT_EDITOR
pkg install vim-tiny
echo "set nocompatible" > /root/.vimrc
  • Enable NTP for setting up time at every startup (the RPi1 has no RTC to keep actual time), edit /etc/rc.conf and add the following:
ntpd_enable="YES"
ntpd_sync_on_start="YES"
  • While you're here, feel free to rename the RPi:
hostname="thevault"
  • Configure your time zone:
tzsetup
  • reboot to celebrate your escape from Groundhog Day
reboot

Prepare the USB hard-drive for backup

Note: I'm using a 4TB USB disk here, do whatever you want.

  • ssh back in the RPi1, and become root
ssh freebsd@192.168.1.XXX
su
  • Plug the USB drive
  • If all goes well, it will show up as da1 (da0 is the USB key)
dmesg | grep da1
  • Create a partition
gpart create -s GPT da1
gpart add -t freebsd-ufs -a 1M da1
gpart show da1
  • Format the new partition
 newfs -U /dev/da1p1
  • Edit /etc/fstab and add an entry for the disk's mount point:
/dev/da1p1      /sendhelp       ufs     rw      2       2
  • Create and mount the mount point
mkdir /sendhelp
mount /sendhelp
chown freebsd:freebsd /sendhelp

BorgBackup configuration

On the server side

  • ssh in the RPi1, and become root
ssh freebsd@192.168.1.XXX
su
  • Install BorgBackup
pkg install py36-borgbackup

On each client

ssh-copy-id freebsd@192.168.1.XXX
  • Make a directory for your machine's backup on the RPi1
ssh freebsd@192.168.1.XXX mkdir /sendhelp/${HOST}

Borg Usage

Introduction

Borg is deduplicating backup program, which means that each file is split into chunks and only chunks that have never been seen before are added to the repository. Borg uses repositories to manage such information, and inside a repository several archives can co-exist, all sharing the same deduplicating database. The nice thing about borg is that it does not assume a particular workflow, it's really up to the user to make archives that are following an incremental model, or snapshots of specific disks, one time folder copies, and of course mix and match of all these.

Note: In theory it means that the RPi1 could be used as one big central repository for different machines to maximise the deduplicatings mechanism. However in practice, each machine needs to maintain a cache of the repos index, so every time a machine creates a new archive, the other machine will need to update their local copy, which will add some time to do to the backup process. Most importantly, because the borg process locks the repos when being used by a machine, it means that no concurrent backup can be made on the same repos. For these reasons each machine were given their own directory, for their own repos, in the client setup above.

Examples

Below some basic commands to get started, read the docs though, this is only scratching the surface. Also see next section for some scripts to make life easier.

Repository

  • Initialize backup repository for current machine (this needs to be done only once)
borg init --encryption=repokey-blake2 freebsd@192.168.1.XXX:/sendhelp/${HOST}
  • Delete a repository for current machine (and all the archives it contains!)
borg delete freebsd@192.168.1.XXX:/sendhelp/${HOST}

Archives

  • Create an archive of /something/to/backup with a timestamp as automatic name and no compression
borg create --compression none freebsd@192.168.1.XXX:/sendhelp/${HOST}::{now} /something/to/backup
  • Create a new archive of the same /something/to/backup with default compression and a top lel name
borg create freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel /something/to/backup
  • Create a new archive of the same /something/to/backup and /something/else with progress
borg create --progress freebsd@192.168.1.XXX:/sendhelp/${HOST}::yolo-{now} /something/to/backup /something/else
  • List all the archives in the repository
borg list freebsd@192.168.1.XXX:/sendhelp/${HOST}
  • List the content of the top lel archive
borg list freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel
  • Delete the top lel archive
borg delete freebsd@192.168.1.XXX:/sendhelp/${HOST}::top-lel

Scripts

Some shell scripts and aliases to automate things further

Aliases

alias borg-list='borg list freebsd@192.168.1.XXX:/sendhelp/${HOST}'
alias borg-space='ssh freebsd@192.168.1.XXX df -h | grep -e "sendhelp\|File" --color=no'

PRO Tips

Outgoing email

Useful for various alerts and whatnot. We will just need to send emails to an existing external mail server, no need to run a full MTA on the RPI1. The following instructions make the assumption that you already have a working email server that you can use (from your ISP, a relay, etc).

  • Make sure FreeBSD's default MTA is not used, in /etc/rc.conf
sendmail_enable="NO"
  • Install ssmtp
pkg install ssmtp
  • Edit /etc/mail/mailer.conf so that:
sendmail        /usr/local/sbin/ssmtp
send-mail       /usr/local/sbin/ssmtp
mailq           /usr/local/sbin/ssmtp
newaliases      /usr/local/sbin/ssmtp
hoststat        /usr/bin/true
purgestat       /usr/bin/true
  • Copy sample config files
cp /usr/local/etc/ssmtp/revaliases.sample /usr/local/etc/ssmtp/revaliases
cp /usr/local/etc/ssmtp/ssmtp.conf.sample /usr/local/etc/ssmtp/ssmtp.conf
  • Edit /usr/local/etc/ssmtp/ssmtp.conf so that:
root=username@example.com
mailhub=mail.example.com:port
rewriteDomain=example.com
hostname=thevault.local
UseTLS=YES
UseSTARTTLS=YES
AuthUser=username
AuthPass=password
  • Note: you may need to adjust the settings above depending on your server config
  • Change root's default Full name: to "root from the Vault", or whatever, with:
chpass
  • Test that you can send emails
echo "Subject: this is a test from the vault" | /usr/sbin/sendmail -v some@where.nice

Monitor the health of backup disk

We do this by getting weekly SMART reports from the drive (Self-Monitoring, Analysis and Reporting Technology).

  • Install smartmontools
pkg install smartmontools
echo 'smartd_enable="YES"' >> /etc/rc.conf
  • Copy sample config file
cp /usr/local/etc/smartd.conf.sample /usr/local/etc/smartd.conf

Monitor disk space on the backup disk

BorgBackup will not be happy (and as a consequence you too) if space gets tight on the backup disk. This daily check will let you know by email if more than 90% of the disk is used (requires the email output recipe above to work).

  • ssh as the regular user
ssh freebsd@192.168.1.XXX
  • If not done already make a small dir for your scripts and binaries
mkdir bin
  • tweak the following script and put it in ~/bin/disk-space-alert.sh
#!/bin/sh                                                                                           
CURRENT=$(df /sendhelp | grep /sendhelp | awk '{print $5}' | sed 's/%//g')
THRESHOLD=90

if [ "${CURRENT}" -gt "${THRESHOLD}" ]
then
  /usr/sbin/sendmail some@where.nice <<EOF
Subject: The Vault is running out of space
From: no-reply@vault.local

/sendhelp is full at ${CURRENT}%.

THIS IS FINE.
EOF
fi
  • Turn that into a daily cron job with crontab -e and add the following line:
0 13 * * * ~/bin/disk-space-alert.sh

Swap file for the rainy days

The RPi1 does not have much RAM. That's enough for BorgBackup, but can be tight if it's used simultaneously by different clients. To be on the safe side and avoid the kernel to start killing processes, we can create a small 512MB swap file for emergencies.

  • as root
dd if=/dev/zero of=/sendhelp/swap bs=1m count=512
chmod 0600 /sendhelp/swap
  • Edit /etc/fstab so that:
md99	none	swap	sw,file=/sendhelp/swap,late	0	0
  • If you dont' want to reboot and start swap away:
swapon -aL
Retrieved from "https://things.bleu255.com/runyourown/index.php?title=FreeBSD_Network_Backup_Thing_from_Scrap&oldid=147"