Home server backups with restic
The home server holds documents, photos and the configs for everything else. Until April, its backup was me remembering to plug in a USB disk.
- Project
- nightly-restic
- Stack
- Bash, restic, systemd timers
- Time spent
- About 3 evenings
- Status
- runs at 03:15
didWhat I did
Two restic repositories: one on a second internal disk, one offsite. A short Bash script backs up to both, applies a retention policy and exits non-zero if anything fails. A systemd timer runs it every night at 03:15.
#!/usr/bin/env bash
set -euo pipefail
export RESTIC_PASSWORD_FILE=/etc/restic/password # chmod 600, root only
SOURCES=(/srv/docs /srv/photos /etc /home/me)
for repo in /mnt/backup/restic "$OFFSITE_REPO"; do
restic -r "$repo" backup "${SOURCES[@]}" \
--exclude-file=/etc/restic/excludes --retry-lock 30m
restic -r "$repo" forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12
done
date -Is > /var/lib/backup/last-success
Once a week, a second unit runs restic check --read-data-subset 5%, which reads back a random slice of the actual data instead of only checking the index. Over twenty weeks that covers the whole repository.
The last line writes a timestamp. It looked unimportant when I added it; it became the hook for the status page two months later.
$ restic snapshots --compact
ID Time Host
3c9e41d0 2026-04-21 03:15:06 den
b17f0a92 2026-04-22 03:15:04 den
e40d6c3b 2026-04-23 03:15:09 den
91a2fe57 2026-04-24 03:15:05 den
-- 25 Apr: locked by prune, skipped --
5d08b7e1 2026-04-26 03:15:07 den
6 snapshots
brokeWhat broke
Prune and backup collided. I first ran forget --prune every night. The first prune on the offsite repository took just over four hours, still held the lock when the next night's backup started, and the backup failed with "repository is already locked". Pruning now happens only on Sunday afternoons, and the backup waits up to 30 minutes for a lock instead of giving up.
A database copied mid-write. One service keeps a Postgres database in a Docker volume. Copying those files while the database is running gives you a backup that may not start. The script now runs pg_dump to a file first, backs up the dump, and excludes the live volume.
The test restore. I restored my home folder to /tmp/restore-test to see if it worked. It did, mostly. File permissions and timestamps were right, but ~/.ssh was missing. The exclude file had a line .* meant for cache folders, and it matched every dotfile in the tree.
# before: too broad
.*
# after: name the caches
/home/me/.cache
/home/me/.local/share/Trash
**/node_modules
Without that restore I would have found out the day I needed the keys.
nextWhat's next
- An alert when
last-successis more than 48 hours old. A backup that stops running is worse than one that fails loudly. - A restore drill every quarter, on the calendar: pick a random folder, restore it, open a few files.
- Write down, on paper, where the repository password is kept. If the server dies, the password file goes with it.