Category: Administration

Backup Strategy Overhaul

Johannes Neubauer Leave a comment

In this post I describe how and why I radically overhauled my backup strategy over the past few years – and which alternatives I consciously ruled out. It is written as a journey (what it was), which hopefully gives you helpful insight, whether my decisions could be yours or not (what is ok for me 🤓). I added takeaway sections, for more general insights.

TLDR – Too long don’t read… the final setup:

  • external USB 3.2 Gen 2×2 RAID enclosure (20 Gb/s) with two WD Red NVMe SSDs in Hardware RAID 1, formatted as EXT4.
  • initial backup (~ 1.3 TB) taken directly on the Mac via ExtFS (Paragon).
  • continuous service on a Raspberry Pi 4 B (8 GB RAM, Ubuntu):
    • enclosure attached over USB 3.0 (5 Gb/s).
    • SuperSpeed 3.1 USB C → USB 3.0 (Typ A) cable since Pi has no USB-C
    • Mac → Pi is capped by Gigabit Ethernet (1 Gb/s).
  • BorgBackup:
    • via local mount for initial backup, and
    • via SSH for continuous service
    • → incremental, chunk‑based deduplication before encryption, transfers only deltas.
Read More

Find Missing Files in a Backup 2nd Iteration

jonny Leave a comment

I already wrote a post about this topic. But as I do at work, I work in an agile manner at home. So here is an update to the post Find Missing Files in a Backup. The script there has been designed to be copied from the clipboard into the terminal. This time, I present you a script, which you may copy to a file, make it executable and reuse it easily. Further on, it fixes some minor issues with the original version (e.g. handling files with spaces and backup folders which are named differently, than the original folder).

For the more general idea of this script, please have a look at the former blog post (see link above). So here it is:

#!/bin/bash

src=$1
tgt=$2

(cd ${src} && ls) | while read file; do
  found=$(find $tgt -name "$file" | wc -l)
  if [ $found -eq 0 ]; then
    echo $file
  fi
done

Copy this into a file e.g. named backup-check.sh and give it exec rights:

$ chmod ugo+x backup-check.sh

Afterwards you can use it like this:

$ ./backup-check.sh original/ backup/

Exciting 🤓.

Find Missing Files in a Backup

jonny Leave a comment

Long time no write… I had many ideas for blog posts, but no time to write them. Hopefully, this will change soon. Here just a small update… As you know (if you read my blog), I have a “special” way of storing and “backupping” my files. All my documents are stored in a folder named YYYY/ (e.g. 2021) and have the format YYYYMMDD-<some-name>.<some-ending>. I further categorize my files by using macOS tags.

Every year I prepare a folder tax-YYYY/, where I copy all the files with relevance to my tax declaration. But sometimes I am not completely sure whether all the files in there are in my main YYYY/ folders, too. This is, because I copy files from my e-mail (like invoices) and some files, which relate to year YYYY are from YYYY+1, e.g. the proof of social security. So, after finishing the tax declaration I double check, whether every file that belongs to a year made its way into the corresponding folder with a dedicated unix command (here exemplarily for 2019). For sure this can be used to check the completeness of backup folders, too. A recursive variant may even be used to search complete backups for missing duplicates/files. So here is the command.

ls 2019* | while read file; do; \
found=$(find ../2019 -name $file | wc -l); \
if [ $found -eq 0 ]; then; echo $file; fi; \
done

Be aware, that this does only search for a file with the same name. It does not check whether it is not the same file (e.g. due to file changes). This can be done by using a hashing algorithm

Exciting 🤓.

Repair a Damaged Package System after Ubuntu Dist-Upgrade

jonny Leave a comment

Happy new year.

My blog runs on a VM at Hetzner with an Ubuntu LTS system. That means 5 years of support… I was running trusty from 2014, so there should be support until 2019. But not every open source software has given you this promise, just the Ubuntanians. So, support for Owncloud run out last year and I thought that the days between years are a good time to switch to a new version.

Hence, I did two dist-upgrades after another from trusty to xenial and from xenial to the current LTS version bionic (every 2 years a new LTS version is coming out). The first upgrade was “successful” with a lot of need for adaption in the configurations afterwards. Then after everything worked again, I did another upgrade, which failed because of this issue.

You do not want your system showing you such a message during do-release-update.

That is, I had to fix a distro upgrade that failed in between… challenge accepted 🤓.

Read More

Inherit TimeMachine Backups

jonny Leave a comment

Lately I had to restore my macbook from time machine. Luckily, the reason was that I got a new MacBook 🤓. So I created a last time machine backup, said goodbye to my old macbook, and moved on to my new machine, starting with restoring from my all new time machine backup. I did this several times now and it went well every time. And… no problem this time either and this is what I tell all my jealous Linux and Windows friends. But what I didn’t tell them is that the next step is less fun: inheriting the backups of the old machine.

Read More

VirtualBox v4.2.18 with Native Hard Drive Partition in Mac OSX Lion

jonny Leave a comment

Most of the time you use desktop virtualization, you simply add a virtual hard drive and install a guest operating system (like (K)Ubuntu Linux) in it. But some times – as in my case – you already have another operation system on a partition on your native hard drive. For a mac it’s either via Bootcamp or via something like rEfit enabling you to install a BIOS boot manager like Grub. Since it is kind of uncommon using such an existing partition for desktop virtualization, it is not supported (and documented) so good in the different virtualization solutions.
Read More

Trouble with Black Screen in Parallels Desktop v9 with Linux Guest OS on Native Partition

jonny Leave a comment

This post is a follow up to Using Parallels Desktop v9 with Linux Guest OS in a Native Partition. After the successful utilization of an existing Linux installation in Parallels Desktop v9 you might want to install the Parallel Tools, in order to have a much nicer integration of the guest system.

But I had the problem, that right after installing the Parallel Tools for Linux AMD 64: The screen went black after the login, when the video driver had been loaded. So I uninstalled the parallel tools again and uninstalled all nvidia drivers which I had installed for dual boot use. Afterwards I reinstalled parallel tools and everything was fine.

Using Parallels Desktop v9 with Linux Guest OS in a Native Partition

jonny Leave a comment

Most of the time you use desktop virtualization, you simply add a virtual hard drive and install a guest operating system (like (K)Ubuntu Linux) in it. But some times – as in my case – you already have another operation system on a partition on your native hard drive. For a mac it’s either via Bootcamp or via something like rEfit enabling you to install a BIOS boot manager like Grub. Since it is kind of uncommon using such an existing partition for desktop virtualization, it is not supported (and documented) so good in the different virtualization solutions.
Read More