Backup Solution - Rsync Wrapper

This script is designed to be a "good enough" solution for home backups. It is a wrapper around rsync via SSH, providing a functional and simple solution that facilitates backups of files from your home computers.

The solution is designed to meet the following three criteria:

1. The data copy is stored (unencrypted) at home on a physical usb hard drive.
2. All computers in the home should be able to easily save a backup of selected data.
3. It should be possible to share data between computers.

This script allows multiple computers to connect to a central rsync server.

graph TD
    A[Computer 1] -->|rsync| B[rsync-server]
    C[Computer 2] -->|rsync| B
    D[Computer 3] -->|rsync| B

Loading

A Raspberry Pi can be set up as the rsync server itself, but other setups are, of course, also possible.

Once everything is set up, file synchronization is started with the following command:

./run_remote_setup.sh sync-folders

Requirements

To run the script, the following two packages need to be installed: jq and parallel.

# Fedora based
sudo dnf install jq parallel
# Debian based
sudo apt-get install jq parallel

Set up the rsync server

Set up the rsync server so that it can be accessed through SSH with a user that has sudo access.

Setting up the Raspberry Pi in headless mode describes how to prepare a Raspberry Pi as an rsync server.

The rsync server should be accessible over SSH with a user that has sudo rights.

In the example below, the user with sudo access is pi, the rsync server has the static IP address 192.168.50.41, and the pi user can access the rsync server over SSH using the key /home/hippo/.ssh/id_ed25519.

Step 1: Set up the USB disk and mount it to /mnt/usb

./run_remote_setup.sh setup-usb-disk -u pi -i 192.168.50.41 -k /home/hippo/.ssh/id_ed25519

Step 2: Set up the shared account user (used when sharing files between computers) and create the folder /mnt/usb/shared/

./run_remote_setup.sh setup-shared -u moln -i 192.168.50.41 -k /home/leonard/.ssh/id_ed25519

Prepare a computer to use the sync-server

Each computer needs its own account on the rsync server. Use the following command to set up an account for a computer.

Use the sudo user and run ./run_remote_setup.sh setup-new-computer. Provide the public key and the name of the computer. In the example below, the name of the computer is hippo.

Step 1: Create an account on the rsync server

./run_remote_setup.sh setup-new-computer hippo "ssh-ed25519 AAAAC3NzaC1lZZI1NTE5AAAIIEGEDX92rbS39FTY2h740yRV4CzQnj/okNa4fKWKatt4 abc@abc.com" -u pi -i 192.168.50.41 -k /home/leonard/.ssh/id_ed2551

account name: Specify the name of the account, hippo is used in the above example. public key: The public key which will be used to grant access to the user.

The script will creat a user and also grant access to the shared user.

Step 2: Create a configuration file on the computer

The configuration file is named ~/.primitive_sync.yaml.

This example shows how to configure which folders to sync.

pi:
  username: "hippo"
  ip_address: "192.168.50.41"
  ssh_key: "/home/hippo/.ssh/id_ed25519"

sync_folders:
  - source: "/home/hippo/projects"
    destination: "/mnt/usb/dhippo/projects"

shared_folders: 
  - source: "/home/hippo/documents/" 
    destination: "/mnt/usb/shared/documents/" 
  - source: "/home/hippo/Pictures/shared/" 
    destination: "/mnt/usb/shared/Pictures/" 

1. username: The user on the rsync server has been given the name hippo (you can choose whatever name you like).

2. ip_address: Specify the static IP given to the rsync server.

3. ssh_key: Specify the key used to access the server.

4. sync_folders: A list of folders to sync which will only be accessible by the configured user (hippo in this case).

   The source points out the folder on the computer to be synced. The destination points out where to sync the folder on the rsync server.

   (The folder for the account will always start with the letter d + username, in this case dhippo)

5. shared_folders: A list of folders to sync which are accessible by all computers.

   The source points out the folder on the computer to be synced. The destination points out where to sync the folder on the rsync server.

Sync folders with the rsync server

To sync, simply run:

./run_remote_setup.sh sync-folders

It's also possible to do a dry run:

./run_remote_setup.sh sync-folders --dry-run

This wrapper does not contain any logic for handling merge conflicts in files. You must be cautious when syncing folders, especially for shared folders where conflicts can arise when the same file is updated from different computers. Remember, the solution is only designed to be good enough.

Setting up the Raspberry Pi in headless mode

This setup will create a pi user and configure the Raspberry Pi in headless mode. You will only be able to access the Pi through SSH using an SSH key for authentication.

1. Install Raspberry Pi Imager

2. Open Raspberry Pi Imager (as root)

3. Select the OS to install CHOOSE OS -> Raspberry Pi OS (other) -> Raspberry Pi OS Lite (64-bit)

4. Select the storage to use

5. Set an appropriate hostname

6. Add your localization settings

7. Create the user that will have sudo authority Add a temporary password.

8. Add Wi-Fi settings if you intend to use Wi-Fi.

9. Enable SSH Use public key authentication Add your public key.

10. Configure your preferences for Raspberry Pi Connect

11. Start writing the image to the memory card

It should now be possible to use SSH to access the Raspberry Pi. Make sure to update all packages.

sudo apt update
sudo apt upgrade