selfhosted-apps-docker/kopia_backup

Kopia

guide-by-example

WORK IN PROGRESS
WORK IN PROGRESS
WORK IN PROGRESS

Purpose & Overview

Backups.

• Official site
• Github

Kopia is a very new open source backup utility with basicly all modern features.
Cross-platform, deduplication, encryption, compression, multithreaded speed, fully build in cloud storage support, GUI versions, snapshots mounting,...

Written in golang.

In this setup kopia cli is installed directly on a docker host.
A kopia repository is set up.
A script that backs up data is scheduled to run regularly
Bonus info on mounting remote NAS storage on boot using systemd

Some aspects of Kopia

• backup configuration is stored in a repository where backups are stored
  this includes global policy, that is global in sense of a repo, not all of kopia
• you connect to a repository before using it, and disconnect afterwards
  only one repository can be connected at the time(at least for cli version)
• currently to ignore some folders, CACHEDIR.TAG file can be placed inside, with specific specific content and set policy: --ignore-cache-dirs true
• Maintence is automatic
• ..

Files and directory structure

/home/
│ └── ~/
│     └── docker/
│         ├── container-setup #2
│         ├── container-setup #1
│         ├── ...
│
/mnt/
│ └── mirror/
│      └── KOPIA/
│            └── arch_docker_host/
│
/opt/
  └── kopia-backup-home-etc.sh

only the script kopia-backup-home-etc.sh in /opt is created
uf, systemd unit files too, but I am not doing /etc/systemd/system/... even this will probably get deleted

The setup

install kopia

for arch linux, kopia is on AUR yay kopia-bin

the initial steps

use of sudo so that kopia has access everywhere

• the policy info and change

sudo kopia policy get --global
sudo kopia policy list
sudo kopia policy set --global --ignore-cache-dirs true --keep-annual 1 --keep-monthly 6 --keep-weekly 4 --keep-daily 14 --keep-hourly 0 --keep-latest 14

• repo creation

mkdir -p /mnt/mirror/KOPIA/docker_host_kopia
sudo kopia repository create filesystem --path /mnt/mirror/KOPIA/docker_host_kopia
sudo kopia repository connect filesystem --path /mnt/mirror/KOPIA/docker_host_kopia
sudo kopia repository status

• manual run

sudo kopia snapshot create /home/spravca/docker /etc
sudo kopia snapshot list

• mounting a backup

sudo kopia mount k7e2b0a503edd7604ff61c68655cd5ad7 /mnt/tmp &
sudo umount /mnt/tmp

the backup script

/opt/kopia-backup-home-etc.sh

#!/bin/bash

#sudo kopia policy set --global --ignore-cache-dirs true --keep-annual 1 --keep-monthly 6 --keep-weekly 4 --keep-daily 14 --keep-hourly 0 --keep-latest 14

REPOSITORY_PATH='/mnt/mirror/KOPIA/docker_host_kopia'
BACKUP_THIS='/home /etc'
export KOPIA_PASSWORD='aaa'

kopia repository connect filesystem --path $REPOSITORY_PATH
kopia snapshot create $BACKUP_THIS
kopia repository disconnect

Automatic execution using systemd

usually cron is used, but systemd provides better logging and control, so better get used to using it

kopia-home-etc.service

[Unit]
Description=kopia backup
Wants=network-online.target
After=network-online.target
ConditionACPower=true
OnFailure=notify-ntfy@%i.service

[Service]
Type=oneshot

# Lower CPU and I/O priority.
Nice=19
CPUSchedulingPolicy=batch
IOSchedulingPriority=7

IPAccounting=true
PrivateTmp=true
Environment="HOME=/root"

ExecStart=/usr/bin/kopia snapshot create --all

kopia-home-etc.timer

[Unit]
Description=Run kopia backup

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

Accessing the backup files

Mounting using systemd

• the name of mount and automount files MUST correspond with the path
  instead of / a - is used, but otherwise it must be the mounting path in name
• for mounting that does not fail on boot, and mounts the target only on request enable automount file, not mount file, so:
  sudo systemctl enable mnt-mirror.automount

mnt-mirror.mount

[Unit]
Description=3TB truenas mirror mount

[Mount]
What=//10.0.19.11/Mirror
Where=/mnt/mirror
Type=cifs
Options=rw,username=kopia,password=aaa,file_mode=0644,dir_mode=0755,uid=1000,gid=1000

[Install]
WantedBy=multi-user.target

mnt-mirror.automount

[Unit]
Description=3TB truenas mirror mount

[Automount]
Where=/mnt/mirror

[Install]
WantedBy=multi-user.target

Remote backup