Sunday, April 16, 2023

Changing the backup approach

Here is my current backup strategy. Basically all devices backup to a NAS (which also hosts shared files). And periodically, backup the whole NAS to an offline PC:

The "backup of backup" setup has a few downsides:

  • The work of digging out and connecting the old PC to the network periodically. Additional hardware / software maintenance.
  • The software RAID 5 needs multiple HDDs, usually retired hardware previously used in the NAS. High risk of failure.

An alternative would be using external USB HDDs for the "backup of backup":

First attempt:

  • Use external HDDs via USB/eSATA for the "backup of backup". Rotate between two external HDDs to minimize the chance of single point of failure
  • Use veracrypt to encrypt the whole external HDD device
  • Within the encrypted volume, use BTRFS and mount with compression. Can also use the snapshot function of BTRFS if necessary
  • Sadly the NAS is running a very old version of Synology software and it doesn't support veracrypt or BTRFS. Will need to do this on a desktop PC.

While BTRFS do compression and snapshots, will need to pay attention on the disk usage. It is inefficient and difficult to re-balance if the free space is low (the main reason why I gave up on using BTRFS on boot partition even when it is the default for OpenSUSE). Should probably do a re-balance after every rsync.

Preliminary results:

  • Only less than 1/10th of the files are actually compressed. Probably because the NAS manly stores photos, movies, and backup archives... in which already in compressed format.
~$ sudo compsize /mnt/backup
Processed 117244 files, 1223867 regular extents (1223867 refs), 17472 inline.
Type       Perc     Disk Usage   Uncompressed Referenced   
TOTAL       95%      1.5T         1.6T         1.6T        
none       100%      1.5T         1.5T         1.5T        
zstd        22%       22G         103G         103G

  • Encounter RAID read errors and veracrypt issues. May need to think about doing checksum after backup
ata7.00: exception Emask 0x0 SAct 0xff800c SErr 0x0 action 0x0
ata7.00: irq_stat 0x40000008
ata7.00: failed command: READ FPDMA QUEUED
ata7.00: cmd 60/08:78:88:92:ed/00:00:73:00:00/40 tag 15 ncq dma 4096 in
                                    res 51/40:01:8f:92:ed/00:00:73:00:00/40 Emask 0x409 (media error) <F>
ata7.00: status: { DRDY ERR }
ata7.00: error: { UNC }
ata7.00: ATA Identify Device Log not supported
ata7.00: ATA Identify Device Log not supported
ata7.00: configured for UDMA/133
sd 6:0:0:0: [sde] tag#15 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=3s
sd 6:0:0:0: [sde] tag#15 Sense Key : Medium Error [current] 
sd 6:0:0:0: [sde] tag#15 Add. Sense: Unrecovered read error - auto reallocate failed
sd 6:0:0:0: [sde] tag#15 CDB: Read(10) 28 00 73 ed 92 88 00 00 08 00
blk_update_request: I/O error, dev sde, sector 1944949391 op 0x0:(READ) flags 0x4000 phys_seg 1 prio class 0
ata7: EH complete
md/raid:md0: read error corrected (8 sectors at 1944947336 on sde1)

To be continued...

Some useful commands, assuming sdd is the external HDD:

# create veracrypt volume
sudo veracrypt --volume-type=normal -c /dev/sdd
# mount veracrypt volume
sudo veracrypt --filesystem=none --slot=1 /dev/sdd
# create BTRFS
sudo mkfs.btrfs -L backup /dev/mapper/veracrypt1
# mount BTRFS with compression
sudo mount -o compress=zstd:15 /dev/mapper/veracrypt1 /mnt/backup
# check BTRFS compression ratio
sudo compsize /mnt/backup


No comments: