Seth's Unix Tips

August 21, 2008

Products

ExpeDat

SyncDat

Mac OS X Backups

Hardware | Software | Recommendations

Introduction

Several people have recently asked me for recommendations about backing up a Mac OS X system (or computers in general). I have been a fervent believer in backups since I accidentally wiped out three months worth of work on an Apple BASIC database application I had been writing back in middle school. Over the years I have worked with a lot of different computer systems and had to do disaster recovery on several occasions. I've come to very much appreciate backup methods that are accurate and reliable, especially when it comes to recovery time.

So here's my advice on backups. Its focused on Mac OS X systems because that's mostly what I now use, but it should apply generally as well. The OS X situation has an extra level of complexity because it combines the Classic MacOS world with the unix world, and there are still a lot of legacy applications (and people) from each world that don't fully understand the other.

The first question is what hardware do you intend to use for your backups? In all cases, your backup media should be stored as far away from your computer as practical, in a cool, dry, dark, fire-proof place, and not near anything electric or magnetic.

The second question will be what software to use.

Or, you can just skip straight to my recommendations.

Hardware

The main issue here is price versus reliability. Remember, it doesn't do any good to make a backup, if you can't recover the data when you need it.

• CD-R

Since most systems have a CD-R drive, and the media is cheap, this is the most basic way to ensure that critical data is backed up.

PROs: CONs:

Cheap.
So cheap you can keep redundant copies.
Better than nothing for critical files. Only 650 MB at a time.
Deciding what you REALLY want to backup takes time.
Media has limited shelf life (months to a few years).
Slow.

• DVD-R

Much like CD-R above, but with more capacity.

PROs: CONs:

Cheap (if you already have the drive).
Better than nothing for critical files. Only 4.7 GB at a time.
Deciding what you REALLY want to backup takes time.
Media has limited shelf life (months to a few years).
Slow.

• Zip, Jazz, DVD-RAM, ORB, etc.

Contrary to what the salesman may tell you, these types of removable media are simply too unreliable to use for backups.

• External Hard Drive

This is probably the most cost effective solution for casual backup. For not much money you can get an external drive (preferably firewire and bootable) with enough space to backup your whole system. Just remember that while the drive is connected to the machine, its vulnerable to all the same disasters that might cause you to need a backup. Likewise, if your system has any problems at the time of the backup (like a corrupted file that you haven't noticed yet), your backup will have those problems too. You can get around this by rotating two external drives, but at that cost level you may want to look into tape.

PROs: CONs:

Very Fast.
Moderate shelf life (a few years if properly cared for).
Moderate cost and getting cheaper every day. Slightly less portable than tapes or CDs.
Need at least two for redundancy.
Vulnerable to usual drive failure issues.

• Tape Drive

Tape drives are the oldest storage technology still in common use and still represent the best option for professional grade backups. BUT, the type of tape system REALLY, REALLY matters. Cheap tape drives and media are worse than useless as they just give you a false sense of security. Current reputable systems are Mammoth (Exabyte 8mm), DLT (Quantum), LTO (HP, IBM, Segate), and AIT (Sony). For maximum media reliability and recovery, I recommend VXA (Exabyte, formerly Ecrix). (VXA drives can recover data from some amazingly abused or neglected tapes.) For best results, you need four sets of tapes: two for monthly full backups, and two for automated daily incremental backups. Don't forget to clean your drive heads as often as the manufacturer recommends and buy new tapes every year or two.

PROs: CONs:

Moderate Speed.
Low maintenance.
Offers the best chance of disaster recovery.
Phenomenal shelf life (decades if properly cared for). High initial cost.
Expensive media.
Requires dedicated backup software.
Initial setup can be difficult.

Software

In July of 2006 I found it necessary to revamp the backup system at my office. Over the course of three days I made an alarming discovery: there is no good backup solution available for Mac OS X.

By "good", I mean able to reliably backup and restore a working filesystem to the exact state it was in at the time of the backup. That really doesn't sound like much to ask, but if such a solution exists, I have not been able to find it. It is therefore necessary to make compromises, which would be a lot easier if all the software producers were honest about what their products can and cannot do.

Below is a table showing five of the most popular backup methods and their distinguishing features. I've lumped Carbon Copy Cloner and the command line tool ditto together, since CCC is mostly a graphical front end for ditto. Note that this chart is only considering system backups. If all you want to do is backup your own documents, then almost anything that claims to do backups will work.

Retrospect BRU Disk Utility SuperDuper ditto /
Carbon Copy
Cloner

Live Backup
Able to backup the drive of a running system YES YES NO YES YES

Bootable Restore
Restored system disk is able to boot up YES NO YES YES YES

File Selection
Only copy files with specified attributes YES Limited NO Limited Limited

Error Tolerance
Backs up what it can, even if some files have problems YES YES ? NO YES

Overall Reliable
Runs without crashing or freezing NO GUI: No
CLI: Yes Easily
Confused* YES YES

Easy to Use
Obvious how to make it work NO HELL NO Backup: Yes
Restore: No YES YES

Fast
Makes efficient use of hardware NO With no
Compression YES YES YES

Tape Drives
Writes directly to tape YES YES
Great Utilities NO NO NO

Folder Dates
Preserves folder date stamps NO YES YES YES Sometimes

Locked Files
Preserves Locked/uchg attribute YES NO YES^* NO NO

Pipes
Backs up "p" files NO YES YES^* NO NO

Sockets
Backs up "s" files NO NO YES^* YES NO

^* Disk Utility becomes easily confused, requiring that all targeted disks be unmounted and the program quit-and-restarted. Making an image is very easy, and all the attributes are saved. But restoring requires that you first "scan image for restore", then restore using "Erase Destination".

All of the utilities above preserve resource forks, permissions, user/group ids, and all the other stuff that is mandatory for accuracy. But the only way to preserve everything is to dismount the drive and make a disk image using Apple's Disk Utility. This is fine for removable disks. It is not bad for a system that can be shutdown to target disk mode and imaged from a different Mac or rebooted from a system disk. But for a stand-alone machine or a server that needs to keep running, it just doesn't work.

Having said all that, any backup is better than no backup... as long as you understand the limitations. So below is a closer look at some of the more common methods people use to at least make some attempt at preserving their most important data.

• Finder

Drag individual documents or document folders to the media. Also note that iPhoto and iTunes have options to let you burn their data directly to CD-Rs or DVD-Rs. If you have 10.3 (Panther), it may be best to use the Finder "Create Archive" command first, then drag the archive to the media. Just remember that those archives will only be restorable on 10.3 or later systems.

PROs: CONs:

Free
Simple.
Reliable for documents (only).
Preserves some MacOS meta data. Slow
May not work for some applications.
Cannot recover a full, bootable system.
Deciding what you REALLY need to backup takes time.
Does not always preserve ownership or other unix meta data.
Does not work with tape drives

• BSD Command Line Tools

SOME of the unix side command line tools can be used to create backups, but others WILL NOT WORK. Traditional unix copy commands such as cp, cpio, dump, tar, zip, and others do not preserve the HFS+ meta data and SHOULD NOT BE USED for backups. (Mac OS X 10.4 makes an effort to preserve meta data with some, but not all, of the unix tools, so don't rely on them.)

CpMac, ditto, and perhaps a few others can produce reliable copies, but using them can be tricky and none are guaranteed to fully restore a working system. For OS X 10.3 and earlier, you must use "ditto -rsrc" to preserve meta data. With 10.4, ditto seems to do the right thing by default and I have successfully used it to replicate bootable systems. Still, you should be wary of using these tools unless you have an intimate understanding of both unix and Mac OS X file system structures

PROs: CONs:

Free.
ditto is fairly accurate with 10.4 and later. Easy to screw up.
May appear to work, but can hide serious flaws that prevent recovery.
Some tools (like tar) will not recover a full, bootable system.
Does not work with tape drives.

• Copying, Cloning, or Synchronization Software

There are a number of applications out there that tout their ability to perform backups. Some claim they will produce a bootable system. Most have flaws. I've been using SuperDuper a while now and find it satisfactory for backing up live systems to an external drive.

PROs: CONs:

Cheap.
Easy to use.
May be fairly accurate. Lots of misleading claims.
May not be reliable or accurate.
Does not work with tape

• Dedicated Backup Software

For Mac OS X , there are only three choices right now: Retrospect, Tolis BRU, and Apple Backup. Apple's "Time Machine" software is set to appear with 10.5 and I'll update this page when I have some experience with it.

Apple Backup is probably the easiest to use and cheapest to acquire. From what I can tell, it does a good job of backup data to iDisk, external drives, CDs or DVDs. It does not support tape, nor can it backup applications, let a lone a full system.

PROs: CONs:

Inexpensive.
Easy to use.
Reliable for documents CONs: Can't backup applications or whole systems.
No tape support.
Only available with a .mac subscription

Dantz/EMC/Insignia Retrospect is the oldest Mac backup software around. Its major claim to fame is that it is the only software that can completely backup and restore an entire bootable system to just about any media in existence. (It can mess up some date stamps on restore, but that's a relatively minor quibble.) Its major drawback is that it is expensive, has a nightmare inducing interface, and is butt-freaking-slow. Absolutely NOT recommended if you are backing up more than a handful of systems as it does not scale well above a couple hundred thousand files. Another drawback is that the software has changed hands a couple of times in recent years and development appears to have almost stopped. This is really bad because the software is becoming increasingly buggy and unstable as it falls further and further behind OS X upgrades.

PROs: CONs:

Almost 100% accurate backup.
Restores bootable drives.
Sophisticated file selection.
Supports tons of backup hardware Expensive.
Insane and buggy interface.
Maddeningly slow.
Crash prone with firewire drives on 10.3 or later

Tolis BRU was new to OS X when I first wrote this a few years ago, and they don't seem to have improved much since. They say they have been doing unix backups for years, but I have no experience with them. They claim 100% coverage of filesystem attributes, but this is just not true: see the chart above. Their graphical interface is not worth mentioning, and their documentation is sparse at best. I really want to like this software because of its unix roots, but its inability to restore a bootable backup and seemingly stagnant development makes me distrustful. Still, it has some nifty features that would be handy in a large environment and is very flexible and fast (provided you avoid the UI). If you are a unix guru used to command line utilities, or if you are looking to do large scale enterprise level backups to huge tape libraries, this is worth trying.

PROs: CONs:

Almost 100% accurate.
Scriptable command line utilities.
Seems to support all common backup hardware.
Nice features for backing up a large network of computers CONs: Expensive.
Requires unix experience.
Limited file selection capability.
Useless GUI.
Restored drive not bootable unless you first install the same operating system version. Hope you have the right install disks!

Recommendations

For the average home user who would not otherwise do backups at all and has no money:: Decide which files are most important to you and drag them to new CD-Rs or DVD-Rs every month or so.
For the average home user whose data is worth spending a few bucks to preserve:: Get Apple Backup or the low-end version of Retrospect and backup to DVD-Rs or an external drive once per month.
For the home user who is serious about backups:: Get an external firewire hard drive and one of the cloning applications.
For the power user, enterprise user, or the just plain paranoid:: Get Retrospect or BRU, a couple of firewire hard drives, and a VXA tape drive. Do full system backups every month and incremental backups every day. Rotate between at least two media sets for each backup type (at least four sets of media) with one set of each stored off-site at all times.
What I use:: BRU with a VXA-1 firewire tape drive daily, SuperDuper with firewire hard drives monthly.