A year has passed since I published my articles on the state of metadata conservation in Mac backup/file copying software (here, here, here, and here).
It is time for a small update on the matter. Despite the numerous justified requests, I have, unfortunately, not found the time to bake the set of hacked-together scripts that I used for testing into a full-blown test suite. As a result, I have neither amended nor updated my test results in the meantime. My apologies.
In March, inik.net published a disk image with metadata-laden files for test purposes. Based on this set of files, inik.net conducted a new survey of backup and file copying tools. See for yourself for the results; things seem to have slightly improved.
However, the following could be a true breakthrough contribution. Testing for metadata conservation has, up to today, still been somewhat of a voodoo skill, and I’m not surprised that to the best of my knowledge only one person seems to have taken a serious shot at it after my posts. But now, this gaping hole has been filled by Nathaniel Gray with a fully automated test suite for metadata conservation. This could possibly make metadata conservation testing feasible for a wide audience.
I haven’t tested it myself yet, but from what Nathaniel writes it looks very promising. It seems to replace all the tedious manual script runs and BBEdit diff runs that I went through. Great!
I’m excited to see test results for all those tools that I couldn’t test. And eventually, I hope that the state of metadata conservation on the Mac will continue to improve.
Tags: apple, cloning, metadata, os x, Tri Backup
Categories: macosx, hacking
April 28th, 2007
Yesterday, an interesting discussion about file metadata has begun on darwin-dev. Apple’s Jordan Hubbard argued how file creation dates should not be preserved when copying files.
In this piece, I counter that treating file creation dates as first-class metadata citizens and preserving them upon copying is the more sensible thing to do, and eventually represents the behavior expected by most Mac users.
Categories: macosx, unix, hacking
Continue Reading June 27th, 2006
In my article The State of Backup and Cloning Tools under Mac OS X, I investigated the metadata preservation capabilities of several command-line utilities, among them Apple’s asr (Apple Software Restore) command-line tool. The same tool is used by the Apple Disk Utility GUI. It seems that its behavior in file-by-file copying mode has changed drastically between OS X 10.4.5 and OS X 10.4.6—see the following table that corresponds to the table in my earlier post. Refer to my earlier post for an explanation of the metadata classes.
|
| | own | SO | perm | BF | FF | lck | MD | CD | FC [a] | RF | EA | ACL | ind |
| ASR (file mode) | Apple (10.4.5) | x | - | x | ? | x | x | x | - | x | x | x | x | -
|
| ASR (file mode) | Apple (10.4.6) | x | - | x | - | x | - | x | - | x | x | - | - | -
|
Basically, the BSD flags (not sure if they were preserved in 10.4.5), the locked flag, HFS+ extended attributes, and ACLs are not preserved.
The conclusion would be that the asr tool, upon which many people rely for backup and machine setup purposes, is badly broken in OS X 10.4.6. This would also invalidate my earlier recommendations made here and here. Can readers confirm this behavior? I’ve filed the bug as Radar #4523878.
Update 2006-04-26: Apple has marked the bug as duplicate, effectively acknowledging it is indeed a bug in asr.
Update 2006-07-11: Apple has not indicated any pertinent fixes in the release notes of OS X 10.4.7, and, indeed, I can attest that the situation is unchanged with respect to 10.4.6.
Tags: 10.4.5, 10.4.6, apple, Apple Disk Utility, Apple Software Restore, asr, backup, cloning, mac, mac os x, metadata, tiger
Categories: macosx, unix, hacking
April 23rd, 2006
Earlier, I wrote about The State of Backup and Cloning Tools under Mac OS X, where I made the point that copying files on Mac OS X is not trivial because of the metadata associated with files.
I analyzed a variety of file copying engines, most of them command-line tools, and demonstrated how they fare in preserving file metadata.
In this article, I will investigate commonly used GUI backup/cloning tools for Mac OS X. The tools vary widely with respect to their feature set; the features are irrelevant here. I will concentrate purely on the underlying functionality of copying files. A backup tool needs to be able to copy files faithfully for a successful restore in case desaster has struck. The surprising conclusion of my investigation is that almost all Macintosh Backup tools fail at their most basic task, the faithful copying of files.
Categories: macosx, hacking
Continue Reading April 23rd, 2006
Introduction
Back in the days of OS 9, backing up files was fairly easy. One would just use the Finder to copy files and directories to another volume, and be done. The simplicity, unfortunately, is gone with OS X. Such a simplistic approach is no longer a guarantee to preserve all data faithfully (neither is it a simple or reliable approach for a regular backup procedure). The trouble on OS X is mostly related to metadata, i.e., data about files and directories (such as modification date, file creator/type, Unix permissions, etc.).
Another problem arises when a complete system partition shall be backed up and be bootable later on. Making a backup bootable is not trivial.
Superficially, one could nourish high expectations about the state of backup solutions on Mac OS X, because the underlying BSD Unix core has made all the mature backup and file copying tools that have been developed for Unix systems available on the platform. However, the fly in the ointment is that these tools are generally not aware of Mac OS X metadata and, hence, fail to produce a faithful backup.
This essay will first investigate means to copy files as completely and reliably as possible on Mac OS X, if possible with free and open-source tools. It will conclude with an (incomplete) survey of dedicated backup tools. The tools covered are not only relevant for backup purposes, but also for the case of migrating machines, when the content of one hard drive is to be cloned to another one.
I will not address common features of backup software such as scheduling, backup management, and incremental backups. This piece will be solely about the bare basics of copying files.
The analysis presented here assumes a recent install of OS X 10.4.5 (Tiger) with all updates. The state of backup and cloning tools has already been worse, so there is no need to shoot ourselves into the foot by using outdated tools.
(more…)
Tags: apple, asr, backup, cloning, ditto, mac os x, metadata, rsync, superduper, tiger
Categories: macosx, unix, hacking
March 5th, 2006
