A year has passed since I published my articles on the state of metadata conservation in Mac backup/file copying software (here, here, here, and here).
It is time for a small update on the matter. Despite the numerous justified requests, I have, unfortunately, not found the time to bake the set of hacked-together scripts that I used for testing into a full-blown test suite. As a result, I have neither amended nor updated my test results in the meantime. My apologies.
In March, inik.net published a disk image with metadata-laden files for test purposes. Based on this set of files, inik.net conducted a new survey of backup and file copying tools. See for yourself for the results; things seem to have slightly improved.
However, the following could be a true breakthrough contribution. Testing for metadata conservation has, up to today, still been somewhat of a voodoo skill, and I’m not surprised that to the best of my knowledge only one person seems to have taken a serious shot at it after my posts. But now, this gaping hole has been filled by Nathaniel Gray with a fully automated test suite for metadata conservation. This could possibly make metadata conservation testing feasible for a wide audience.
I haven’t tested it myself yet, but from what Nathaniel writes it looks very promising. It seems to replace all the tedious manual script runs and BBEdit diff runs that I went through. Great!
I’m excited to see test results for all those tools that I couldn’t test. And eventually, I hope that the state of metadata conservation on the Mac will continue to improve.
Tags: apple, cloning, metadata, os x, Tri Backup
Categories: macosx, hacking
April 28th, 2007
Yesterday, an interesting discussion about file metadata has begun on darwin-dev. Apple’s Jordan Hubbard argued how file creation dates should not be preserved when copying files.
In this piece, I counter that treating file creation dates as first-class metadata citizens and preserving them upon copying is the more sensible thing to do, and eventually represents the behavior expected by most Mac users.
Categories: macosx, unix, hacking
Continue Reading June 27th, 2006
In my article The State of Backup and Cloning Tools under Mac OS X, I investigated the metadata preservation capabilities of several command-line utilities, among them Apple’s asr (Apple Software Restore) command-line tool. The same tool is used by the Apple Disk Utility GUI. It seems that its behavior in file-by-file copying mode has changed drastically between OS X 10.4.5 and OS X 10.4.6—see the following table that corresponds to the table in my earlier post. Refer to my earlier post for an explanation of the metadata classes.
|
| | own | SO | perm | BF | FF | lck | MD | CD | FC [a] | RF | EA | ACL | ind |
| ASR (file mode) | Apple (10.4.5) | x | - | x | ? | x | x | x | - | x | x | x | x | -
|
| ASR (file mode) | Apple (10.4.6) | x | - | x | - | x | - | x | - | x | x | - | - | -
|
Basically, the BSD flags (not sure if they were preserved in 10.4.5), the locked flag, HFS+ extended attributes, and ACLs are not preserved.
The conclusion would be that the asr tool, upon which many people rely for backup and machine setup purposes, is badly broken in OS X 10.4.6. This would also invalidate my earlier recommendations made here and here. Can readers confirm this behavior? I’ve filed the bug as Radar #4523878.
Update 2006-04-26: Apple has marked the bug as duplicate, effectively acknowledging it is indeed a bug in asr.
Update 2006-07-11: Apple has not indicated any pertinent fixes in the release notes of OS X 10.4.7, and, indeed, I can attest that the situation is unchanged with respect to 10.4.6.
Tags: 10.4.5, 10.4.6, apple, Apple Disk Utility, Apple Software Restore, asr, backup, cloning, mac, mac os x, metadata, tiger
Categories: macosx, unix, hacking
April 23rd, 2006
Earlier, I wrote about The State of Backup and Cloning Tools under Mac OS X, where I made the point that copying files on Mac OS X is not trivial because of the metadata associated with files.
I analyzed a variety of file copying engines, most of them command-line tools, and demonstrated how they fare in preserving file metadata.
In this article, I will investigate commonly used GUI backup/cloning tools for Mac OS X. The tools vary widely with respect to their feature set; the features are irrelevant here. I will concentrate purely on the underlying functionality of copying files. A backup tool needs to be able to copy files faithfully for a successful restore in case desaster has struck. The surprising conclusion of my investigation is that almost all Macintosh Backup tools fail at their most basic task, the faithful copying of files.
Categories: macosx, hacking
Continue Reading April 23rd, 2006
Oh happy news! Apple has today released Mac OS X 10.4.6, which includes the new iSync 2.2. I didn’t consider it possible, but at last Apple seems to have extended iSync support to Nokia Series 40 devices! Nokia Series 40 devices include popular phones such as the 6021, 6230/6230i, and 6820/6822. Until today, iSync support has been limited to Series 60 smart phones.
List of iSync supported devices
This welcome addition finally makes Nokia users who didn’t want to invest in a clumsy smart phone first-class citizens on the Mac, and probably at the same time destroys a large part of the business of third-party developer MacMedia.
I haven’t tried it myself, but will soon. I am curious how practicable the sync mechanism will be.
Tags: 10.4.6, 6021, 6230, 6820, apple, isync, Nokia, os x, Series 40, synchronization
Categories: default
April 4th, 2006
Introduction
Back in the days of OS 9, backing up files was fairly easy. One would just use the Finder to copy files and directories to another volume, and be done. The simplicity, unfortunately, is gone with OS X. Such a simplistic approach is no longer a guarantee to preserve all data faithfully (neither is it a simple or reliable approach for a regular backup procedure). The trouble on OS X is mostly related to metadata, i.e., data about files and directories (such as modification date, file creator/type, Unix permissions, etc.).
Another problem arises when a complete system partition shall be backed up and be bootable later on. Making a backup bootable is not trivial.
Superficially, one could nourish high expectations about the state of backup solutions on Mac OS X, because the underlying BSD Unix core has made all the mature backup and file copying tools that have been developed for Unix systems available on the platform. However, the fly in the ointment is that these tools are generally not aware of Mac OS X metadata and, hence, fail to produce a faithful backup.
This essay will first investigate means to copy files as completely and reliably as possible on Mac OS X, if possible with free and open-source tools. It will conclude with an (incomplete) survey of dedicated backup tools. The tools covered are not only relevant for backup purposes, but also for the case of migrating machines, when the content of one hard drive is to be cloned to another one.
I will not address common features of backup software such as scheduling, backup management, and incremental backups. This piece will be solely about the bare basics of copying files.
The analysis presented here assumes a recent install of OS X 10.4.5 (Tiger) with all updates. The state of backup and cloning tools has already been worse, so there is no need to shoot ourselves into the foot by using outdated tools.
(more…)
Tags: apple, asr, backup, cloning, ditto, mac os x, metadata, rsync, superduper, tiger
Categories: macosx, unix, hacking
March 5th, 2006
Update 2006-03-10: It seems that the testing servers for natcheck at MIT have been shut down, so natcheck does not work any more. I have added a few more links to the post.
Pretty much all chat applications inherently have trouble stting up a direct connection when both participants are behind a NAT router. Such a direct connection is needed for audio/video chat and file transfer. iChat is one of the apps having notorious problems…
When the router providing NAT has “consistent port translation”, everything should be fine. However, especially some Netgear wireless routers seem to have trouble with this feature.
(more…)
Tags: apple, audio chat, iChat, mac os x, NAT, p2p, router
Categories: macosx, hacking
March 4th, 2006
The current version of cvs2svn fails on Tiger with the error message
ERROR: your installation of Python does not contain a suitable
DBM module -- cvs2svn cannot continue.
See http://python.org/doc/current/lib/module-anydbm.html to solve.
That’s because there’s no suitable db module in Apple’s python install. However, there is an easy solution that works with the python 2.3 supplied by Apple:
install Berkeley db42 via fink (or however you like)
download bsddb3 from http://pybsddb.sourceforge.net
unpack and perform inside the unpacked directory
python setup.py build
sudo python setup.py install
This should do the job.
See also:
Tags: apple, bsddb3, cvs2svn, mac os x, python, subversion
Categories: macosx, unix, hacking
March 4th, 2006
