Apple’s asr Badly Broken in Mac OS X 10.4.6?

April 23rd, 2006

In my article The State of Backup and Cloning Tools under Mac OS X, I investigated the metadata preservation capabilities of several command-line utilities, among them Apple’s asr (Apple Software Restore) command-line tool. The same tool is used by the Apple Disk Utility GUI. It seems that its behavior in file-by-file copying mode has changed drastically between OS X 10.4.5 and OS X 10.4.6—see the following table that corresponds to the table in my earlier post. Refer to my earlier post for an explanation of the metadata classes.

ownSOpermBFFFlckMDCDFC [a]RFEAACLind
ASR (file mode)Apple (10.4.5)x-x?xxx-xxxx-
ASR (file mode)Apple (10.4.6)x-x-x-x-xx---

Basically, the BSD flags (not sure if they were preserved in 10.4.5), the locked flag, HFS+ extended attributes, and ACLs are not preserved.

The conclusion would be that the asr tool, upon which many people rely for backup and machine setup purposes, is badly broken in OS X 10.4.6. This would also invalidate my earlier recommendations made here and here. Can readers confirm this behavior? I’ve filed the bug as Radar #4523878.

Update 2006-04-26: Apple has marked the bug as duplicate, effectively acknowledging it is indeed a bug in asr.

Update 2006-07-11: Apple has not indicated any pertinent fixes in the release notes of OS X 10.4.7, and, indeed, I can attest that the situation is unchanged with respect to 10.4.6.

Categories: hacking, macosx, unix

Tags: , , , , , , , , , , ,

39 Comments Add your own

  • 1. Vic  |  April 24th, 2006 at 21:47

    It might be a good idea to submit your findings to various forums to encourage discussion and further investigation.

    Which forums? Ars Technica, Super Duper! forums, Apple Discussions, to name a few.

    And I wish that Apple would just fix Rsync. We’re at version 10.4.6 of OS X and it still doesn’t work right.

  • 2. Kevin Purcell  |  April 25th, 2006 at 17:44

    Vic: What is wrong with rsync in Tiger?

  • 3. caleban  |  April 25th, 2006 at 18:01

    One thread among many http://discussions.apple.com/thread.jspa?threadID=383125

    Many people get errors with the extended attributes flag turned on.

  • 4. Kevin Purcell  |  April 25th, 2006 at 18:07

    Ah , a link is always good.

    What is wrong with rync see a previous post here.

  • 5. Paul Atkins  |  April 26th, 2006 at 03:18

    I think we are going to see more and more restriction coming out of Apple as it gets deeper into Intel processors, EFI, HDCP, and BlueRay.

    Cloning a boot drive will not be a option because it copies copy protection schemes.

    I have not updated any our our lab machines to 10.4.6, we rely upon cloning as a vital restore feature.

  • 6. maurits  |  April 26th, 2006 at 03:25

    Paul: my personal opinion is that it’s a bit far-fetched to draw a connection between the mundane bug described in my post and political issues such as copy protection, trusted computing, etc.

    Holding off the update might indeed be a good idea if this proves to be true, and the clobbered metadata is of importance for you.

  • 7. Dano  |  April 29th, 2006 at 12:14

    Paul: I would surmise from this information that you would only need to avoid 10.4.6 on the drive used to create your disk image. It shouldn’t matter that 10.4.6 is the version of the OS in your disk image.

  • 8. Dano  |  April 29th, 2006 at 15:16

    Paul: …..or if you are cloning from one hard drive to another and not creating disk images the source volume can still be 10.4.6.

  • 9. C  |  May 5th, 2006 at 17:10

    Maurits : I read this article as well as the two other you wrote about backup. Very interesting, thank you very much.

    I’d just want to be shure I understood well. Making a clone with the asr tool in device-mode is still fine (the problem would just be with the file-mode) and still the most efficient way of making a bootable backup, right ?

    I read we can do asr with “Disk Utility” but I didn’t really understand how. In “Disk Utility” I would also like to know what action use the “file-mode” and what action would use the “device-mode”.

    Thank you.

  • 10. Patrick Hüsler  |  May 9th, 2006 at 21:52

    When creating a new image in “Disk Utility” you can choose between a folder or volume as your image source. Thats probabely the difference.

    Has anyone tested asr’s multicast features?

  • 11. Dano  |  May 10th, 2006 at 14:13

    I am working on a NetBoot imaging workflow here at work. Currently I am just finishing off the post processing (ByHost settings resolution, naming computers, etc) required. My next task will be to incorporate multicast into the mix. I will try to post something here then.

  • 12. C  |  May 13th, 2006 at 13:38

    Thank you Patrick for your answer. Do you, or anybody else, know if the ASR (device mode) is damaged as well or if it is just the ASR (file mode) that is ?

  • 13. maurits  |  May 13th, 2006 at 14:05

    C: from the asr man page: –erase erase erases target and is required if a fast block-copy restore is desired.

    Obviously, device-level copying is only used when you work on complete volumes. When restoring from an image, the image needs to be prescanned, I guess.

    The device-level copying mode should not be buggy.

  • 14. Anonymous  |  May 13th, 2006 at 16:58

    So is it actually the most efficient way to clone a Hard Drive ? Even better than SuperDuper (because of the inode) ?

  • 15. C  |  May 13th, 2006 at 17:01

    I forgot to to put my nam on message before. And also to say I speak about the ASR (device-mode)

  • 16. RM  |  June 14th, 2006 at 23:13

    I am very new to apple… I had a problem that i think related to this topic. I booted on my Mac OS X CD (10.4.4) and i created (using disk utility) a read write image. After doing a fresh install and all updates (wich got me 10.4.6) i couldn’t read the image to get my data. I got no mountable files error. I tried to mount it on a friend Mac (10.3.9) it mounted but the images inside the image won’t work (I have 3 previous images in it that worked fine before packing the disk into an image) with the same error. I use an iMAC Intel.

  • 17. gopher  |  July 1st, 2006 at 23:55

    I hope you submitted a bug report to http://bugreporter.apple.com/

    Also see if the issue has been fixed with the recent 10.4.7 upgrade.

  • 18. maurits  |  July 2nd, 2006 at 12:13

    gopher: Please read my post. I even included the radar #. Haven’t updated to 10.4.7 yet, will test ASAP.

  • 19. NM  |  July 10th, 2006 at 23:31

    hello, in 10.4.7 there is /usr/sbin/asr asr -v gives asr: version 64.6

    best regards N

  • 20. maurits  |  July 11th, 2006 at 01:22

    NM, true – asr has been updated in 10.4.7, but this bug has not been fixed. See also my above update to the post.

  • 21. Dano  |  August 20th, 2006 at 14:18

    After experimenting with mASR, I’ve discovered that it is necessary to create the disk image “from folder”. Creating a disk image “from device” will not work from mASR as mASR will not stream these types of disk images. Unfortunately some of the missed metadata is problematic in my case. I create images utilizing a number of desktop publishing/graphics applications and a disk image created from folder loses come ColorSync workflows. These color workflows are preserved when creating the image from the device.

  • 22. Dano  |  August 20th, 2006 at 14:25

    One more note. The issue I noted in the previous post was an issue in OS (Tiger) versions prior to v10.4.6. In my case I noted this to be true going back to 10.4.2 (and maybe back to 10.4 but my first production Tiger image was based on 10.4.2). The real problem is that disk images from folder or from device should retain all metadata.

  • 23. Grape  |  October 15th, 2006 at 07:43

    What is the status of asr in the latest release, 10.4.8? Have the bugs been fixed?

  • 24. a  |  November 3rd, 2006 at 16:32

    New asr version in OS X 10.4.8

    $ asr -v asr: version 66

  • 25. Alan  |  December 19th, 2006 at 22:32

    I must be missing something, by OS X 10.4.8 asr version is:

    $ asr -v asr: version 64.6 $ _

  • 26. Yvo  |  January 3rd, 2007 at 08:28

    My asr in 10.4.8 (Intel based) is 66 as well: stewie-the-2nd:~ yvo$ asr --version asr: version 66

  • 27. tyler  |  January 26th, 2007 at 19:27

    how about an update in 10.4.8???

  • 28. Chaz  |  February 4th, 2007 at 17:49

    I have 10.4.8 with a Core 2 Duo:

    asr version asr: version 68

    Also, is it then determined that asr device mode will copy all meta data?

  • 29. Jack Givens  |  February 12th, 2007 at 19:43

    Great article. My experience matches your results as well. Thanks for filling in some gaps. This I believe is the biggest failure in Apple design and implementation of OS X. I have used Apple Backup and have stopped using it. It stores data in sparse disk images ultimately (if you look in the backup bundle). Since I often got bad catalogs I had to resort to manually mounting these spare images and recovering what I could. I do not recommend Apple Backup. Another form of backup that has issues on OS X is svn. As a developer I often can not add some files to a svn repository becuase of resource forks. I wish Apple would force all bundle operations to XML and move other meta-data to flat files like .DS_Store.

  • 30. Patrick  |  February 21st, 2007 at 10:43

    @Jack Givens As for Subversion, it is generally a bad idea to backup the repository as such, because this could lead to a corrupt repository. This is for the very same reason, that a database (eg. mysql) is backed up using dumps and not backing up the db file(s). You might wanna look into svnadmin dump, this is by far the best way to backup your repositories. Backupscripts are available on the net.

  • 31. Norm  |  May 22nd, 2007 at 23:50

    In 10.4.9 (PPC) I’m still at asr version 64.6. I tried to clone a bootable volume (RAID mirror to RAID mirror, if it makes a difference) using the command:

    sudo asr restore –source / –target “/Volumes/Boot RAID” –erase

    and asr quickly responds:

    Validating target…done Validating source…done

    The target volume will not be bootable. Continue anyway? [yn]:

    and indeed creates a volume which is not bootable. Any ideas what’s up with this?

  • 32. robert  |  December 11th, 2007 at 11:07

    my mac comes on but the screen dont turn on

  • 33. nicola  |  January 13th, 2008 at 15:24

    Can you confirm what I am experiencing now with Leopard (10.5.1), namely, that asr in file copy mode still does not preserve ACL, extended attributes and quarantine information and that, moreover, the creation date is set equal to the modification date (BF and lck are preserved, however — not sure about SO)?

    Instead, asr in block copy mode (what you call dev mode) works flawlessly, fortunately. I’ve successfully copied my startup volume to an external hd with Disk Utility’s Restore function: to do that, I had to boot from Leopard’s dvd (as far as I’ve understood from the badly written asr man page, the source disk must be unmounted for block copy to take place), set my startup disk as source, my external hd as destination and check “Erase destination” (a further requirement for block copy to occur).

    Another solution, which you do not mention, is that the suggested way (from man page) to backup a volume (or a directory) is to use “Image from Folder…” in Disk Utility to create a read-only or compressed disk image that can then be restored by asr. That uses hdiutil, which does a file by file copy (as per hdiutil man page). In my tests, however, inodes and creation dates are not preserved and, if a folder is copied, Spotlight comments at the top level of the directory hierarchy are lost (this is not an issue if a volume is copied).

    My conclusion (I’d like to hear others’ experience with Leopard) is that asr is a very good choice for backups at a device level, provided that block copy is used. For data in one’s home directory, in my opinion the best idea is just to create a blank disk image and, in the Finder, drag and drop a folder into it. The Finder preserves everything but inodes and the BSD arch flag (permissions should not be a problem for one’s own data). Even if inodes change, Finder aliases are properly dealt with.

    Btw, any chance to see an updated article for Leopard?

  • 34. nicola  |  January 13th, 2008 at 16:09

    Just a remark: BSD flags are not preserved by the Finder (as I’ve mistakenly reported).

  • 35. lst  |  January 30th, 2008 at 06:22

    Great article! I’m hope more information comes in. fyi, asr is version 72 is 10.4.11.

    So can anyone comment on the current state of Apple’s backup utilities in 10.4.11 and 10.5.1?

    I’m still on Tiger right now and I use Disk Utility for the majority of my backups, although I have been comparing them with Super Duper!, CCC, and Chronosync lately. I’m not sure which one is the best, though DU has been producing bootable backups.

    Is Disk Utility’s New Image from folder option better than it’s New Image from Device option? I’m wondering what other people’s experiences are with creating a sparse disk image and using Finder to drag files into it. Btw, Filevault puts your entire home directory inside and encrypted disk image. In theory this would allow you to preserve all your metadata simply by copying the disk image to another drive. Also, you could simply try putting the majority of your files in disk images (encrypted or not) in the first place, and working from there. That should allow complete metadata preservation, at the expense of some inconvenience in having to wait for them to load.

    Maybe the best thing is to simply avoid using the types of metadata which are known to cause problems until Apple fixes these problems.

  • 36. lst  |  January 30th, 2008 at 06:27

    Here’s a pair of handy links by the way. The second one includes a link to a handy set of files to help test metadata backup.

    Geek to Live: Complete, free Mac backup

    File copying/synchronization software and your metadata (and data!) | iNik

  • 37. Rob  |  February 6th, 2008 at 20:28

    I can confirm that asr in its file-copy mode is still badly broken in Tiger 10.4.11. (The asr version is 72).

    I created two disk images and put some files in the first. Then I used asr to copy the files from the first disk image to the second.

    1. No extended attributes were copied.
    2. No BSD flags were copied.
    3. No locked flags were copied (which are just a type of BSD flag).
    4. Symlink ownership was not copied.

    And it really messed up the Symlink ownership. One of symlinks had the owner of rob:dummy. When the symlink was copied, it became root:rob. How it did this is beyond my comprehension.

    Come on Apple. Why don’t you fix these long-standing bugs!

  • 38. Carl Williams  |  March 3rd, 2008 at 07:34

    I guess this explains a lot. I swapped out my MacBook drive for a bigger one, partitioned for dual-boot, used Apple’s disk utility to clone the original drive to a suitably large HFS+ partition. This has worked fine in the past on an old TiBook, and appeared OK at first on the MacBook.

    (GUID partition scheme, BTW, the dual boot is with Linux and I’m using rEFIt to choose OS at boot-time, since Apple so kindly withdrew BootCamp Beta in that oh-so-typical let’s-renege-on-our-advertising -promises-after-all-they’re-only-customers style.)

    The clone system boots and generally runs OK but a fair few owners and permissions were wrong, including the Unix sticky-bit flags missing from here and there, and there are infuriating issues with icon associations, including the following somewhat frustrating effect (which only appears to affect certain apps):

    The icons for files associated with VLC have turned, mostly, into the blank “file” default icon.

    If I use file…info to change the app associated with, say, an avi file, its icon changes, If I click “change all” it changes back to the blank one, and no others change.

    If I have my original system disc mounted, via a caddy, as well as the clone, then the icons appear and behave properly. Clearly the cloning process has managed to preserve a very persistent association with the original VLC icon’s inode, or that of some or other icon cache, and I’ve been unable to track it down.

    I’ve tried deleting various cache files, ALL .DS_Store and Desktop DB/DF files, restarting the finder, logging out, rebooting, all the rest.

    This is only vaguely related to the brokenness of asr, and any suggestions about where to look for the relevant evil cache or whatever would be highly off-topic (but very welcome via a link to a separate discussion or something)

    I’m getting increasingly pissed off with Apple’s poorly documented and bewildering mess of overt, hidden, semi-hidden, inaccessible and whatever meta-data, and it comes as no real surprise to me that Apple’s own tools don’t deal properly with half of it. While unsurprising, though, it’s very disappointing, especially as they KNOW about it and have done for some time.

    My asr version is 72, Tiger 10.4.11, by the way.

    I wonder if my original software disc is early enough to have a less broken asr on it, and if my problems might be fixed by using that to clone my old drive to the new one?

  • 39. g  |  August 21st, 2008 at 00:47

    So is carbon copy cloner better than using apple’s DU 10.4.11?

    I have two externals that I used ccc to clone my powerbooks drive to.

Leave a Comment

hidden

Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Trackback this post  |  Subscribe to the comments via RSS Feed


Feeds

Categories

Archives

Most Recent Posts