mac.column.ted: The trouble with creating bootable drives and discs [Part 1]

Ted Landau
November 2006

I harbor no nostalgia for Mac OS 9. Aside from a couple of old games, I never even think about using OS 9 anymore. With one exception...I long for OS 9's simplicity in creating bootable drives and discs.

How simple was it? For hard drives, it was as simple as dragging a System Folder from one drive to another. Actually, it was even simpler than that. You could usually get by with just copying the System and Finder files that resided inside the System Folder. The rest of the System Folder was desirable but optional. An Installer utility? It was there but not required. Invisible files? None to worry about. Permissions problems? Non-existent.

To create a bootable optical disc (or, similarly, a bootable Zip or floppy disk) was a bit more involved, but not by much.

So how does all of this compare to the situation with Mac OS X today? Here's how:

Bootable hard drives. Creating a bootable hard drive in Mac OS X is not difficult, but is still a significant undertaking. The most common method is to boot from a Mac OS X Install disc, run the Installer utility and install a new copy of Mac OS X. The other common alternative is to create a bootable (mirrored or cloned) copy of an existing bootable drive using backup software (such as Retrospect, DataBackup or SuperDuper!). In either case, you need to work with specialized software. Forget about simply copying a System and Finder file from the Finder. You need to copy tens of thousands of files from the source to the destination. A bootable hard drive requires all the essential files in the /System/Library and /Library folders as well as the invisible Unix directories. Even if you copied all of these files from the Finder, you would not have a bootable drive. You'd still have to worry about additional essential invisible files (most critically the mach.sym and mach_kernel files at the root level of the drive), the "volume bootability characteristics" (as can be set via the Unix bless command), and the proper permissions settings (including the critical "Ignore ownership on this volume" setting in a drive's Info window). Plus, you need a functioning admin user account. In theory, assuming you knew what to do, you could set all of this up yourself. But few users try. In almost all cases, Mac OS X users are dependent on utility software to create bootable drives.

Intel vs. PowerPC. The arrival of Intel Macs has added yet more wrinkles. A drive that boots on PowerPC Macs will not boot on Intel Macs and vice versa. This is partly because each platform uses its own version of Mac OS X. An Apple-supported universal version of Mac OS X does not yet exist (although it's supposedly coming in Leopard). Further complicating matters is that the drive format required for each platform is different. As Apple states: "A PowerPC-based Mac can only install Mac OS X on a disk with the Apple Partition Scheme; An Intel-based Mac can only install Mac OS X on a disk with the GUID Partition Scheme" (see this Apple Knowledge Base article for more details). All of this can lead to failures when trying to install Mac OS X or when trying to startup from a supposedly bootable drive, often without meaningful error messages to explain the cause (see this TidBITS article for some examples).

Even making a successful clone of a bootable drive can get a bit tricky (see this article for a look at just how tricky it can get to restore from a bootable backup). Your success in booting from an external drive will also vary with different hardware. For example, Intel Macs can boot from USB 2.0 drives; PowerPC Macs cannot (at least not without some low level fiddling, as covered in this Mac OS X Hints posting).

If you are determined, despite all of the above, to create a dual-platform "universal" bootable drive, it can be done. But it will take some effort. You'll likely need the guidance of tutorials, such as the one here at MacFixIt or other similar postings. If your eyes start to glaze over as you read these articles, don't be surprised.

Looking back, it's hard to believe how far we have strayed from the simplicity of Mac OS 9. Some may say this is the price we have to pay for the modern advantages of Mac OS X. I am not sure. Regardless, I am surprised at how passively Mac users have accepted all of this without complaining. Yes, the situation will improve with the release of Leopard. It will also be less of a concern as the number of active PowerPC Macs declines. But if you are holding out hope that the simplicity of OS 9 may someday reincarnate in Mac OS X, don't hold your breath. It's not gonna happen.

Bootable discs and flash drives. Matters only get worse when trying to create bootable optical discs (or flash drives). For starters, Mac OS 9 bootable discs had two desirable features largely absent in Mac OS X:

• The discs contained a Finder. This meant that, when booting from optical discs, you went to a real Desktop and your internal hard drive mounted as a separate volume. From here, you could easily access files on the drive, launch applications, or even modify documents.
  
  
• The discs were easy to customize. You could put (and run) whatever applications you wanted on the disc, including third-party utilities. This was ideal for making your own Emergency Utilities disc.

Unlike with hard drives, you can't simply use the Mac OS X Install disc to install the needed software on a CD or DVD. If you can even get Installer to correctly target on an optical disc, it won't work for at least two reasons: the discs are typically not large enough to accommodate the required software and/or optical discs require special boot software that is not provided via the Installer utility.

Still, if all you want is a backup copy of a bootable disc, it is easy enough to do. It's especially easy if you have disc-duplicating software (such as Toast Titanium) and two optical drives. Otherwise, you can use Disk Utility. This method requires the intermediate step of creating a disk image file of the DVD, which you then use to burn to a blank DVD. Even here, if you have a recent Install DVD, you may get stuck at the starting gate: the DVD may be a dual-layer disc while your SuperDrive may only burn single-layer discs.

But what if you seek the Holy Grail here? That is, what if you would like to create your own custom bootable disc, such as an Emergency Disk with your own selection of utilities on it? And on top of that, what if you want it to be a universal disk, capable of booting on both Intel and PowerPC platforms? This turns out to be an almost insurmountable hassle.

As to the custom bootable disc part, you can get most of the way there easily enough, before you trip just before the finish line:

Use Disk Utility to create a read-write disk image of a Mac OS X Install disc. Mount the image. Assuming there is enough room, add your desired utilities to the image. If you need more room, you can delete non-essential files from the image. Then burn the image.

The result: a bootable disc with your custom utilities on it. It all seems fine except for one little problem: you can't access the software you added. There is no Finder available when booting from an optical disc. Instead, the disc defaults to launching one and only one utility on startup. The Mac OS X Install disc, for example, defaults to launching the Installer utility. This means that, while your custom software is on the disc, you have no direct way to access and launch any of the software. What to do?

The quickest, although not the most user-friendly, solution is to take advantage of the fact that you can launch Terminal from the Installer's Utilities menu (available when you boot from a Mac OS X Install disc). Had you booted from a hard drive, you could use the Unix open command to launch applications from Terminal. But Apple did not include this command on the Mac OS X Install disc. Happily, there is an alternative command that works: Type a period (.) followed by the path to the executable application. For example, after booting from a Mac OS X Install disc, here's how to launch the Console application in the Utilities folder on your internal hard drive:

1. Type: cd /Volumes/"{name of your hard drive boot volume}"/Applications/Utilities
   This moves you to the Utilities directory on your hard drive, simplifying the next and critical step.
   
   
2. Type: ./{name of application}.app/Contents/MacOS/{name of application}
   Thus, to launch Console, you would type:
   ./Console.app/Contents/MacOS/Console 

[When you type the last command, you're actually launching the executable binary inside Console.app (which is actually a folder). While you can simply double-click Console.app to execute the binary in the Mac OS X Finder, you need to directly call the binary when using this Terminal procedure]

If you created a custom disc, with your own added software, you could use the same method to locate and launch any of the programs on the disc.

Unfortunately, your success here will be limited by two factors:

• Some software requires writing to the startup disc in order to function. As you cannot write to a CD or DVD, this can be a deal-breaker. Apple provides a potential solution to this problem: it creates a RAM disk as part of the startup process. Files can be written here instead of to the disc itself. However, this does not appear to be accessible for applications launched by the user after startup is over.
  
  
• Some applications require accessing additional software in order to function. This may include software in the Frameworks and PrivateFrameworks folder of the /System/Library. It may also include software that the program expects to find (or create) in the Library folder of a user's account. This software is often not found in the subset of Mac OS X files included on a boot CD/DVD.

The overall result is that many programs will fail to launch from a boot disc. Even if they do launch successfully, they may not function properly (menu commands may not work, for example). In the case of the Console application, it launches fine but there are no logs on the startup disc to examine.

If you prefer to avoid Terminal, there are other solutions, but they require working with third party software:

Clone X 2. Although Clone X is known primarily as backup software, a new feature was added to it in 2.0.1: the ability to create a custom bootable CD. It does this by using the files on your current startup drive as the source. This should guarantee that the disc will work successfully on your Mac. It should result in greater success in running applications from the disc than via the Terminal route.

When you first setup to burn the CD, you are additionally given the chance to select what applications you want to run. These are copied to the CD. The resulting bootable CD launches an application launcher utility (included as part of Clone X) on startup. Whatever software you selected appears in the launcher. Cool. I tried it out and it worked perfectly.

TechTool Protege. Micromat's TechTool Protege is a bootable FireWire flash drive. Similar to how Clone X's bootable CD works, Protege goes directly to an application launcher on startup. Again, you can select in advance the utilities you want to be accessible from the launcher. Micromat's TechTool Pro and DiskStudio software are included by default.

BootCD. BootCD appears to be a near perfect solution. It creates a bootable CD that, amazingly, even manages to squeeze in a functioning Finder. Unfortunately, the current version of BootCD does not work with Tiger and, from the looks of things, I suspect it will never be updated to work in Tiger or any future OS X version. If you are still using Panther, however, it's worth a look.

None of these solutions creates a single universal dual-platform disc. A disc created by Clone X will boot whatever platform was used to create the disc, Intel or PowerPC, but not both. TechTool Protege comes with software that can be used to create either an Intel or PowerPC version of the drive, but you cannot create a single version that works on both platforms. Most people predict that Leopard will resolve this issue, allowing for universal boot discs. However, while we're waiting, a few developers have come up with solutions that already work in Tiger:

TechTool Pro. Micromat's TechTool Pro ships on a single DVD that contains two separately mounting partitions. One partition contains PowerPC software and the other Intel software. When you boot from the disc, your Mac automatically locates the bootable partition for that platform and boots from it, launching the TechTool Pro application.

Drive Genius. The people at Prosoft managed to perform an even cooler trick with their Drive Genius bootable disc. It is a CD (not a DVD) and it contains a single universal bootable system on it, successfully starting up on either a PowerPC or an Intel Mac. After booting, it launches the Drive Genius software. I suspect that even the folks at Apple could learn a thing or two from this disc.

Apple's position. Speaking of Apple, a question may be occurring to you by now: Given that it's obviously possible to create a customizable boot disc, and even make it dual-platform, why doesn't Apple provide the tools to allow end users to do this? Why the almost complete absence of even a mention of this subject in Apple's Knowledge Base documents?

I have asked these questions to several informed sources (outside of Apple) and have still not gotten a clear answer. The all seem to agree that it has something to do with Apple wanting to prevent piracy of Mac OS X or otherwise protect its copyright. I point out that there were no similar restrictions in Mac OS 9, and there did not seem to be any great ill effect. I go on to point out that the availability of BootCD (for Jaguar and Panther) did not lead to any problems that I was aware. So what's the big concern exactly? Can't Apple at least provide a way for end-users to do what can now be done using Clone X, so they could create their own custom emergency discs without needing third-party software? This would not conflict with the need for legal restrictions on those making a commercial product. At this point, the usual reply is a shrug and a desire to change the subject.

But what about the developers, such as Micromat and Prosoft, that produce bootable discs? How do they do it? Don't they get some help from Apple? Yes, Apple does give them support. But you would have to jump through several hoops before you could get it. First, you need to be an dues-paying Apple Developer. Second, you need to get a license to use Apple's boot disc technology, plus pay an annual fee to keep the license. Although there is an Apple Developer page that provides information regarding software licensing, it makes no mention of licensing bootable disc technology. You have to know to ask. And, if you do ask, Apple will tell you that, before they even consider giving you a license, you need to show them your proposed software in order to demonstrate why it requires a bootable disc. Even after all of that, you don't get the tools to put a Finder on a disc (as could be done with BootCD). You are restricted to the more limiting technology of having a single alternate utility launch at startup.

To be continued. Clearly (and sadly), Apple does not want to assist end users (or even most developers) in creating custom bootable optical discs. They certainly weren't going to provide any help to me. If I wanted to learn more about how all of this works, I would have to look elsewhere. So I did. Within some limits, I eventually learned how to create my own custom bootable CDs and DVDs, without using any third-party software. I'll tell you all about it in the conclusion of this topic, coming in next month's column.

This is the latest in a series of mac.column.ted columns by Ted Landau. To see a list of previous columns, click here. To send comments regarding this column directly to Ted, click here. To get Ted's latest book, Mac OS X Help Line, click here.

Resources