Bootable Thumb Drive on Mac

Hello Saabye,

     To prevent you from thinking to yourself "get to the point, SpawnHappy," as you read my response, I'll let you know now that the trick is to have a bootloader tailored for EFI in a FILE, yes, unbelievably, in a FILE, located under /System/Library/CoreServices on the thumb drive named "boot.efi".
     However, saying the above is premature. It SHOULD be said AFTER explaining a traditional BIOS to operating system sequence of events, and then an EFI to operating system sequence of events so that anyone who reads this thread will know what the heck I'm talking about and what they are dealing with. Also, I don't know your level of understanding of booting, so it's best to explain the whole thing.
      Making things bootable is one of my hobbies. I keep trying to make this "How to Make Things Bootable" YouTube series, but never have the time.

       Hopefully I'm correct in thinking that all this background information will be helpful.

       On the motherboard is a chip that holds a small amount of special software. On the average user computer (not Macs) this software comprises a BIOS, which stands for "Basic Input/ Output System". The BIOS was designed in the Eighties. This software, the BIOS, has the overall purpose to get a bootloader loaded to ultimately get you booted into an operating system.
      When you turn on a computer with a BIOS, the processor immediately starts to execute the instructions on that BIOS chip. Now the BIOS is loaded.
      The BIOS looks at a separate chip where its settings are stored to determine how it is to operate.
      In its settings is its "boot priority list". It might be "First CD, second floppy, third IDE hard drive, fourth USB thumb drive." It is the order by which it looks for a bootloader. The first bootloader it finds, it loads. If you have CD "first" and hard drive "second" and you have both a bootable CD and a bootable hard drive, it will boot the CD not the hard drive because it found the bootloader on the CD first. If you have CD "first" and hard drive "second", don't have a bootable CD present, but do have a bootable hard drive, it will look for a bootable CD first, then move on to your hard drive and boot it. Thus you get a quicker boot if you go into your BIOS's settings and set the hard drive to be "first."
      A bootloader is a program that loads an operating system. Except for in nontraditional modern techniques, a bootloader CANNOT be a file. If it WAS a file on a partition of a certain filesystem, that would mean that for the BIOS to load it, the BIOS would have to be able to make sense of the filesystem (whether it be FAT32, NTFS, HFS+, etc) which would mean that it would need to have detailed instructions for making sense of every filesystem it would ever load a bootloader off of. No one would expect the simple BIOS, which has to fit on a very low capacity chip, to be able to interpret filesystems, find the bootloader file in the filesystem's table, then follow the table to all the clusters that make up the file, in the proper sequence, and pass those instructions onto the processor. Remember, instructions take up space, and the chip that holds the BIOS doesn't have much space.
     Instead, a bootloader is written into a certain sector, outside any filesystem (won't show up in the file browser, whether that be Finder, Nautilus, Explorer, or whatever), that the BIOS can just go to and read off to the processor. Little interpretation required.
     In an MBR-structured disk, the first sector (the first sector is the first 512 bytes of the drive, provided the sector size is set to 512 bytes, which it almost always is) of the whole hard drive (including thumb drives structured this way), is called the MBR, which is the Master Boot Record. In the MBR is the partition table, an area that says where each partition of the drive starts and ends. Also in the partition table are labels for the partitions. The partition table has four slots and cannot be expanded. Thus you can only have 4 primary partitions. You can make a primary partition that, instead of being formatted to a filesystem, is used as an area to put more partitions, and in the beginning of this primary partition is a second partition table with unlimited slots in which you can specify where secondary partitions start and end that are within the special primary partition for this purpose.
     Bear in mind that a sector is usually 512 bytes (half a kilobyte). The whole MBR is thus usually only 512 bytes. The partition table takes up only a small fraction of these 512 bytes. Much of the rest of these 512 bytes can hold a bootloader.
     So when the BIOS tries to boot a hard drive, it looks to the hard drive’s first sector for a bootloader it can read off to the processor.

      The first 63 sectors (including the MBR) of an MBR-structured hard drive, provided the geometry of the drive says that 63 sectors equals one track, the first 63 sectors form track0, the first track of the hard drive or thumb drive. Even though thumb drives don't physically have tracks, the thumb drive storage is cut up into logical tracks, cylinders, and heads.
       In an MBR-structured hard drive, no partition is allowed to reside in track0. So if the first sector is the MBR, and there's 62 other sectors, and no partition can use them, what goes there? It's up for grabs, really.
       Grub2, dare I say the best bootloader, has two parts. The first part goes in the MBR, and the rest overflows into the 62 sectors of "no man's land". The BIOS has the processor execute the instructions is finds in the MBR, which loads GRUB2. This first stage of GRUB2 then looks to the other 62 sectors to load the rest of itself.

       What if there is no bootloader in track0 (including the MBR) in an MBR structured hard drive? Does the BIOS give up there and move on? No. It looks in the partition table to see which, if any, partition has been marked as "active". If no partition is marked as active, it might move onto the next drive in the list depending on the BIOS. If there is an active partition, it looks to the first sector of the partition, which is reserved as the VBR, the volume boot record. No file or part of a file that a partition is holding is allowed to go in the reserved VBR. A bootloader can go in the VBR, which the BIOS can send to the processor to execute.

      Once a bootloader is loaded, it can either load another bootloader, or load an operating system. If you load GRUB2, you can tell it to load the VBR of another partition, loading that bootloader, even if the partition isn't marked as "active". Of course, there has to be a bootloader there.
      Note that since a secondary partition isn’t in the primary partition table, which is where the BIOS looks to see if a partition is bootable, the BIOS will not find and boot a secondary partition. I’ve even tried installing a bootloader to the first secotor of a secondary parition and having GRUB2 chainload that parition…didn’t work.

      On really slick trick is that you can copy a sector or even a whole track0 containing a bootloader into a file, and have GRUB2 load the bootloader out of the file. This is possible because there are modules you can load into GRUB2 that give it the ability to interpret filesystems. So if you want to load a bootloader from a file that is resident on a partition formatted as FAT32, you need to have the fat module loaded into GRUB2.
      And JSYK, a BIOS or a bootloader can leave a driver in memory for the bootloader or operating system it is going to load to use.

      And of course there's rules for determining which hard drive to check for a bootloader 1st, 2nd, 3rd etc. if more that one hard drive is present, and the BIOS is on "hard drives" as it’s going down the boot priority list. Same if you have more than one CD drive or more that one floppy drive or whatever. If there's more than one controller (thing that drives plug into), the BIOS goes to the first controller, and the first drive of that controller. Then the second drive of that controller. Then if there's no bootable drives in that controller, it moves onto the second controller and checks the first drive of the second controller, then the second drive of the second controller, and so on. It stops when it either runs out of drives or finds a bootloader.

     For the BIOS to check to see if a bootable CD is present, it looks to see if there is a CD, and if there is, it follows the El Torito boot specification to try and find a bootloader, which, again, is not a file on the CD, but rather is in a certain sector that can be found by following the El Torito boot specification. If it finds a bootloader, it loads it.

    If the BIOS is checking for a bootable floppy, it just looks to the first sector of the floppy to see if it contains a bootloader or not.

    Floppies do not have an MBR, partition table, or a "track0" in the sense mentioned above. They are just one partition. The first sector of a floppy diskette is the first sector of the partition on the floppy, and the last sector of the floppy diskette is the last sector of the partition on the floppy. Thus the first sector is a VBR, not an MBR.
    A thumb drive can be formatted as a "super floppy" meaning that instead of using MBR structuring and giving the thumb drive an MBR, partition table, and track0 "no man's land", you just put one partition on the thumb drive across the whole thing. The first sector of the super floppy is the first sector of its one and only partition. You can put a bootloader in this first sector. But you can't have more than one partition.
     I think structuring a thumb drive as a "super floppy" is pretty pointless if you can just structure it after the MBR standard, give it an MBR, a "no man's land" and a partition table, install GRUB2 to the track0, and chainload whatever bootloader you want to from GRUB2, or even better, directly load whatever operating system you want to boot from your thumb drive with GRUB2.

     There are some modern very sophisticated BIOSes somehow able to boot off a bootable network drive. It must be black magic or something. Don't know what's going on there.

     Also note that on bootup, you can hit F12 or whatever to go into the boot menu and pick what you want to boot into rather than letting go through the boot priority list.


    Ok, now we all understand booting and have the proper educational foundation to move on to how Macs boot.

    On the motherboard of your Mac is a chip that holds a small amount of special software. This software comprises an EFI, which stands for "Extended Firmware Interface". The EFI was designed by Intel to replace the “outdated” BIOS of the Eighties. This software, the EFI, has the overall purpose to get a bootloader loaded to ultimately get you booted into an operating system.
    When you turn on a computer with an EFI, the processor immediately starts to execute the instructions on that EFI chip. Now the EFI is loaded.
   The EFI looks at a separate chip where its settings are stored to determine how it is to operate. On a Mac, these settings are saved as NVRAM variables. NVRAM is “non-volatile” RAM separate from the RAM. Normal RAM loses everything in it when the power is cut. NVRAM keeps its data.
   To change the EFI’s settings, you actually boot into an operating system that runs a program that can read and write to the NVRAM. In contrast, to change the BIOSes settings, you hit F2 or whatever on bootup before a bootloader is found and go into a special interface provided by the BIOS to set the settings.
     In its settings is not a "boot priority list", but rather the path to a file containing a bootloader on a drive somewhere.
    Here we have a nontraditional modern technique, where a bootloader CAN be a file. That would mean that for the EFI to load it, the EFI would have to be able to make sense of the filesystem (whether it be FAT32, NTFS, HFS+, etc) which would mean that it would need to have detailed instructions for making sense of every filesystem it would ever load a bootloader off of.  It can interpret filesystems, find the bootloader file in the filesystem's table, then follow the table to all the clusters that make up the file, in the proper sequence, and pass those instructions onto the processor.
    On a Mac, unless you hold down “option”, the EFI boots the bootloader contained in the file that the NVRAM settings tell it to. If you hold down “option”, you get to pick what you want to boot into. This is similar to the option of hitting F12 or whatever during bootup on a BIOS machine, which brings up a boot menu that lets you pick what you want to boot into.

      The EFI looks for a file called “boot.efi” in the CoreServices folder in the Library folder in the System folder on all partitions of all drives that it finds if you hold down “option,” provided it can interpret the filesystem the file is on.

      Let’s take a look at a Mac OS X install DVD. As far as I know, it doesn’t have a sector with a bootloader outside of a filesystem that can just be read off to a processor. It does have a bootloader in a file at /System/Library/CoreService/boot.efi. When you hold down “option” on a Mac with the install DVD present, and select the DVD, it loads the boot.efi bootloader.
      So, literally all you would have to do to make a thumb drive bootable on a Mac that boots into the Mac OS X installer is copy all the files off the DVD and onto a partition on a thumb drive. Insert the thumb drive, hold down option, and it will see that boot.efi file on the thumb drive, and you can choose it and boot into the thumb drive Mac OS X installer.

     However, the Mac OS X install DVD has certain files in more than one location on the disc. Instead of having more than one copy of the same file, it uses a filesystem that can do symlinks, which are symbolic links, which are shortcuts to files that look like the actual file rather than a shortcut, to take up less space. So to copy the files off the DVD to a partition on a thumb drive, that partition must be formatted to a filesystem that can do symlinks.

    To make anything bootable on a BIOS machine, you have to install a bootloader to certain sectors, you can’t just copy files over.

    So you need an efi bootloader file that has a bootloader that can load what you want AND is compatible for EFI.

   GRUB2 can be compiled for EFI and you can obtain a grub2 efi file. GRUB2 can also mimic a BIOS to chainload a bootloader that is written for BIOS and not EFI.
    I followed some instructions on the internet one time about how to compile GRUB2 from source for EFI, and it didn’t work for me. I got to a GRUB2 command line (on my Macbook), and it didn’t see any drives. Clonezilla comes with GRUB2 for EFI already built, so I think I’ll try that build next.

     Know that Apple hasn’t released to the public, or even to graphics card driver writers, how to use the 3D abilities of a graphics card on their EFI. So if you do boot into something via an EFI bootloader off your thumb drive on your Mac, you won’t be able to use the 3D abilities of the graphics card (unless it’s Mac OS X). But if you have GRUB mimic a BIOS or use Apple’s BIOS compatibility layer for EFI, then you can use normal BIOS drivers for the graphics card and use the 3D abilities, but you get a slower bootup because you have to wait for the BIOS compatibility layer to load.
    I think there is a way you can dump the video BIOS or something with dd and have GRUB2 use those dumps and boot something via EFI without emulating a BIOS and still use the 3D capabilities of the graphics card on the Mac if you use those dumps.

     If you use GRUB2 for EFI, then you will need to make a GRUB2 configuration file. Again, you can use the one that comes with Clonezilla as an example (look at their grub.cfg).

     You could still install GRUB2 to track0 and have a GRUB2 for EFI efi file so that your thumb drive is bootable on both an EFI computer and a BIOS computer.

      You’re in luck because Linux supports running on top of EFI as well as BIOS.

    Hope it helps.

Cheers,
Jake