After multiple times of trying to power on the OS X VM, each resulting in the ESXi host's crash, I decided to remove ALLdevices enabled in ESXi for Direct Path I/O and only after I restarted ESXi 5.1, was I able to power up the OS X VM without it crashing the ESXi host running on Mac Mini 6,2.
However, the farthest I could get was the login prompt in OS X (via ESXi's VM console). Once I supplied the password for the account, the OS X showed the spinning wheel and the spinning beach ball. I left it like this for 1.5 hours, and finally decided to repair the OS X installation virtual disk using Disk Utilities from the Installation DVD image.
In order to boot from the image of the installation DVD (InstallESD.dmg), I had to boot the VM into EFI and change the boot order. After booted from InstallESD.dmg, I ran the check on the OS X virtual volume, but found no errors. Then I ran the permission repair, and many permissions were out of whack. After that repair completed, I decided to go ahead and reinstall OS X Mountain Lion from the DVD image. This was not a clean install because I did not reformat the virtual volume on which OS X Mountain Lion (not working) was installed - my hope was to preserve all of the user data on this volume. After the reinstallation of the Mountain Lion completed, I was able to log in to the Mountain Lion (finally!) and then enabled Desktop Sharing via the ESXi's VM console (which had been enabled before the first ESXi crash, but was disabled now). At this point, I was able to remote into the VM, using Screen Sharing.
I found out that the Open Directory master that I had enabled (before my fiasco with Direct Path I/O for Bluetooth) was no longer enabled, and I had to recreate the master. It failed after the first attempt, but worked on the second attempt.
Even though file sharing was already enabled, I had to disable file sharing over AFP and SMB and then re-enable them in order for my clients to be able to mount shares on the OS X Mountain Lion running in the VM on the Mac Mini 6,2. At this point, I was able to gain access to my files, which seem to be all intact.
So, this was a very educational experience with the following conclusions:
1. Always back up your files before attempting anything crazy like using non-working features on unsupported hardware, which is exactly what it was with using a USB Controller Direct Path I/O passed through to an OS X Mountain Lion VM running on Mac Mini 6,2 under ESXi 5.1.
2. When people report that a feature is broken, believe them. I was apprehensive about enabling one (or both) USB Controller for Direct Path I/O, but for some reason, when I learned that one of these controllers was responsible for Bluetooth, I got so excited that I forgot about the warning.
3. Before enabling a major feature, such as Direct Path I/O, always take a snapshot of the VM. In this case, I would have had to power down the VM and take a snapshot (with all of my valuable files already having been copied to this VM), but I failed to do so. Had I done so, I would have been able to restore this VM in a few minutes instead of 7+ hours I spent on this.
4. Think logically, go slow, and do not give up.
Now, I have a couple questions for the VMware guys reading this.
1. It's obvious now that USB controllers are not reliable (at least in ESXi 5.1) when enabled for Direct Path I/O. Is this something that's going to be corrected in the next release of ESXi 5.1?
2. After I tried to pair my OS X VM with a Bluetooth keyboard, which crashed my ESXi, the USB controller responsible for Bluetooth and IR Receiver was no longer enabled for Direct Path I/O upon the hard restart of the ESXi host. However, the VM kept crashing the ESXi host when I tried to power the ESXi host. When I tried to re-enable this USB controller for Direct Path I/O and restart the ESXi, Direct Path I/O would become disabled again upon the ESXi host's graceful reboot. Question, why was Direct Path I/O no longer working for this USB controller, and why did the OS X VM continue to crash the ESXi host?
3. The only way I was able to stop the OS X VM from crashing the ESXi host was by disabling Direct Path I/O on ALL other devices that had been working prior to the Bluetooth fiasco. These devices are: Audio, Wi-Fi, and FireWire. Question, why was this step necessary? Is there a bug with other devices (besides USB) being enabled for Direct Path I/O in ESXi 5.1 running on Mac Mini 6,2?
4. I'm not going to attempt to enable any devices for Direct Path I/O until the next release of ESXi 5.1. So, are you guys aware of the issues that I encountered and are the fixes being worked on? Obviously, without using Direct Path I/O, OS X running in a VM can only be used as a headless server. However, with being able to enable video, audio, wi-fi, FireWire, Bluetooth, and IR Receiver for Direct Path I/O, an OS X VM could be used as a client while being able to run other servers concurrently on the same hardware. So, is this something that's being given proper attention to make sure these physical devices can be passed through to an OS X VM reliably and not result in the ESXi host's crashes like what I've experienced?
5. Will the next release of ESXi 5.1 incoporate the SMC fix so that it can be run on the Mac Mini 6,2 without any non-official modifications of the installation image?
Thank you everyone for all of your work!