Once the GT28 nodes are BIOS-updated to the AGESA v188.8.131.52+ release, a few adjustments are needed to to support my boot-from-flash deployment model. If you are not familiar with boot-from-USB-flash, there are many helpful blogs issued on the subject like this one from vm-help.com. Suffice to say, boot-from-USB-flash for ESXi in a relatively simple process to setup:
- Make sure your BIOS supports boot-from-USB-flash;
- Download the latest release of ESX 3i from VMware;
- Mount the ISO image of the 3i installer;
- Find the “VMvisor-big” image as a “.dd.bz2” file in the mounted image;
- Un-bzip the VMvisor-big image to a temporary directory;
- Plug-in your “donor” USB flash device (I’m using the Sandisk Cruzer 4GB);
- Find the device handle of the mounted USB device and unmount it (i.e. “umount /dev/sdm”;
- Use dd to copy the VMvisor image to the flash device (i.e. “dd if=/tmp/VMware-VMvisor-big-3.5.0_Update_3-123629.i386.dd of=/dev/sdm”);
- Eject the USB device and label it as ESXi;
- Insert the USB flash device into a USB 2.0 port on your equipment and boot;
Preparing the BIOS
To prepare my GT28 for ESX 3i and boot-from-USB-flash, I insert the USB “thumb drive” into one of the rear ports and turn-on the GT28. Hitting the “delete” key on boot gets me to the BIOS setup. I will start with the BIOS “Optimal Defaults”, and make modifications from there; these adjustments are (follow links for screen shots):
- Reset BIOS to “Optimal Defaults”;
- Adjust Northbridge IOMMU window from 128MB to 256MB;
- Disable AMD PowerNow in BIOS;
- Adjust PCI Latency Timer from 64 to 128 (optional);
- Disable nVidia MCP55 SATA controller (ESXi has no driver ESXi has a driver, however may be issues with nVRAID);
- Adjust USB Mass Storage, setting the USB flash drive to Hard Disk;
- Disable the CD/DVD boot devices to avoid boot conflicts;
- Select the USB flash drive as the only boot device;
- Finally, save the BIOS changes and reboot;
- Now, the system should boot into ESXi for initial configuration;
As you can see, boot-from-USB-flash is “wicked simple” to implement (at least on this platform) and open-up all kinds of testing scenarios. In this case, the ESXi image is now running from USB flash, and only the basic configuration tasks remain. However, it is a good idea to know which Ethernet ports are which on the rear panel of the GT28.
If the PCI bus scan order is configured for “Ascent” the LAN ports will be configured as indicated in the image shown. If you modify the bus scan for “Descent” (i.e. to accommodate a RAID controller) then E2/E3 becomes E0/E1 and E0/E1 becomes E2/E3 due to the new initialization sequence. You may want to, therefore, be cautious when making such a change since ESXi will re-enumerate the interfaces (although any used interface will be pinned to the MAC address.)
Initial Configuration of ESXi
Once your network connections are plugged-in, you should have already mapped-out some IP assignments and VLAN and/or trunking arrangements. While these steps are not strictly necessary in testing, they are a good practice maintain even in testing. To make the initial configurations to ESXi, from the console do the following:
- Hit “F2” to enter the configuration screen;
- Set the “root” password for the ESXi server;
- Skip “Lockdown mode” for now;
- Configure the management network of the ESXi server;
- Select the network adapter(s) to be used for management;
- If not using DHCP:
- Fix the management IP address
- Fix the management IP Subnet mask
- Fix the management IP Default gateway
- Fix the management DNS configuration;
- Update the DNS suffix(es) for your local network;
- Hit “Enter” to save (“Escape” exits without change);
- Test the management network and restart if necessary;
- Exit the configuration menu and hit “F12” to restart;
Initial Management with VI Client
Once the ESXi server is up and online, you will need to grab the VMware Infrastructure Client from the web service provided by the ESXi server (http://<ESXi_IP_ADDRESS>/) and install it on your Windows client. If you don’t run windows (like me) you should have a version running in a VM for just such an occasion. I find VirtuaBox to be a better (free) choice for workstation-on-workstation applications and VMware Server a good choice if the client is to be minimal and accessible from multiple hosts.
Once the VI Client is installed, run the application and enter the ESXi server’s hostname/IP-address, root username and root password where requested. A VI Client window will eventually open and allow you to complete the setup of the ESXi server as needed.
That’s really all there is to it: we have a reliable, running ESXi platform in mere minutes with minimal effort.
Updated January 15, 2009. Corrected statement that ESX 3i update 3 is not MCP55 aware – support has been added in ESX 3i update 2 and newer. In my test configuration (with the SATA controller enabled) ESX 3i update 3 does properly identify and configure the MCP55 SATA Controller as an abstracted SCSI controller.
# vmkvsitools lspci 00:00.00 Memory controller: nVidia Corporation 00:01.00 Bridge: nVidia Corporation 00:01.01 Serial bus controller: nVidia Corporation 00:02.00 Serial bus controller: nVidia Corporation 00:02.01 Serial bus controller: nVidia Corporation 00:05.00 Mass storage controller: nVidia Corporation MCP55 SATA Controller [vmhba0] 00:06.00 Bridge: nVidia Corporation 00:10.00 Bridge: nVidia Corporation 00:11.00 Bridge: nVidia Corporation 00:12.00 Bridge: nVidia Corporation 00:13.00 Bridge: nVidia Corporation 00:14.00 Bridge: nVidia Corporation 00:15.00 Bridge: nVidia Corporation 00:24.00 Bridge: Advanced Micro Devices [AMD] 00:24.01 Bridge: Advanced Micro Devices [AMD] 00:24.02 Bridge: Advanced Micro Devices [AMD] 00:24.03 Bridge: Advanced Micro Devices [AMD] 00:24.04 Bridge: Advanced Micro Devices [AMD] 01:05.00 Display controller: 04:00.00 Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic0] 04:00.01 Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic1] 05:00.00 Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic2] 05:00.01 Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic3]
This enables the use of the MCP55 SATA controller for flash drives at least. I will do further tests on this platform to determine the stability of the NVRAID component and its suitability for local storage (i.e. embedded VM, like virtual SAN/NAS) needs.