ALERT: ESX 4.0 to 4.0 Update 1 can cause an Unrecoverable system.

November 25, 2009

I just saw this come across twitter. (BTW: This is a GREAT example of why Twitter is a valuable technology tool if used correctly.)

Upgrading ESX 4.0 to Update 1 using Update Manager when there are 3^rd party management agents in the Console will result in an UNRECOVERABLE system.

Here is the official word from the KB as of now (11/25/2009 : 5 pm EST)

Upgrading ESX 4.0 to 4.0 U1 using Update Manager fails or times out and rebooting host results in a purple diagnostic screen

Symptoms

When attempting to upgrade ESX 4.0 to ESX 4.0 Update 1, you may experience the following symptoms:

Update Manager (VUM) upgrade operation fails or times out at 33%.
Upon reboot, the host that was being upgraded displays a purple diagnostic screen with the following error:
COS PaniC:\ Int3 @ mp_register_ioapic

Who is affected
Customers using VMware vSphere 4 upgrading to Update 1 with 3^rd party management agents running.

Solution

To avoid this issue, prior to the update, disable all 3rd party management agents running on the ESX 4.0.0 server before applying the update.

Note: The 3rd party management agents can be enabled after the upgrade is completed.

WARNING: Rebooting the host means the host has to be reinstalled because it is not recoverable after a reboot.

WARNING: If you have virtual machines running on local storage, they may not be retained if you reinstall ESX 4.0 as a result of this issue. Contact VMware Support for assistance in recovering those virtual machines.

The official KB is here : https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1016070 and is being updated as information is released; I would imagine a patch link will appear there when it is ready.