VAAI in vSphere 4.1 COULD break your RecoverPoint consistency groups!

January 25, 2011

Jeff Miller sent over this warning for environments leveraging vSphere 4.1 and RecoverPoint. Here is all the technical gooeyness.
If you are not running FLARE R30 Q410 type 2 patch 4.30.000.5.509, keep on reading. (This issue is corrected in 509)

We recently had a problem where our Exchange Consistency groups in RecoverPoint were all stuck at initializing 0% for several days. This seemed to occur out of the blue after running for over a year. I tried to force a re-sweep, I tried to rebuild the consistency groups, however it was still stuck at 0% initialized. After some urgent calls to EMC support, they advised me to disable VAAI in VMware (All ESX Host servers). vSphere 4.1 enables VAAI by default.

At the time the environment consisted of the following components and versions :
(vSphere 4.1 , Flare Code R30 4.30.000.5.507, RecoverPoint 3.3 SP1 & Clariion Splitters)

To disable VAAI commands:
1. In the vSphere client inventory panel, select the host.
2. Click the Configuration tab and from the Software menu, select Advanced Settings.
3. To disable Hardware-Assisted Locking, click VMFS3, and set the value of VMFS3.HardwareAcceleratedLocking to 0.
4. To disable Full Copy, click DataMover, and set the value of DataMover.AcceleratedMove to 0.
5. To disable BlockZeroing, click DataMover, and set the value of DataMover.AcceleratedInit to 0.
6. To save the changes, click OK.
7. Repeat across all Host ESX servers in your RecoverPoint environment.

Refer to the EMC Primus Article emc255099 for more information on VAAI.
Obviously, the other option is to upgrade your FLARE Code.
-Jeff, thanks for the tip!

Scott Lowe also posted some information regarding this issue:
https://blog.scottlowe.org/2011/01/20/additional-recoverpoint-vaai-update/