“Short changed” by Citrix Provisioning Services!
By Jacques Bensimon.
A couple of years ago, a client showed me a freshly P2V’ed Windows image on which Microsoft Office programs were no longer functioning correctly (as just one of many issues, one could for example open Outlook, but Inbox messages would not open when double-clicked). A quick Process Monitor trace session later, the cause of the issues became clear: a large number of Office components, both EXEs and DLLs, are registered via their “short” or “8.3” paths (Registry entries that start with something like “C:\PROGRA~2MICROS~2Office12…”), but somewhere during the P2V process the short name of the “Microsoft Office” folder had changed from its original “MICROS~2” (on the source disk) to “MICROS~1” (on the virtualized disk image), the net effect being that many registered components could no longer be located. Similar issues were observed with many other programs that had fallen victim to a similar folder short name change. I didn’t attach much importance to the issue at the time, figuring that it was the result of whatever (clearly flawed) process had been used for P2V [heck, the running joke at IPM is that I’m the old fuddy-duddy endlessly railing against wanton virtualization, so in my world view, these people were just asking for it, no? ;)] and I wound up cobbling together a custom script to restore all short names on the virtual disk to whatever they were on the source disk (which thankfully was still available for comparison).
I later ran into a similar issue at another client when a share hosting a number of network-based applications was migrated from one storage system to another (I believe RoboCopy was the main migration tool on that occasion) and an important application suddenly stopped working, again (as it turned out upon investigation) the result of its folder having acquired a new short name that no longer matched its registration on the various client computers. Fixed easily enough, so still not panicking.
What took the cake and recently set me on a course to expose and eradicate this class of issues was something that hit much closer to home: after reverse-imaging a Citrix Provisioning Services vDisk back to the physical disk of a XenApp server (using the standard Citrix BNImage tool, which I happen to prefer to the alternative XenConvert, an opinion that is now under serious review), it occurred to me to perform a quick spot check of the resulting file and folder short names versus those on the source disk (can you say DIR /X ?) and was horrified [too strong? You don’t know how seriously I take my images! :)] to discover a large number of differences.
According to this Citrix KB article entitled “When Capturing an Operating System using BNImage.exe, Microsoft Office 2010 Might not Function as Expected” (duh!), using XenConvert (which doubles as Citrix’s P2V tool in the XenServer arena) instead of BNImage avoids the issue. Be that as it may, now aware that most mass copy tools (including RoboCopy and the ubiquitous XCopy) suffer from this lack of any respect for existing 8.3 short names, I generalized and extended my old batch script into a full-blown ex post facto 8.3 name “compare & repair” utility, the attached Match83 (32-bit and 64-bit versions included – you cannot use the 32-bit version on a 64-bit OS). Running it against my reverse image repaired literally *thousands* of mismatches in both folder and file names, most of which I’m sure wouldn’t have done any harm, but many of which would most definitely have.
To conclude, a few notes:
(1) There is apparently no functionality in SMB for remotely modifying short names, so Match83 (and the Windows tool fsutil.exe that it uses under the covers to make short name changes using the syntax fsutil file setshortname <file-specification> <new-short-name>) must target items on a local NTFS-formatted drive.
(2) Why do short name mismatches occur in such large numbers during copy operations (using short-name-unaware copy tools)? Two main reasons:
a) The original short name of a file or folder was in most cases auto-generated by the file system during the creation of the item, so for example if the folder “Microsoft Silverlight” was created before “Microsoft Office” within the same parent folder, it would likely have acquired the short name “MICROS~1” while “Microsoft Office” acquired “MICROS~2”, but when the parent folder and all its contents are copied to a new location, the operation generally proceeds alphabetically, with the result that “Microsoft Office” is copied first and acquires “MICROS~1” while “Microsoft Silverlight” later acquires “MICROS~2” or even something else like “MICROS~5” if other “Microsoft …” folders have been added to the source since its original creation.
b) Some original short names were not auto-generated but rather were the result of an app installation (often MSI-based) by an installer program that explicitly specified particular short names for files and folders, names that bear no resemblance to what the file system would have auto-generated. For example, on the Windows 7 computer on which I’m writing this, I have a folder called “Microsoft Research” with the short name “MI4430~1” – no way that the auto-generated short name of a copy of that folder would ever match the original.
(3) It’s of passing interest to note that Microsoft is well aware of the issues that can arise from component registrations that use short names: starting with the Vista/2008 version of fsutil, the command fsutil 8dot3name scan [/s] [/v] <folder-path> became available to report all Registry entries that could potentially be invalidated by short name changes under the specified folder.
(4) Under some rare circumstances, Match83 might be unable to correct a particular item’s short name. On the assumption that no permissions issue is involved (you should of course run this tool as a full admin), the reason will be that a *new* item has been created on the target since the original copy operation and that its system-generated short name has “hijacked” what should have been another item’s short name. If you have reason to believe that the original item “needs” its original short name (unlikely), you can use the fsutil syntax provided above to manually swap their short names.
Happy hunting. (and New Year!)
Jacques.