64 bit Musings; aka Writing scripts in a 64 bit Windows Environment.
I’m sure most of us will or have been installing 64 bit operating systems more and more so when I saw this note from my colleague Jacques Bensimon about some of the differences between 32 bit and 64 bit Operating Systems behaviors, I thought I’d repost for everyone. JB refers to this as musings; I think it is closer to a white paper! 😉
The purpose of what follows is (A) to describe the system behaviors experienced by 32-bit programs or scripts running on a 64-bit platform and (B) to describe some techniques whereby your own 32-bit programs or scripts (e.g. KiXtart or AutoHotkey programs) can be made “64-bit aware” (i.e. bypass some of the automatic “redirections” that a 64-bit OS imposes on 32-bit programs).The information is based on some reading and some experimentation on 64-bit Windows Server 2008, Windows 7 and Windows Server 2008 R2, so it may not fully apply to prior 64-bit environments (e.g. XP and 2003 – I don’t care, do you?!)
(A) System behaviors: So the first thing to note here is that a run-of-the-mill 32-bit program that neither knows about nor cares about 64-bit environments is automatically presented a “32-bit view” of a 64-bit environment and should install and operate without issues (unless it for example tries to install a 32-bit driver or something of similar “depth”). What is this “32-bit view”? I see three main areas:
1. Environment Variables: The standard environment variables that are automatically modified when presented to a 32-bit app (from now on, it will be understood that we’re talking about a 64-bit OS) are typically
PROCESSOR_ARCHITECTURE=x86 (instead of AMD64 or IA64)
ProgramFiles=C:\Program Files (x86)\ (instead of C:\Program Files\)
CommonProgramFiles=C:\Program Files (x86)\Common Files\ (instead of C:\Program Files\Common Files\)
The last two are very mild and hardly enforced attempts at convincing 32-bit setup programs to put their program folders in the 32-bit (i.e. …(x86)) Program Files folder. Nothing actually prevents a 32-bit program from placing (32- or 64-bit) files in the “C:\Program Files\” folder and such files can be accessed and executed by 32-bit programs without problems, i.e. there is no automatic redirection to “C:\Program Files (x86)\”. Notice that the all-important PATH environment variable is not on this list, so 32- and 64-bit programs get the same search path, but as you’ll see it is treated differently in some respects because of the file system redirection I’ll describe next.
2. File System: With one important exception, a 32-bit program can access any folder in the file system directly by its real name, including C:\Windows\ (aka %SystemRoot%\). The exception is C:\Windows\System32\ (and most of its subfolders): any attempt by a 32-bit program to access this folder directly by name is in most cases automatically and transparently redirected to C:\Windows\syswow64\ (the “32-bit System32” folder – I know, the names are unintuitive, but it’s all about backward compatibility). This folder is populated initially by Windows during setup to contain 32-bit versions of most system programs and DLLs, and it continues to be populated with 32-bit components whenever a 32-bit setup program attempts to write to C:\Windows\System32\ (by virtue of the aforementioned automatic redirection). Of course, 64-bit aware setup programs can also purposefully place 32-bit EXEs, DLLs, OCXs, etc in SysWOW64 (maybe along with 64-bit versions in the “real” System32). By the way, there is no automatic redirection that applies to 64-bit programs: if a 64-bit process accesses SysWOW64, it is assumed that it knows what it’s doing (since it was written for a 64-bit environment) and it is not redirected back to System32. [Not that it matters much, but 32-bit processes can also access SysWOW64 directly by name (and can see it as a subfolder of Windows) – it will be identical to accessing System32].
Why did I say in most cases above? Certain subfolders of System32 are exempt from redirection, most notably drivers\etc (so everybody sees the same hosts, lmhosts, etc) and spool (so only one set of print drivers, print jobs, etc). The is also one (very special and unique) additional file system redirection that Windows imposes on 32-bit processes aside from System32/SysWOW64: %SystemRoot%\regedit.exe (64 bit program) is redirected to %SystemRoot%\SysWOW64\regedit.exe (the 32-bit version) – wouldn’t have been necessary if somebody hadn’t screwed up way back when in Windows history and had instead put regedit.exe in System32 along with all the other system tools — see workaround later in section (B).
The automatic System32 redirection explains why, even though they see the same PATH variable (which as always starts by default with %SystemRoot%\\system32\), 32-bit and 64-bit processes will in fact be accessing different executables, DLLs, etc whenever these components are found in one or both of the two “System32” folders. So, for example, if you’re running a 32-bit process (for example an AutoHotkey script) and launch “notepad.exe” or “CMD.exe”, you’ll launching the 32-bit versions of these programs. However, as you’ll see in section (B), it is very easy for a 32-bit process to access the real (64-bit) System32 if it needs to, and you can do this in your scripts and programs without having to resort to fancy API calls. ß Tease to keep you reading! J
Before ending this section, let me state explicitly what I already did implicitly when saying that the System32/SysWOW64 automatic file system redirection is the only one that a 64-bit OS imposes on 32-bit processes: nothing in a user profile folder structure is subject to automatic redirection, so 32-bit and 64-bit processes see the exact same profile folders (Documents, AppData, Cookies, Favorites, Desktop, TEMP folder, etc., etc.). Of course, 32- and 64-bit versions of the same programs are free if they wish to save their settings (or anything else they want) in different areas of the profile folders, but there’s nothing automatic about it. Just thought I’d point it out since we [Citrix Consultants] live and die with user profiles.
3. Registry: With two important exceptions, 32-bit processes can access all parts of the Registry directly and will see the same thing as 64-bit processes. The exceptions are most parts of HKLM\Software\ and some parts of HKCU\Software\Classes\: special cases aside (see below), attempts by 32-bit processes to access the affected subkeys of these keys “in a normal way” (i.e. without using special 64-bit aware API calls) will be redirected to their “32-bit subkeys”, HKLM\Software\Wow64\32\Node and HKCU\Software\Classes\Wow64\32\Node respectively. Exempt from redirection are, among others, HKLM\Software\Classes (but not HKLM\Software\Classes\CLSID and …TypeLib), HKLM\Software\Clients\, HKLM\Software\Policies, many parts of HKCU\Software\Classes\ and a bunch of other special cases that make sense when you think about them individually: you want everybody (by which I mean “all apps”) to agree on file associations, you want everybody to agree on what the default e-mail client and browser are, machine policies should apply to all in the same way (but on the other hand you want 32-bit programs to have their own 32-bit component registrations, etc). I found a decent listing of the exceptions at https://msdn.microsoft.com/en-us/library/aa384253(v=VS.85).aspx.
As you’ll see in section (B), unlike the File System redirection, there’s no easy way for a 32-bit program or script (by “easy” I mean “without use of API calls”) to override the redirection and, for example, see all parts of the “real” HKLM\Software\.
And once again to stress the obvious, there’s no redirection applied to the “non-Classes” part of HKCU, including HKCU\Software, so the user profile is again the same regardless of whether the app looking at it is 32-bit or 64-bit.
(B) Now for techniques, you can use in your own programs and scripts to make them 64-bit aware and “see” everything as it “really is”. First for the obvious solution: write 64-bit programs! For example, batch files that run under the 64-bit CMD.exe (the default behavior when launched from Explorer) and VBScript/JavaScript programs running under the 64-bit CScript.exe and WScript.exe are naturally 64-bit programs and, providing they don’t invoke 32-bit utilities under the covers, will automatically see everything. Be careful however if you launch these scripts from 32-bit processes (e.g. an AutoHotkey program or a KiXtart script), because if you don’t take the special steps described below, you’ll be launching the 32-bit versions of CMD, CScript and WScript and will be subject to the OS’s “funny business”. OK, back to 32-bit techniques:
1. Identification: For a 32-bit program to be 64-bit-aware, it first needs to be able to identify the fact that it is running on a 64-bit platform in order to take appropriate measures where necessary. This is made very easy by the 64-bit operating system automatically providing 32-bit processes with the PROCESSOR_ARCHITEW6432 environment variable (typically set to AMD64 – by the way, this doesn’t mean you’re necessarily using an AMD processor, only that you’re using AMD’s flavor of 64-bit processing, now the most common standard even with Intel processors except for the IA64 line which for some reason continues to be supported by Windows). If a 32-bit program retrieves an empty value for this variable, then it’s running on a 32-bit platform, otherwise, it’s running on a 64-bit platform. (I’ll ignore the possibility of a perverse joke being played on a program by creating this variable on a 32-bit platform!)
2. File System access: This is too easy: (a) the automatic redirection of %SystemRoot%\system32\ to %SystemRoot%\SysWOW64\ is easily circumvented by instead accessing the “pseudo-folder” %SystemRoot%\SysNative\ – this is not a junction or a hard link and it won’t appear in any directory listings, it’s just something that the file system understands to mean the “real” System32, so if you want to launch the 64-bit version of CMD.exe or Reg.exe from a 32-bit program, use the full path %SystemRoot%\SysNative\CMD.exe or %SystemRoot%\SysNative\Reg.exe and Bob’s your uncle! J (By the way, if you’re writing in a “real” language capable of API calls, there’s also a Wow64DisableWow64FsRedirection API call that can be used to turn off automatic redirection for a given thread, but it can be dangerous if the thread tries to load a new System32 DLL while under its influence) and (b) the presentation of the %ProgramFiles% and %CommonProgramFiles% environment variables as their “(x86)” versions can be ignored by using instead the %ProgramW6432% and %CommonProgramW6432% variables when wishing to operate on the 64-bit folders.
One more note: as mentioned earlier, the file Regedit.exe is a special case because the 64-bit version doesn’t live under System32, so there’s no %SystemRoot%\SysNative\regedit.exe. If you want to launch the 64-bit Regedit from a 32-bit program (and don’t want to make an extra copy of it somewhere else for easy access), I found that you can instead launch %SystemRoot%\SysNative\RegEdt32.exe since, as you know, RegEdt32.exe is now just a stub that passes on to Regedit.exe (and I’ve verified that it will pass on any command line parameters you specified).
Which brings us to
3. Registry access: Here, unfortunately, there’s no easy answer to the redirection of HKLM\Software\ to HKLM\Software\Wow64\32\Node etc, and this time there’s no “magic pseudo-key” available (HKLMSoftNative ?) nor any API call to temporarily disable this redirection. The official Microsoft answer is to use the KEY_WOW64_64KEY flag with the RegCreateKeyEx, RegDeleteKeyEx and RegOpenKeyEx API functions, but that’s only useful if you’re writing in a “real language”. So the answer here is either to use a 64-bit language (batch, VBScript, AutoHotKey_L 64-bit, etc) or to invoke the 64-bit versions of Reg.exe or Regedit.exe from your 32-bit program, depending on what you’re trying to do.