This behavior can occur if the desktop heap in the WIN32 subsystem is depleted. To correct this problem, increase the size of the desktop heap:
1. Run Registry Editor (Regedt32.exe).
2. From the HKEY_LOCAL_MACHINE subtree, go to the following key: \System\CurrentControlSet\Control\Session Manager\SubSystems
3. Select the Windows value.
4. Edit the value. Decrease the last number of the "SharedSection=" value to 256 or less (it should not be less than 128). In other words change SharedSection=1024, 3072, 512 to SharedSection=1024, 3072, 256.
The default data for this registry value will look something like the following (all on one line):
%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024, 3072, 512 Windows=On SubSystemType=Windows ServerDll=basesrv, 1 ServerDll=winsrv:UserServerDllInitialization, 3 ServerDll=winsrv:ConServerDllInitialization, 2 ProfileControl=Off MaxRequestThreads=16
For more information, see the following Microsoft technical notes:
ME184802, and
ME126962.