In our case the problem appeared because the file msi.dll was missing. When we tried to install software directly from an install CD this event was generated. Copying the CD to the local hard drive and attempting to install from the local hard drive generated the same event. Attempting to run Setup.exe directly returned "Windows cannot access the specified device, path, or file. You may not have the appropriate permissions to access the item". Additionally, we were unable to open the Add or Remove Programs applet. Finally, we ran msiexec /unreg and received the following error message: "This application has failed to start because msi.dll was not found. Reinstalling the application may fix this problem". We copied msi.dll from another server and successfully unregistered and reregistered msiexec.
From a newsgroup post: "The problem with reading shared files disappeared when the SATA RAID controller was moved to another PCI-X slot in the machine. I guess that in the previous configuration there was a subtle hardware or driver conflict between the RAID controller and the integrated NIC. It is very strange that Windows Server 2003 did not issue any warnings or errors regarding this".