The Wix 3 installer version of our product successfully executed a batch file using the following configuration:

<SetProperty Id="RunBatch" Value=""[SetupDir]batch.bat""
    Before="RunBatch" Sequence="execute" />
<CustomAction Id="RunBatch" Execute="deferred" 
    Return="check" Impersonate="no" BinaryKey="WixCA" DllEntry="WixQuietExec64" />

<InstallExecuteSequence>
    <Custom Action="RunBatch" Before="RunDBSetup" >
        NOT Installed AND NOT REMOVE
    </Custom>
</InstallExecuteSequence>

After converting the wxs file to Wix 4 (the schema used by Wix 5), the above elements were converted to:

<SetProperty Id="RunBatch" Value=""[SetupDir]batch.bat""
    Before="RunBatch" Sequence="execute" />
<CustomAction Id="RunBatch" Execute="deferred"
    Return="check" Impersonate="no" DllEntry="WixQuietExec" BinaryRef="Wix4UtilCA_$(sys.BUILDARCHSHORT)" />

<InstallExecuteSequence>
    <Custom Action="RunBatch" Before="RunDBSetup" Condition="NOT Installed AND NOT REMOVE" /></InstallExecuteSequence>

After building the wxs file we run msiexec and get the following error for this custom action:

MSI (s) (5C:E8) [19:05:48:239]: Executing op: ActionStart(Name=RunBatch,,)
MSI (s) (5C:E8) [19:05:48:242]: Executing op: CustomActionSchedule(Action=RunBatch,ActionType=3073,Source=BinaryData,Target=WixQuietExec,CustomActionData="C:Program FilesCompanyProductsetupbatch.bat")
MSI (s) (5C:E8) [19:05:48:244]: Creating MSIHANDLE (165) of type 790536 for thread 9192
MSI (s) (5C:3C) [19:05:48:246]: Invoking remote custom action. DLL: C:WINDOWSInstallerMSI5719.tmp, Entrypoint: WixQuietExec
MSI (s) (5C!1C) [19:05:48:252]: Creating MSIHANDLE (166) of type 790531 for thread 15644
WixQuietExec:  Error 0x80070002: Command failed to execute.
MSI (s) (5C!1C) [19:05:48:253]: Closing MSIHANDLE (166) of type 790531 for thread 15644
MSI (s) (5C!1C) [19:05:48:253]: Creating MSIHANDLE (167) of type 790531 for thread 15644
WixQuietExec:  Error 0x80070002: QuietExec Failed
MSI (s) (5C!1C) [19:05:48:254]: Closing MSIHANDLE (167) of type 790531 for thread 15644
MSI (s) (5C!1C) [19:05:48:254]: Creating MSIHANDLE (168) of type 790531 for thread 15644
WixQuietExec:  Error 0x80070002: Failed in ExecCommon method
MSI (s) (5C!1C) [19:05:48:255]: Closing MSIHANDLE (168) of type 790531 for thread 15644
CustomAction RunBatch returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

We later tried the following changes with no luck:

• Confirmed that the batch file runs without error when called directly from cmd line
• Run the install file with a modified “empty” version of the batch file with only an echo statement. Also tried terminating the batch file with a call to exit 0
• Restrict to using only X86 and X64 versions of Wix4UtilCA_
• Set Impersonate=”yes”
• Running installer from powershell with admin priviledges

We are expecting that calling the batch file will work without errors as it did with Wix 3.

This is all running on Windows 11.

Any ideas on what might be causing this error? Thank You so much!