SourceDir property is incorrect for non-cached MsiPackage called from a bootstrapper

[AlexKubiesa]

Description

Which version of WiX are you building with?

Version 3.11.

Which version of Visual Studio are you building with (if any)?

Visual Studio 2019, Version 16.7.6

Which version of the WiX Toolset Visual Studio Extension are you building with (if any)?

Version 1.0.0.4

Which version of .NET are you building with?

If this is referring to the version installed on my development machine, then 4.8.

If the problem occurs when installing your packages built with WiX, what is the version of Windows the package is running on?

Windows 10 Pro, Version 1909, OS Build 18363.1256.

Describe the problem and the steps to reproduce it.

I am creating a bootstrapper and some MSI packages. Some of the MSIs need to install files over 2GB, so I cannot achieve this with a File element and end up needing to use CopyFile. The line looks like this (imagining "file2.txt" is a large database instead):
<CopyFile Id="CopyFile2" SourceProperty="SourceDir" SourceName="file2.txt" DestinationDirectory="INSTALLFOLDER" DestinationName="file2.txt" />

When I use CopyFile and run the MSI directly, the large files get copied fine. But, when I call this MSI from a bootstrapper, the files don't get copied.

Looking at the logs in %TEMP%, I noticed that the MSI is copied to the Package Cache before being executed and SourceDir is set to that location in the Package Cache, so I set Cache="no" on the MsiPackage. However, when I install from the bootstrapper again, the large files are still not copied! Looking at the log files again, I see that SourceDir still gets set to some location within the Package Cache, even though the MSI doesn't get cached there, and in fact the directory doesn't even exist.

Describe the behavior you expected and how it differed from the actual behavior.

I expected SourceDir to be set to the original location of the MSI, since that is the only relevant location in this case.

Reproduction steps

Here's a minimal solution structure that reproduces the problem.

Directory structure:
SetupProject1\
SetupProject1\Bootstrapper1\
SetupProject1\SetupProject1\
SetupProject1\SetupProject1.sln
SetupProject1\Bootstrapper1\Bootstrapper1.wixproj
SetupProject1\Bootstrapper1\Bundle.wxs
SetupProject1\SetupProject1\file1.txt
SetupProject1\SetupProject1\file2.txt
SetupProject1\SetupProject1\Product.wxs
SetupProject1\SetupProject1\SetupProject1.wixproj

The SetupProject1 project is an MSI installer project, while Bootstrapper1 is a bootstrapper. file1.txt is a "small file" which gets included in SetupProject1 as a File element. file2.txt is a "large file" which gets copied via CopyFile.

SetupProject1\SetupProject1\Product.wxs:

<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
	<Product Id="*" Name="SetupProject1" Language="1033" Version="1.0.0.0" Manufacturer="Placeholder" UpgradeCode="9a72dae4-71bd-49d6-80a3-9a56a2fc9be2">
		<Package InstallerVersion="200" Compressed="yes" InstallScope="perMachine" />

		<MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." />
		<MediaTemplate EmbedCab="yes" />

		<Feature Id="ProductFeature" Title="SetupProject1" Level="1">
			<ComponentGroupRef Id="ProductComponents" />
		</Feature>
	</Product>

	<Fragment>
		<Directory Id="TARGETDIR" Name="SourceDir">
			<Directory Id="ProgramFilesFolder">
				<Directory Id="INSTALLFOLDER" Name="SetupProject1" />
			</Directory>
		</Directory>
	</Fragment>

	<Fragment>
		<ComponentGroup Id="ProductComponents" Directory="INSTALLFOLDER">
			 <Component Id="ProductComponent">
				<File Id="File1" Source="file1.txt" />
				<!-- Note: To remove file2.txt on uninstall, add a RemoveFile element -->
				<CopyFile Id="CopyFile2" SourceProperty="SourceDir" SourceName="file2.txt" DestinationDirectory="INSTALLFOLDER" DestinationName="file2.txt" />
			 </Component> 
		</ComponentGroup>
	</Fragment>
</Wix>

SetupProject1\SetupProject1\SetupProject1.wixproj: (Copies file2.txt manually to output folder)

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" InitialTargets="EnsureWixToolsetInstalled" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">x86</Platform>
    <ProductVersion>3.10</ProductVersion>
    <ProjectGuid>f98d3b0a-51b3-460c-861b-0d72e0ceb935</ProjectGuid>
    <SchemaVersion>2.0</SchemaVersion>
    <OutputName>SetupProject1</OutputName>
    <OutputType>Package</OutputType>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|x86' ">
    <OutputPath>bin\$(Configuration)\</OutputPath>
    <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
    <DefineConstants>Debug</DefineConstants>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x86' ">
    <OutputPath>bin\$(Configuration)\</OutputPath>
    <IntermediateOutputPath>obj\$(Configuration)\</IntermediateOutputPath>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="Product.wxs" />
  </ItemGroup>
  <ItemGroup>
    <None Include="file1.txt" />
    <LargeFiles Include="file2.txt" />
  </ItemGroup>
  <Import Project="$(WixTargetsPath)" Condition=" '$(WixTargetsPath)' != '' " />
  <Import Project="$(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets" Condition=" '$(WixTargetsPath)' == '' AND Exists('$(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets') " />
  <Target Name="EnsureWixToolsetInstalled" Condition=" '$(WixTargetsImported)' != 'true' ">
    <Error Text="The WiX Toolset v3.11 (or newer) build tools must be installed to build this project. To download the WiX Toolset, see http://wixtoolset.org/releases/" />
  </Target>
  <Target Name="CopyLargeFiles" AfterTargets="AfterBuild">
    <Copy SourceFiles="@(LargeFiles)" DestinationFolder="$(OutputPath)" />
  </Target>
  <!--
	To modify your build process, add your task inside one of the targets below and uncomment it.
	Other similar extension points exist, see Wix.targets.
	<Target Name="BeforeBuild">
	</Target>
	<Target Name="AfterBuild">
	</Target>
	-->
</Project>

SetupProject1\Bootstrapper1\Bundle.wxs:

<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
	<Bundle Name="Bootstrapper1" Version="1.0.0.0" Manufacturer="Placeholder" UpgradeCode="88d3e27f-d370-4a6b-b4d8-6094afa498cb">
		<BootstrapperApplicationRef Id="WixStandardBootstrapperApplication.RtfLicense" />

		<Chain>
			<MsiPackage SourceFile="..\SetupProject1\bin\Debug\SetupProject1.msi" Cache="no" />
		</Chain>
	</Bundle>
</Wix>

For good measure, I put copies of file2.txt in

• SetupProject1's source directory
• SetupProject1's output directory
• Bootstrapper1's output directory

but none of these were copied when I ran the bootstrapper.

[rseanhall]

The WiX Toolset does not use GitHub issues for support. I have converted this issue into a discussion item, which is an experiment that we're trying out. Alternatively, you may want to ask your question on the wix-users mailing list or Stack Overflow.

For security reasons, the bootstrapper will only execute packages from the package cache after it has verified that all of its files are exactly the same as when the bundle was built. Cache="no" means that the package should be removed from the cache at the end of execution.

You need to add the loose file(s) as a Payload element as a child of the MsiPackage element, with Compressed="no" if you never it want it compressed into the bundle.

[AlexKubiesa]

Thank you, I had no idea it could be done that way!

Unfortunately now I get the following error (perhaps due to the size of the payload): light.exe error LGHT0001: Arithmetic operation resulted in an overflow.

[rseanhall]

That seems to be #4008, which no one has fixed yet.

[AlexKubiesa]

Ah, that's a shame. What would you recommend in this case to make sure the large files get copied correctly? A custom action?

[rseanhall]

Well, it's open source so you could help everyone and fix that bug. Otherwise, figure out how to break it into smaller files so you can rely on the security provided by the bundle. As a last resort, there is a built-in Variable that you could pass to the MSI in the MsiProperty element to use that path instead of SourceDir (I am purposely not giving the name here since it is not recommended to blindly copy files from an untrusted location to a trusted location).

[AlexKubiesa]

The way I did it in the end was to zip the files before the build, and include a custom action in the MSI to unzip the files to the install directory on installation. With regard to fixing the bug - I'll have a go!