Added WiX Toolset v4 and newer support

[juliusl]

Addreses #174 and #293

• Validate test matrix
• Add unit tests
  -- Current code coverage stats --

|| src\toolset\ext.rs: 88/102~ 86%
|| src\toolset\includes.rs: 32/74 ~ 43%
|| src\toolset\mod.rs: 118/174 ~ 67%
|| src\toolset\project.rs: 51/65 ~ 78%
|| src\toolset\source.rs: 29/46 ~ 63%

• Enable --toolset disambiguation in init command (Ended up going with --schema flag to disambiguate, updated print command as well)
• Add standalone command for "upgrade" and other procedures
• Fix work_dir logic if file names are repeated under different directories
• Fix sxs mode, need to change the wxs source enumeration to reflect the changes

[volks73]

Thank you so much for creating and sharing this PR. It is great work.

The automatic upgrade is a nice workaround to existing v3 WXS files. I would have added a separate subcommand: cargo wix upgrade or cargo wix migrate and have it be a separate "step". Having the upgrade be a separate command and step would allow users to debug failures and manually resolve issues. Upgrading only needs to be done once and only if v3 files already existed.

A separate subcommand also provides a nice separation of concern.

New users/projects do not need to upgrade. The ultimate solution is have two separate mustache files, one for v3 and one for v4. Then, roll over to v4 as the default toolset for new/existing projects.

That said, this implementation is a very nice "in-between" solution and iteration. I would like to see a --no-upgrade flag/feature that disables the check and upgrade if desired. If the upgrading is moved to a separate subcommand, then this flag is not needed.

I wonder if it makes sense to also have the upgrading and v4 behind a feature flag?

I have not had a chance to test this PR locally, but I wanted to share some initial thoughts and provide some feedback. Excellent work!

[juliusl]

That said, this implementation is a very nice "in-between" solution and iteration. I would like to see a --no-upgrade flag/feature that disables the check and upgrade if desired. If the upgrading is moved to a separate subcommand, then this flag is not needed.

Sure, actually as-is all of this is opt-in, so without the flag it defaults to no-check and no-upgrading same with the toolset default.

A separate subcommand also provides a nice separation of concern.

Sure, I don't mind splitting this up into a different subcommand, it was just easier to bake it into create because create has all of the source file plumbing.

Thanks for the review, I'll address your comments as soon as I can!

[juliusl]

@volks73 I added a "setup" subcommand. Here is what the cli help would look like for cargo wix setup:

Perform various wix setup operations such as: converting from WiX3 wxs files to Wix4 format or
restoring wix extension packages by analyzing extension dependencies found in wxs files.
Can also enable vendoring of wix dependencies for offline build

If no additional setup flags are passed such as: `--restore`, `--upgrade`, `--sxs`, or `--vendor`
and no mode is set via `-m` or `--mode`
Then this will apply to 'project' setup mode, meaning all wxs files will be converted in place
and wix extensions will be installed globally

Usage: cargo wix setup [OPTIONS] [INPUT]

Arguments:
  [INPUT]
          If no value is provided, then the current
          working directory (CWD) will be used to locate a package's
          manifest. An error will occur if a manifest cannot be
          found. A relative or absolute path to a package's manifest
          (Cargo.toml) file can be used. Only one manifest is
          allowed. The creation of an installer will be relative to
          the specified manifest.

Options:
  -p, --package <package>
          Selects the package within a project organized as a workspace. Workspaces have one or more members, where each member is a package. This option selects the package by name.

  -I, --include <include>
          Includes a WiX source (wxs) file for a project, where the wxs file is not located in the default location, i.e. 'wix'. Use this option multiple times to include multiple wxs files.

      --restore
          May be combined with --vendor but will be ignored if --sxs is passed.

          If `--restore --vendor` are used, equivalent to `-m restore-vendor` or `--mode restore-vendor`

          Otherwise, if `--restore` is used alone equivalent to `-m restore` or `--mode restore

      --upgrade
          May be combined with --sxs but will be ignored if --vendor is passed.

          If `--upgrade --sxs` are used, this would be equivalent to `-m upgrade-sxs` or `--mode upgrade-sxs`

          Otherwise, if `--upgrade` is used alone Equivalent to `-m upgrade` or `--mode upgrade`

      --sxs
          By using side-by-side mode, all setup all setup operations
          will be executed in a folder corresponding to the major wix version detected. (Ex. \wix4\..)
          This is suited for complex wix projects where bake
          time maybe needed between major wix versions. Implicitly vendors dependencies in the same folder.

          Equivalent to `-m vendor` or `--mode vendor`

      --vendor
          By using vendoring mode, all extension dependencies will be installed in the current directory
          This is suited for proejcts that must be built in environments without network access and/or
          If tracking extension packages in SVC is desired

          Equivalent to `-m vendor` or `--mode vendor`

  -m, --mode <mode>
          Will explicitly configure the setup mode. Note: Using `none` will not have any effect with the setup command.

          Possible values:
          - none:           Do not apply any setup logic. (Default)
          - project:        Upgrade source files in place and install extensions to the global extension cache
          - vendor:         Setup project in "vendored" mode
          - sxs:            Upgrade files side by side and install extensions to the sxs folder
          - restore:        Only Restore any missing extensions to the global WiX extension cache
          - restore-vendor: Only Restore any missing extensions to the current directory
          - upgrade:        Upgrade files in place
          - upgrade-sxs:    Upgrade source files in a SxS manner

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

and for create:

Build Windows installers using the WiX Toolset

Usage: cargo wix [OPTIONS] [INPUT] [COMMAND]

Commands:
  clean  Deletes the 'target\wix' folder
  init   Generates files from a package's manifest (Cargo.toml) to create an installer
  print  Prints a template
  purge  Deletes the 'target\wix' and 'wix' folders
  sign   Signs an installer
  setup  WiX project setup utilities
  help   Print this message or the help of the given subcommand(s)

Arguments:
  [INPUT]
          If no value is provided, then the current working directory (CWD) will be used to locate a package's manifest. An error will occur if a manifest cannot be found. A relative or absolute path to a package's manifest (Cargo.toml) file can be used. Only one manifest is allowed. The creation of an installer will be relative to the specified manifest.

Options:
  -b, --bin-path <bin-path>
          Specifies the path to the WiX Toolset's 'bin' folder, which should contain the needed 'candle.exe' and 'light.exe' applications. The default is to use the path specified with the WIX system environment variable that is created during the installation of the WiX Toolset. Failing the existence of the WIX system environment variable, the path specified in the PATH system environment variable is used. This is useful when working with multiple versions of the WiX Toolset.

  -c, --culture <culture>
          Sets the culture for localization. Use with the '-l,--locale' option. See the WixUI localization documentation for more information about acceptable culture codes. The codes are case insensitive.

  -C, --compiler-arg <compiler-arg>
          Appends the argument to the command that is invoked when compiling an installer. This is the same as manually typing the option or flag for the compiler at the command line. If the argument is for an option with a value, the option's value must be passed as a separate call of this option. Multiple occurrences are possible, but only one value per occurrence is allowed to avoid ambiguity in argument parsing. For example, '-C -ext -C WixUtilExtension'.

  -t, --target <target>
          Tells cargo to build the given target, and instructs WiX to build an installer targeting the right architecture.

  -d, --dbg-build
          Uses the Debug profile when building the package using Cargo. The default is to build the package using the Release profile.

      --profile <profile>
          Uses the given profile when building the package using Cargo. The default is to build the package using the Release profile.

  -D, --dbg-name
          Adds the '-debug' suffix to the file stem (content before the file extension) for the installer's file name. This should be used to indicate the binary distributed within the installer was built using debugging information and optimizations. Generally, this should be used in combination with the '-d,--dbg-build' flag to build the binary with the Debug profile.

  -I, --include <include>
          Includes a WiX source (wxs) file for a project, where the wxs file is not located in the default location, i.e. 'wix'. Use this option multiple times to include multiple wxs files.

  -i, --install-version <install-version>
          Overrides the version from the package's manifest (Cargo.toml), which is used for the installer name and appears in the Add/Remove Programs control panel.

  -L, --linker-arg <linker-arg>
          Appends the argument to the command that is invoked when linking an installer. This is the same as manually typing the option or flag for the linker at the command line. If the argument is for an option with a value, the option's value must be passed as a separate call of this option. Multiple occurrences are possible, but only one value per occurrence is allowed to avoid ambiguity in argument parsing. For example, '-L -ext -L WixUIExtension'.

  -l, --locale <locale>
          Sets the path to a WiX localization file (wxl) which contains localized strings. Use in conjunction with the '-c,--culture' option.

  -n, --name <name>
          Overrides the 'name' field in the package's manifest (Cargo.toml), which is used in the file name of the installer (msi). This does not change the name of the executable within the installer.

      --no-build
          The installer is created, but the 'cargo build --release' is not executed.

      --target-bin-dir <target-bin-dir>
          Sets the CargoTargetBinDir variable that will be substituted into main.wxs. Use in conjunction with --no-build to fully handle builds.

      --nocapture
          By default, this subcommand captures, or hides, all output from the builder, compiler, linker, and signer for the binary and Windows installer, respectively. Use this flag to show the output.

      --install
          Creates the installer and runs it after that.

  -o, --output <output>
          Sets the destination file name and path for the created installer, or the destination folder for the installer with the default file name. If the path is to an existing folder or has a trailing slash (forward and backward), then the default installer file name will be used and the installer will be available in the folder after creation. Otherwise, this value overwrites the default file name and path for the installer. The default is to create an installer with the '<product-name>-<version>-<arch>.msi' file name in the 'target\wix' folder.

      --toolset <toolset>
          Before WiX4 the binaries that were used to build the installer were light.exe and candle.exe.
          The WiX toolset now uses a single binary, wix.exe, that performs all the previous operations.
          In addition WiX3 did not have a unified package installation method, where-as WiX4 and beyond
          can be installed via `dotnet install`.

          This argument can be used to opt-in to the modern toolset, however it is important to note that
          while WiX4+ are backwards compatible, WiX3 .wxs files are not compatible and must be converted
          to WiX4 semantics. Luckily the modern wix toolsets contain a convert tool, however there are
          several key differences after WiX3 that must be addressed before the files may be built.
          If the `toolset-upgrade` argument is provided in addition to this command, cargo-wix can ensure
          that the included wix source files are updated to the modern format and that the dependencies used
          in those source files are installed in the current context. This also provides the benefit between
          WiX4+ versions, as the wix toolset extensions now depend on the WiX toolset version, `cargo-wix`
          can ensure that before the build starts all wix extensions are up to date.

          Possible values:
          - legacy: The default wix toolset uses "candle.exe" and "light.exe" to build the installer
          - modern: Modern wix toolsets use just "wix.exe" to build the installer

      --setup <setup>
          (Only applied if `--toolset modern` is present)
          When enabled, this will perform checks to ensure that the current wix environment
          is up to date before attempting to build the installer which includes:
                1) Ensuring that *.wxs files are in the correct format according to the wix toolset setting
                2) Installing extensions that are required to build the installer
          There are several modes that can be used with inplace being the simplest and sxs being the least destructive

          Possible values:
          - none:           Do not apply any setup logic. (Default)
          - project:        Upgrade source files in place and install extensions to the global extension cache
          - vendor:         Setup project in "vendored" mode
          - sxs:            Upgrade files side by side and install extensions to the sxs folder
          - restore:        Only Restore any missing extensions to the global WiX extension cache
          - restore-vendor: Only Restore any missing extensions to the current directory
          - upgrade:        Upgrade files in place
          - upgrade-sxs:    Upgrade source files in a SxS manner

  -p, --package <package>
          Selects the package within a project organized as a workspace. Workspaces have one or more members, where each member is a package. This option selects the package by name.

  -v, --verbose...
          Sets the level of verbosity. The higher the level of verbosity, the more information that is printed and logged when the application is executed. This flag can be specified multiple times, where each occurrence increases the level and/or details written for each statement.

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

[volks73]

I like the creation and organization of a toolset module to put all of the toolset, v3, v4, etc., handling code into a separate module. Keeps it nice and clean.

Thank you for breaking out the upgrade-related activity into a separate subcommand. I like the additional utilities and functionality you have added within this subcommand.

I would like the subcommand to be renamed from setup to either: (1) upgrade or (2) migrate. The setup subcommand "conflicts" with the cargo wix init command. As a first time user, am I supposed to use the cargo wix init or cargo wix setup command? Based on this question, I would lean towards the migrate subcommand to avoid confusion.

WiX v4 extensions are stored in the .wxs folder or are these installed globally? I have to read up on WiX v4 extension system.

[juliusl]

Extensions end up being stored in a .wix/extenstion folder in the same directory unless you specify --global. The "global" cache is the dir %userprofile%\.wix\extensions. (I'll make sure to document this when I update the docs)

Sure I can rename the command. I had a hard time figuring out how to name it because the restore logic is actually handy between wix versions as the normal wix extension installer won't pick the right version for its current self it just goes with the latest version. 🤦🏻‍♂️

[juliusl]

Starting to work on the new template, here are the baseline things changed by convert. Putting this here for posterity

D:\rs\wix_init_Scratch\wix\main.wxs : information WIX0005: [Converted] This file contains an XML declaration on the first line. (DeclarationPresent)
D:\rs\wix_init_Scratch\wix\main.wxs(58) : information WIX0009: [Converted] The namespace 'http://schemas.microsoft.com/wix/2006/wi' is out of date. It must be 'http://wixtoolset.org/schemas/v4/wxs'. (XmlnsValueWrong)
D:\rs\wix_init_Scratch\wix\main.wxs(60) : information WIX0023: [Converted] Using '*' for the Product Id attribute is unnecessary. Remove the attribute to remove the redundancy. (AutoGuidUnnecessary)
D:\rs\wix_init_Scratch\wix\main.wxs(60) : information WIX0030: [Converted] The Product and Package elements have been renamed and reorganized for simplicity. (ProductAndPackageRenamed)
D:\rs\wix_init_Scratch\wix\main.wxs : information WIX0058: [Converted] The TARGETDIR directory should no longer be explicitly defined. Remove the Directory element with Id attribute 'TARGETDIR'. (TargetDirDeprecated)
D:\rs\wix_init_Scratch\wix\main.wxs(69) : information WIX0024: [Converted] The Feature element's Absent attribute has been replaced with the AllowAbsent attribute. Use the 'AllowAbsent' attribute instead. (FeatureAbsentAttributeReplaced)
D:\rs\wix_init_Scratch\wix\main.wxs(79) : information WIX0024: [Converted] The Feature element's Absent attribute has been replaced with the AllowAbsent attribute. Use the 'AllowAbsent' attribute instead. (FeatureAbsentAttributeReplaced)
D:\rs\wix_init_Scratch\wix\main.wxs : information WIX0060: [Converted] UI, custom action, and property reference WixUI_FeatureTree has been replaced with strongly-typed element. (ReferencesReplaced)
D:\rs\wix_init_Scratch\wix\main.wxs(122) : information WIX0022: [Converted] Using Publish element text is deprecated. Use the 'Condition' attribute instead. (InnerTextDeprecated)
D:\rs\wix_init_Scratch\wix\main.wxs(122) : information WIX0026: [Converted] Adding Condition='1' on Publish elements is no longer necessary. Remove the Condition attribute. (PublishConditionOneUnnecessary)
D:\rs\wix_init_Scratch\wix\main.wxs(123) : information WIX0022: [Converted] Using Publish element text is deprecated. Use the 'Condition' attribute instead. (InnerTextDeprecated)
D:\rs\wix_init_Scratch\wix\main.wxs(123) : information WIX0026: [Converted] Adding Condition='1' on Publish elements is no longer necessary. Remove the Condition attribute. (PublishConditionOneUnnecessary)
D:\rs\wix_init_Scratch\wix\main.wxs(185) : information WIX0023: [Converted] Using '*' for the Component Guid attribute is unnecessary. Remove the attribute to remove the redundancy. (AutoGuidUnnecessary)

[volks73]

The tests are coming along nicely. You are moving fast! Great work.

[juliusl]

@volks73 This test is running into issues, linker_args_flags_only_works from tests/create.rs getting this error, OK result: Command("light", 1105, true) . Is light required to be on PATH for this test to succeed?

[volks73]

Is light required to be on PATH for this test to succeed?

@juliusl Yes, it appears to be that way. This is probably a bug.

[juliusl]

@volks73 so sanity check question, from your experience are there any wix projects that do this,

./somedir/main.wxs
./main.wxs

Looking around on the web, it seems like that would be really awkward but not impossible? I'm trying to decide how much energy to expend on ensuring that use case works correctly.

[volks73]

@juliusl No, not that I have seen in my experience. The usual use case:

./main.wxs
./somedir/include_1.wxs
./somedir/include_2.wxs

or

./wix/main.wxs
./wix/include_1.wxs
./wix/include_2.wxs
./wix/include_3.wxs

or

./wxs/main.wxs
./wxs/includes/1.wxs
./wxs/includes/2.wxs
./wxs/includes/3.wxs

I also do not recall receiving a request or issue with that WiX project layout, so if it is becoming too much of a burden, then please skip and we can address in the future.

[juliusl]

@volks73 sweet, I'll put a warning on the screen at least or maybe just panic and say "migrate does not support nested .wxs source files"

[juliusl]

Lol the classic unix vs windows file path conflicts

left: "{\".test\\\\test_project_upgrade_extension_detection\\\\main.test_project_upgrade_extension_detection.wxs\": WixSource { wix_version: V4, path: \".test\\\\test_project_upgrade_extension_detection\\\\main.test_project_upgrade_extension_detection.wxs\", exts: [(\"WixToolset.BootstrapperApplications.wixext\", \"bal\", \"[http://wixtoolset.org/schemas/v4/wxs/bal\](http://wixtoolset.org/schemas/v4/wxs/bal/)"), (\"WixToolset.ComPlus.wixext\", \"complus\", \"[http://wixtoolset.org/schemas/v4/wxs/complus\](http://wixtoolset.org/schemas/v4/wxs/complus/)"), (\"WixToolset.Dependency.wixext\", \"dep\", \"[http://wixtoolset.org/schemas/v4/wxs/dependency\](http://wixtoolset.org/schemas/v4/wxs/dependency/)"), (\"WixToolset.DirectX.wixext\", \"directx\", \"[http://wixtoolset.org/schemas/v4/wxs/directx\](http://wixtoolset.org/schemas/v4/wxs/directx/)"), (\"WixToolset.Firewall.wixext\", \"fw\", \"[http://wixtoolset.org/schemas/v4/wxs/firewall\](http://wixtoolset.org/schemas/v4/wxs/firewall/)"), (\"WixToolset.Http.wixext\", \"http\", \"[http://wixtoolset.org/schemas/v4/wxs/http\](http://wixtoolset.org/schemas/v4/wxs/http/)"), (\"WixToolset.Iis.wixext\", \"iis\", \"[http://wixtoolset.org/schemas/v4/wxs/iis\](http://wixtoolset.org/schemas/v4/wxs/iis/)"), (\"WixToolset.Msmq.wixext\", \"msmq\", \"[http://wixtoolset.org/schemas/v4/wxs/msmq\](http://wixtoolset.org/schemas/v4/wxs/msmq/)"), (\"WixToolset.Netfx.wixext\", \"netfx\", \"[http://wixtoolset.org/schemas/v4/wxs/netfx\](http://wixtoolset.org/schemas/v4/wxs/netfx/)"), (\"WixToolset.PowerShell.wixext\", \"powershell\", \"[http://wixtoolset.org/schemas/v4/wxs/powershell\](http://wixtoolset.org/schemas/v4/wxs/powershell/)"), (\"WixToolset.Sql.wixext\", \"sql\", \"[http://wixtoolset.org/schemas/v4/wxs/sql\](http://wixtoolset.org/schemas/v4/wxs/sql/)"), (\"WixToolset.UI.wixext\", \"ui\", \"[http://wixtoolset.org/schemas/v4/wxs/ui\](http://wixtoolset.org/schemas/v4/wxs/ui/)"), (\"WixToolset.Util.wixext\", \"util\", \"[http://wixtoolset.org/schemas/v4/wxs/util\](http://wixtoolset.org/schemas/v4/wxs/util/)"), (\"WixToolset.VisualStudio.wixext\", \"vs\", \"[http://wixtoolset.org/schemas/v4/wxs/vs\](http://wixtoolset.org/schemas/v4/wxs/vs/)")] }}"
 right: "{\".test/test_project_upgrade_extension_detection/main.test_project_upgrade_extension_detection.wxs\": WixSource { wix_version: V4, path: \".test/test_project_upgrade_extension_detection/main.test_project_upgrade_extension_detection.wxs\", exts: [(\"WixToolset.BootstrapperApplications.wixext\", \"bal\", \"[http://wixtoolset.org/schemas/v4/wxs/bal\](http://wixtoolset.org/schemas/v4/wxs/bal/)"), (\"WixToolset.ComPlus.wixext\", \"complus\", \"[http://wixtoolset.org/schemas/v4/wxs/complus\](http://wixtoolset.org/schemas/v4/wxs/complus/)"), (\"WixToolset.Dependency.wixext\", \"dep\", \"[http://wixtoolset.org/schemas/v4/wxs/dependency\](http://wixtoolset.org/schemas/v4/wxs/dependency/)"), (\"WixToolset.DirectX.wixext\", \"directx\", \"[http://wixtoolset.org/schemas/v4/wxs/directx\](http://wixtoolset.org/schemas/v4/wxs/directx/)"), (\"WixToolset.Firewall.wixext\", \"fw\", \"[http://wixtoolset.org/schemas/v4/wxs/firewall\](http://wixtoolset.org/schemas/v4/wxs/firewall/)"), (\"WixToolset.Http.wixext\", \"http\", \"[http://wixtoolset.org/schemas/v4/wxs/http\](http://wixtoolset.org/schemas/v4/wxs/http/)"), (\"WixToolset.Iis.wixext\", \"iis\", \"[http://wixtoolset.org/schemas/v4/wxs/iis\](http://wixtoolset.org/schemas/v4/wxs/iis/)"), (\"WixToolset.Msmq.wixext\", \"msmq\", \"[http://wixtoolset.org/schemas/v4/wxs/msmq\](http://wixtoolset.org/schemas/v4/wxs/msmq/)"), (\"WixToolset.Netfx.wixext\", \"netfx\", \"[http://wixtoolset.org/schemas/v4/wxs/netfx\](http://wixtoolset.org/schemas/v4/wxs/netfx/)"), (\"WixToolset.PowerShell.wixext\", \"powershell\", \"[http://wixtoolset.org/schemas/v4/wxs/powershell\](http://wixtoolset.org/schemas/v4/wxs/powershell/)"), (\"WixToolset.Sql.wixext\", \"sql\", \"[http://wixtoolset.org/schemas/v4/wxs/sql\](http://wixtoolset.org/schemas/v4/wxs/sql/)"), (\"WixToolset.UI.wixext\", \"ui\", \"[http://wixtoolset.org/schemas/v4/wxs/ui\](http://wixtoolset.org/schemas/v4/wxs/ui/)"), (\"WixToolset.Util.wixext\", \"util\", \"[http://wixtoolset.org/schemas/v4/wxs/util\](http://wixtoolset.org/schemas/v4/wxs/util/)"), (\"WixToolset.VisualStudio.wixext\", \"vs\", \"[http://wixtoolset.org/schemas/v4/wxs/vs\](http://wixtoolset.org/schemas/v4/wxs/vs/)")] }}"

[volks73]

I'll put a warning on the screen at least or maybe just panic and say "migrate does not support nested .wxs source files"

@juliusl Maybe not panic, but gracefully exit with the message.