Best practices for deprecating an MSI in a standard bootstrapper app and moving files into another existing MSI in the bootstrapper?

[AndySmiffy]

We have a Wix Standard Bootstrapper App Installer created with Wix 314, which contains five MSI packages. Which packages are installed can be selected on first installation. All upgrades are major, and we use the remember pattern using the registry to ensure that only those MSIs selected on first installation are subsequently upgraded (in passive mode).

One of the MSI packages no longer makes sense to be a separate choice, so I would like to remove that MSI and include the files in one of the other MSI's as the functionality logically fits.

What would be the best practice in doing this so that the major upgrade "just works" ? I've already tried this by moving the components over but they are not being installed, so I have obviously missed something.

Unfortunately the files in the original (to be deprecated) MSI were all put under one component, so did not have their own File Id or Component Id if this matters / helps, like this:

<File Source="..\..\External References\Newtonsoft.json\13.0.1\Newtonsoft.Json.dll"/>
<File Source="..\..\External References\SimpleTCP\1.0.24\SimpleTCP.dll"/>
<File Source="..\..\External References\Star Micronics\bin_anycpu/StarIO.dll"/>
<File Source="..\..\External References\Star Micronics\bin_anycpu/StarIOPort.dll"/>

The moved components now have their own Id's:

 <Component Id="cmpSalesscreen_Newtonsoft.Json.dll" Guid="657BE42D-83C0-45EE-AF1A-9801DBF521F4">
   <File Id="Newtonsoft.Json.dll" Name="Newtonsoft.Json.dll" KeyPath="yes" Source="..\..\External References\Newtonsoft.json\13.0.1\Newtonsoft.Json.dll" />
</Component>

I can't see anything in the logs as to why the "new" files (moved from the deprecated MSI) aren't being installed, for instance for Newtonsoft.Json.dll which is one of the new files thats not installed:

Executing op: FileCopy(SourceName=ao4lhgfy.dll|Newtonsoft.Json.dll,SourceCabKey=Newtonsoft.Json.dll,DestName=Newtonsoft.Json.dll,Attributes=512,FileSize=712232,PerTick=65536,,VerifyMedia=1,,,,,CheckCRC=0,Version=13.0.1.25924,Language=0,InstallMode=58982400,,,,,,,)
File: C:\Program Files (x86)\XXX\SalesSetup\Newtonsoft.Json.dll;	To be installed;	Won't patch;	No existing file
Source for file 'Newtonsoft.Json.dll' is compressed 

doesn't seem to be any different to one that IS updated:

Executing op: FileCopy(SourceName=0gorqtt-.con|PrinterMonitor.exe.config,SourceCabKey=PrinterMonitor.exe.config,DestName=PrinterMonitor.exe.config,Attributes=512,FileSize=5983,PerTick=65536,,VerifyMedia=1,,,,,CheckCRC=0,,,InstallMode=58982400,HashOptions=0,HashPart1=-1954599102,HashPart2=1023239144,HashPart3=1663202745,HashPart4=-1470250837,,)
File: C:\Program Files (x86)\XXX\SalesSetup\PrinterMonitor.exe.config;	To be installed;	Won't patch;	No existing file
Source for file 'PrinterMonitor.exe.config' is compressed

NB Currently none of these newly changed Bootrappers are Live, only internal, hence why I'm after a best practice as what I have tried has failed.
Many thanks.

[wmanning]

Has the old MSI been uninstalled before installing the new combined MSI?

If they are still being installed to the same location and the GUIDs remain the same but the old MSI is not uninstalled my expectation is they would not be installed. Because of component rules.

[AndySmiffy]

Well, the files were being installed into the same directory, and the guids (where they existed) were different between the original 2 MSIs. The new combined MSI now has IDs on both file and components, rather than the files being combined under a handful of components.

Order may well be a factor.
The (now deprecated) MSI was one of the last be be installed, and the other (which now contains the deprecated MSIs files with new guids) was one of the first.

The installer uses a major upgrade, so my understanding of uninstall/install order (for a bundle with 2 MSIs a & b) is :

Uninstall existing a.msi
Install new a.msi
Uninstall existing b.msi
Install new b.msi

as opposed to

Uninstall existing a.msi
Uninstall existing b.msi
Install new a.msi
Install new b.msi

Therefore if in the above example b.msi is deprecated / removed from the bundle, then assuming the process thinks the files (now in a.msi) are referenced already, then they won't get installed by a.msi, and then they won't be re-installed by b.msi since b.msi won't get re-installed since it's no longer in the bundle?

I'm not sure how I can get around that? (apologies if my explanation / understanding is lacking! )

[wmanning]

Default for major upgrade is to remove the existing MSI and then install the new MSI so you are correct there. Schedule attribute of https://wixtoolset.org/docs/v3/xsd/wix/majorupgrade/.

If you manually uninstall the two previous MSIs and install the new combined MSI does it work as expected?
Does the new combined MSI have different GUIDs (that the two original MSIs) on the new components?

The component is the smallest (aka atomic) piece of install. The MSi installer (windows) would look to see if the component is installed and to determine that it doesn't look at each file in the component, only the keypath file. Each component has a keypath (e.g. file) and GUID which determines if the component is installed or not. I'm not sure what the IDs do; I think they just uniquely identify the component in the MSI and are not used otherwise.

[AndySmiffy]

Thanks. To answer your questions :

If you manually uninstall the two previous MSIs and install the new combined MSI does it work as expected?

Yes, All files are present and correct.

Does the new combined MSI have different GUIDs (that the two original MSIs) on the new components?

Yes. The original MSIs had many files under very few components (80 files in 6 components). 
The new MSI has a component for each file such as....

<Component Id="cmpSalesscreen_Newtonsoft.Json.dll" Guid="657BE42D-83C0-45EE-AF1A-9801DBF521F4">
   <File Id="Newtonsoft.Json.dll" Name="Newtonsoft.Json.dll" KeyPath="yes" Source="..\..\External References\Newtonsoft.json\13.0.1\Newtonsoft.Json.dll" />
</Component>

The component is the smallest (aka atomic) piece of install. The MSi installer (windows) would look to see if the component is installed and to determine that it doesn't look at each file in the component, only the keypath file. Each component has a keypath (e.g. file) and GUID which determines if the component is installed or not. I'm not sure what the IDs do; I think they just uniquely identify the component in the MSI and are not used otherwise.

Keypath is an interesting one, as the original MSIs only used keypath when setting the registry entries, not on the files. 
In the new MSI keypath has been set on all file elements (not the parent component) as in the example above.
This doesn't seem to be enough on it's own since files aren't (re)installed during the upgrade.

[wmanning]

If keypath is not specified then Wix uses a default per: https://wixtoolset.org/docs/v3/xsd/wix/component/

If KeyPath is not set to 'yes' for the Component or for a child Registry value or File, WiX will look at the child elements under the Component in sequential order and try to automatically select one of them as a key path. Allowing WiX to automatically select a key path can be dangerous because adding or removing child elements under the Component can inadvertantly cause the key path to change, which can lead to installation problems.

To go over this again you use to have two MSIs (say 1 and 2) but now the contents of MSI-1 were moved into the contents of MSI-2 so MSI-1 does not exist anymore.

In MSI-2, I think you may need an upgrade element and a upgrade version as a child under that element to remove MSI-1.
https://wixtoolset.org/docs/v3/xsd/wix/upgrade/
https://wixtoolset.org/docs/v3/xsd/wix/upgradeversion/
Note

• Set the ID to the MSI-1's upgrade code (guid) on the upgrade element.
• Set the OnlyDetect attribute on the upgradeversion element to false. This will cause the MSI to uninstall if the conditions match.
• set other attributes as applicable

Another option is maybe to use the Uninstall condition on the MsiPackage element for MSI 1 to have it uninstall. Seems if false it'll always uninstall but the docs doesn't mention upgrade though.
https://wixtoolset.org/docs/schema/wxs/upgradeversion/

InstallCondition (String) : A condition to evaluate before installing the package. The package will only be installed if the condition evaluates to true. If the condition evaluates to false and the bundle is being installed, repaired, or modified, the package will be uninstalled.

I would lean towards the upgrade/upgradeversion solution since MSI 2 owns MSI 1. It seems more logical.

Warning, I never done any of this. I do use maintain a burn package and MSIs but never had a case like yours.