"Compatible" shared packages?

[MatthiasRInst]

Hello,

We are developing several bundles that share some common MSI packages. We are still using WiX 3.11, trying to learn about 4.0 before migrating.

The various bundles can be installed side-by-side (with the shared products going to a shared location), and I cannot predict the order or the versions in which the user installs different bundles.

All the packages get major upgrades from time to time.

It is possible that the user first installs:

Bundle A in version 3.0 with version 3.0 of SharedPackage (which has ProductCode {33333333-F197-48F8-B6AC-5102D36BBCE0})
then
Bundle B in version 2.0 with version 2.0 of SharedPackage (which has ProductCode {22222222-F197-48F8-B6AC-5102D36BBCE0})

where the SharedPackage product code has changed from version 2.0 to 3.0

The WiX code for both bundles contains:

		<MsiPackage Id="SharedPackage" [..] >

			<!-- Generic dependency provider for SharedPackage versions >= 2: -->
			<dep:Provides Key="MyCompany.SharedPackage"/>
			<!-- SharedPackage v1 product guid (kept here for legacy compatibility, since we did not know how to use the dependency extension back then):-->
			<dep:Provides Key="{11111111-F197-48F8-B6AC-5102D36BBCE0}"/>

In the above scenario, I expect Bundle B to not install the SharedPackage, but to register the current bundle as dependency on both providers listed above.

But I get a different behaviour. Bundle B install log file tells me:

i103: Detected related package: {33333333-F197-48F8-B6AC-5102D36BBCE0}, scope: PerMachine, version: 3.0.0.0, language: 0 operation: Downgrade
i000: Detected existing related package SharedPackage (product: {33333333-F197-48F8-B6AC-5102D36BBCE0}) at version 3.0.0.0, which will be Downgrade
[..]
i101: Detected package: SharedPackage, state: Obsolete, cached: None
and a bit later:
i201: Planned package: SharedPackage, state: Obsolete, default requested: None, ba requested: None, execute: None, rollback: None, cache: No, uncache: No, dependency: None

and this dependency is really not registered! As a consequence, the SharedPackage gets removed on Bundle A uninstallation even though it is still needed by Bundle B.
So the generic dependency provider key seems not to do here what we introduced it for. This is a behaviour I do not understand.

What is even more strange:

If I add the SharedPackage v2.0 product code as dependency provider key to v3.0 bundles (here: Bundle A)

		<MsiPackage Id="SharedPackage" [..] >

			<!-- Generic dependency provider for SharedPackage versions >= 2: -->
			<dep:Provides Key="MyCompany.SharedPackage"/>
			<!-- SharedPackage v1 product guid (kept here for legacy compatibility, since we did not know how to use the dependency extension back then:-->
			<dep:Provides Key="{11111111-F197-48F8-B6AC-5102D36BBCE0}"/>
			<!-- SharedPackage v2 product guid (Why do I need it???):-->
			<dep:Provides Key="{22222222-F197-48F8-B6AC-5102D36BBCE0}"/>

Bundle B install log file says:

i108: Detected compatible package: SharedPackage, provider: {22222222-F197-48F8-B6AC-5102D36BBCE0}, installed: {33333333-F197-48F8-B6AC-5102D36BBCE0}, version: 3.0.0.0, chained: {22222222-F197-48F8-B6AC-5102D36BBCE0}
i103: Detected related package: {33333333-F197-48F8-B6AC-5102D36BBCE0}, scope: PerMachine, version: 3.0.0.0, language: 0 operation: Downgrade
i000: Detected existing related package SharedPackage (product: {33333333-F197-48F8-B6AC-5102D36BBCE0}) at version 3.0.0.0, which will be Downgrade
[..]
i101: Detected package: SharedPackage, state: Obsolete, cached: None
[..]
i201: Planned package: SharedPackage, state: Obsolete, default requested: None, ba requested: None, execute: None, rollback: None, cache: No, uncache: No, dependency: Register

and everything works fine.

So from my understanding, a major-upgraded shared package is "compatible" with its current version if and only if its current product code is listed as dependency provider key in the current bundle.

So are we missing something?

Best regards

Matthias Reuss

[BMurri]

Think of the shared packages like shared files. Do your versioned bundles need specific versions of the shared packages, or will the latest versions of those shared packages always work just fine?

If the shared packages need to be versioned, they must be installed in a SxS (side-by-side) fashion, meaning that they must always be changing their installation paths (both filesystem and registry) for every version and they must not share UpgradeCodes (since they are no longer "upgrades" to each other), and you should continue using ProductCodes as dependency keys.

If the latest version always just works, then you should continue building them the same way as today, but use the MSI UpgradeCode (or any other relevant but fixed identifier) as the dependency keys, for the behavior you are looking for.

If your scenario is between these two, then you need to create your own dependency key schema. See this page for more detail (note that page explicitly states that using ProductCodes (the default) does not support upgrades (and by implication, downgrades) when used as the key, which is what you are seeing in your experience above).

[MatthiasRInst]

Hi Blair,

thank you for the reply.

In our case, we just need the latest versions of the shared packages.
That's why we do create our own dependency key schema. We do use what I called a "generic dependency provider key" - in our case, we do not use the upgrade code, but some more friendly name like MyC'ompany.SharedPackage. At least this one is shared by both versions of the shared package.

Moreover, there are some explicitly added GUID provider keys that - more or less by chance - match some historical product codes. But all these provider keys that we added there are persistent. Registry holds more than one key for each shared package of ours, and once created, they will be used again and again by the later versions of the shared package. That is different to the default case (without explicit dep:Provides), where the provider key does not support upgrades.

To be more concrete:

At the time when I install the bundle with the obsolete version of the Shared package, the dependency provider with the generic key MyCompany.SharedPackage has already been registered with the other bundle. So I would expect the 2nd bundle to register with that provider, which it also provides.

[BMurri]

If you are only providing your new keys (and the legacy keys) why does " {33333333-F197-48F8-B6AC-5102D36BBCE0}" exist at all as a dependency key? From what you just described as your current usage, that's not a legacy dependency key, aka shouldn't ever be in any Provides or Requires attribute anywhere.

It would be clearer if you also showed us all of your Requires authoring, because domain-wide these two attributes are intimately related to each other.

[MatthiasRInst]

{33333333-F197-48F8-B6AC-5102D36BBCE0} is the product code of v3 of the shared package. It is not used as a dependency key.

We do not use any Requires tag. Should we? From my understanding, the dependencies are registered using the Provides tags, and premature uninstallation is (or should be) inhibited by refcounting the dependencies registered for each provider - very similar to shared components in MSI.

Is there now some comprehensive documentation on dependency authoring? In WiX3 documentation, that page only contains the text "ToDo".

[BMurri]

If there are no Requires, premature uninstallation is a nonsensical concept. It's unclear to me what is or isn't prevented or even provided for in that scenario.

Each package that requires any given shared package should have a Requires with the ProviderKey attribute set to your "MyCompany.SharedPackage" key (repeat for each shared package required by that package). Also include the "legacy" key, as long as that legacy package still can satisfy your dependency needs.

You will probably need to set an Id for each of those elements, just make them unique inside that package.

[MatthiasRInst]

So I will try with Requires elements.

As to the concept:
Shared package is installed by 2 or more bundles, each bundle is registered as a dependency on the dependency provider of the package (ideally, we need just 1 provider).
For me this means that all these bundles actually need (or require? or depend on?) that shared package.
So only the uninstallation of the last bundle that depends on the shared package shall remove that shared package.
Does this make sense?

[BMurri]

Yes, that is what I would expect.

BTW, the ", dependency: Register" at the top means that in that case Burn was registering the shared package it was NOT installing as a provider, NOT registering any type dependency on that package. I suspect because of the behavior you saw you might have been interpreting the message in the log that way.

[MatthiasRInst]

Ok, so it is a concept that does make sense. And from my understanding, this does not need any explicit Requires setting (in the default case that all versions of the shared package are compatible). The Requires element would then just allow the developer to add some further restrictions. And I do not want the 1st bundle uninstall to err, just to keep the shared package in place.

I do not understand your last paragraph, though. "Dependency:Register" in the Plan phase means that the dependency will be registered, regardless of whether or not the package will be installed. In this case, the dependency registration can be read from the log of the Apply phase - and from Registry.
In contrast to that, "Dependency:None" in the plan phase means that no dependency will be registered.

My expectation in the use case I described was:

1. 1st install shall create the providers listed (here the providers have the keys "MyCompany.SharedPackage" and "{11111111-F197-48F8-B6AC-5102D36BBCE0}" and register the 1st bundle on both providers.
2. On 2nd install, the shared package shall not be installed (obsolete version), but dependency shall be registered on both providers (both versions of the shared package contain the same Provides elements) - so after the 2nd bundle install, poth providers will have dependencies on both bundles.

Expectation 2 is not fulfilled (and I still do not understand why), but it strangely gets fulfilled when I add one more provider key to the newer version of the shared package.

[BMurri]

And I do not want the 1st bundle uninstall to err, just to keep the shared package in place.

The point of dependency registration is to keep the shared package in place. Registering bundles as providers means that the bundles are kept in place when other packages that require packages installed by them are still installed. Uninstalling a bundle will normally attempt to uninstall all packages in that bundle. If a package cannot be uninstalled because it is still required, then dependency is doing its job. If two installed bundles provide the same key, I believe (but haven't verified) that the related bundle will be repaired to ensure that shared dependencies aren't broken by the removal.

While adding more keys makes the provider register itself as a provider, that doesn't mean it will prevent package uninstallation (since nothing depends on it). It may be an interesting byproduct that it appears to work.

[MatthiasRInst]

The point of dependency registration is to keep the shared package in place. Registering bundles as providers means that the bundles are kept in place when other packages that require packages installed by them are still installed. Uninstalling a bundle will normally attempt to uninstall all packages in that bundle. If a package cannot be uninstalled because it is still required, then dependency is doing its job.

Fully agreed. BTW, uninstallaing a package that is still required is what I called "premature uninstallation" above.

So how to achieve this goal?
The shared MsiPackage element contains a Provides element <dep:Provides Key="MyCompany.SharedPackage"/> which makes both Bundles depend on the shared package. First installation registers the dependency provider key MyCompany.SharedPackage:

   Registering package dependency provider: MyCompany.SharedPackage, version: 3.0.0.0, package: SharedPackage

and then registers the current bundle as dependency on that provider:

  Registering dependency: {f30cda77-28a8-4506-b011-803f55695cc0} on package provider: MyCompany.SharedPackage, package: SharedPackage

Second installation registers the second bundle as dependency on the same provider. The result can then be viewed in the Registry under HKCR\Installer\Dependencies.

Uninstallation of either bundle will then notice that there is another dependency on the same provider (sort of "refcounting") and inhibit the removal of the shared package, so the goal is achieved.

Registering a dependency can be done does not require that the shared package is being installed - there are cases with

  i201: Planned package: SharedPackage, state: Obsolete, default requested: None, ba requested: None, execute: None, rollback: None, cache: No, uncache: No, dependency: Register

which will allow for refcounting and inhibit the premature removal on uninstall.

But there are also cases with
i201: Planned package: SharedPackage, state: Obsolete, default requested: None, ba requested: None, execute: None, rollback: None, cache: No, uncache: No, dependency: None

So what does actually determine whether or not a dependency gets registered? At least in our case, we assume all versions of the shared package to be compatible with older bundles.

And why should the behaviour be different if one MsiPackage element brings two dependency providers instead of one?

Documentation on dependency handling is rather poor - I would have loved to write that page ...

[BMurri]

@robmen @barnson I don't recall who wrote the dependency code, but this might be a good time to push for the long-never-written documentation of this feature of WiX.

[barnson]

At this point, it would be writing down observed behavior and maybe looking for bugs...