Handling VMs when the same Host / VMs are managed by an Infra Provider and Cloud Provider

[hari-g-m]

Hi @Fryguy @agrare ,

In case of IBM Power providers, when both HMC Provider (Infrastructure Provider) and PowerVC Provider (Cloud Provider) are added, the VMs will show up from both i.e. PowerVC manages the VMs through HMC or the same system can be managed by both PowerVC and HMC such that the same VMs will show up from both and end up in duplicate VMs showing up in total.

One option could be that the VMs are not shown in the Infra Provider and shown only in the Cloud Provider , but the system and VIOS are shown in the Infra provider. Another option is , its shown in both providers when a drill down view is shown, however is reports such as total number of VMs etc, the duplicates needs to be handled.

Can you review and suggest options for the same?

Thanks

[Fryguy]

Oh this is interesting. Is it required to have both providers, or could you have HMC alone or PowerVC alone? I think we have a similar problem already with vsphere (infra) and vcloud (cloud)...I don't think we ever solved this.

Aside from the "must you hasve both or not" question, the other part of the problem is how end users are allowed to see things. That is, from an RBAC perspective, do you want the user to see the Infra side or the Cloud side? If the answer is "sometimes one or sometimes the other", I'm not sure how we could avoid having both available for presentation. Regardless I don't think we can not collect them. Things like events and operations must have both because the events need to link from the one side or the other.

Note that we do have a cross-provider linkage where we can say "this VM on provider X is this other VM on provider Y" - we use this for example to say that an OpenShift node is a VMware VM. This still keeps the two separate objects though, so it doesn't solve the double reporting. Maybe it's just something the report author must be aware of (whether manually, or maybe there's a tag that says that it's a duplicate that could be ignored in the report).

@agrare Thoughts?

[agrare]

This is a pretty unique situation where we have multiple providers that can be added individually or separately AND collect the same inventory. I've heard the vSphere / ESXi analogy brought up but I don't think that is accurate because if you add ESXi you'd get duplicate VMs as well. We typically just don't add ESXi hosts because vSphere lets you do everything through the VC.

The only similar case I can think of is VMware vSphere (InfraManager) + VMware NSX-t (NetworkManager), we attempt to link a Lan on the infra side to a CloudNetwork on the network side but both work in isolation if the other isn't added. But those are different models so you don't get the "duplicated" issue that you describe.

I think it'd be good to understand exactly what you want to avoid happening so we can make decisions on how best to approach it.
E.g.: do you want the HMC VMs to now show up in chargeback, not show up in the UI on "all vms" screens but do show up if you go directly to the HMC provider?

I think the simplest thing to do would be to simply not collect VMs from the HMC provider if it is paired with a PowerVC provider, are there operations on VMs that can only be done by the HMC API?

[hari-g-m]

@agrare @Fryguy - PowerVC requires HMC or Novalink to manage the system. So when PowerVC is added, it means that there is already HMC or Novalink managing the system. User need not add both, but PowerVC is focused on Virtualization management actions and there are certain specific actions at system/VIOS level which will be done at the HMC level.

There are also some cases , where some operations can be done from HMC but not from PowerVC, but for most of the common VM level operations, users should be able to perform it via PowerVC, if there is PowerVC provider , but need to look at option later to see if we want to allow operations from both, but in reporting total VMs , remove the duplicates.

Is it possible to still list the VMs , but not allow actions from the HMC provider and then in queries like total VMs etc, remove the duplicates? If we can do this, then in the future, specific actions as needed can be enabled. i.e. VMs will be in read only mode under the infrastructure provider , but actions will be allowed under the cloud provider.

Thanks.

[hari-g-m]

Is it possible to still list the VMs , but not allow actions from the HMC provider and then in queries like total VMs etc, remove the duplicates?

Not natively. In the database, they are not the same VM, even though we humans know they are the same VM. The reporting engine doesn't have that context and just reports on what is in the database. Same goes for the chargeback engine.

Yes, we will still have them in the DB and there is a UUID field for Systems and VMs which can be used to compare if its the same VM or system, so this has to be something at the API layer which needs to handle the duplicate systems or VMs.

[agrare]

@hari-g-m if a PowerVC provider is added, is there any need for the HMC provider to collect VMs? Is there anything you can do to a VM from HMC that you can't do from PowerVC?

[agrare]

The API layer is generic so we wouldn't want to be doing any specific power filtering logic there ideally

[hari-g-m]

@hari-g-m if a PowerVC provider is added, is there any need for the HMC provider to collect VMs? Is there anything you can do to a VM from HMC that you can't do from PowerVC?

There are some cases where certain actions are not supported, but for most part , its okay to probably not support actions from HMC when PowerVC is managing the same system. However, when a customer drills down into a HMC, they will still see the Virtual IO Servers (a special partition type which is primarily helping IO virtualization) and is not managed by PowerVC and also might get confused on why VMs are not listed in HMC.

[hari-g-m]

The API layer is generic so we wouldn't want to be doing any specific power filtering logic there ideally

Is there another place you recommend for this filtering ? We definitely need it before the API response.

[hari-g-m]

As a follow up to this, there is also the scenario of 2 HMCs managing the same system and both can be added as providers , in this case, will be 2 Infrastructure providers. We need a mechanism to remove the duplicates in that case as well. While we can pick one of the HMC as primary based on a criteria or have user set one, still need to filter the VMs in the reports.

[Fryguy]

How does that work on the Power side when there are 2 HMCs - couldn't you get into situations where, as an example, one HMC makes a decision to move a VM, then the other HMC decides to move it back, and it just flip-flops back and forth?

[hari-g-m]

How does that work on the Power side when there are 2 HMCs - couldn't you get into situations where, as an example, one HMC makes a decision to move a VM, then the other HMC decides to move it back, and it just flip-flops back and forth?

Its user initiated and the system is managed in an active-active mode. When one HMC makes a change, other HMC gets to know via events and updates its repository/inventory automatically, so doesn't move it back. User can use any of the HMC to make the changes as long as the right authorizations are available.

[agrare]

We could have multiple endpoints on a single HMC provider, that would prevent getting duplicate inventory by having 2(+?) HMC providers. We don't have any logic for any other providers today that handle active-active clusters without a single top-level manager though so some of the logic around e.g. credential verification

[Fryguy]

Actually duplicate endpoints could also act as a failover scenario.

@hari-g-m - why would a user want to add 2 endpoints as separate providers if they are active-active - Are they not full active-active or something?

[hari-g-m]

Actually duplicate endpoints could also act as a failover scenario.

@hari-g-m - why would a user want to add 2 endpoints as separate providers if they are active-active - Are they not full active-active or something?

They are fully active, its primarily for redundancy. Also, there might be some systems which are managed by both HMCs and some which are managed by 1 out of the 2 HMCs.

[Fryguy]

Here's my idea. If PowerVC requires an HMC, then I think we can go with a hybrid approach on the PowerVC side. That is, when someone adds a PowerVC, we have some way to also connect to the HMC (not necessarily a separate provider, but perhaps just a second connection on the PowerVC provider, similarly to how we add an OSV connection to an existing OpenShift provider). Then, from an inventory collection perspective, we'd watch both APIs and we merge the 2 inputs into a single VM. This way, we have both the PowerVC ems_ref and the HMC ems_ref (we'd have to store the latter in a different field unless they happen to be identical). With 2 connections we can pick and choose what's important from eac; events could be merged or if we don't need both just pick one. The user doesn't even really have to know what came from which side nor where we send a particular operation.

[hari-g-m]

@Fryguy @agrare - Attached is a view with the different possible combinations , as per our discussion.

To summarize the discussions so far at a high level

1. For Dual HMC managing the system, user can tag one set as backup and filter that out in reports for now. We need to enable the multiple endpoints approach , so that automatic redundancy can be handled.
2. For scenario where PowerVC is managing via HMC or Novalink, PowerVC will provide VM level management. HMC/NL will provide VIOS level management. In case, HMC is not controller, VIOS will still shpw up, but no actions can be done.