Synchronise 2023.1 with upstream #153
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
At present, we can modify the properties in the instance system_metadata through the sub command image_property of nova-manage, but there may be inconsistencies between their values and those in request_specs. And the migration is based on request_specs, so the same image properties are also written to request_specs. Closes-Bug: 2078999 Change-Id: Id36ecd022cb6f7f9a0fb131b0d202b79715870a9 (cherry picked from commit 2a1fad4) (cherry picked from commit ebae97c) (cherry picked from commit ee30457)
This change reproduces a bug in the db load of old instance numa toplogy json blobs in _migrate_legacy_dedicated_instance_cpuset that failed to account for defaulting pcpuset to the empty set when it is not in the json blob. Related-Bug: #2080556 Change-Id: Ia0f327c501f65786d5b2538b2742ec2786486956 (cherry picked from commit 521db4a) (cherry picked from commit c00b76b) (cherry picked from commit 7ae3afe)
This change ensures that if we are upgrading a InstanceNUMACell object created before victoria <1.5 that we properly set pcpuset=set() when loading the object form the db. This is requried to support instances with a numa topology that do not use cpu pinning. Depends-On: https://review.opendev.org/c/openstack/python-openstackclient/+/929236 Closes-Bug: #2080556 Change-Id: Iea55aabe71c250d8c8e93c61421450b909a7fa3d (cherry picked from commit 2a87032) (cherry picked from commit 13c780a) (cherry picked from commit e7d61fe)
In Victoria InstanceNUMACell ovo got the new pcpuset field and a connected in place data migration. If the cpu_policy is dedicated the content of the cpuset is moved to to the new pcpuset field during the load of the data from the DB and the ovo is persisted back to the DB. However during this data migration the version of the ovo is not bumped to the latest, 1.6, version. Therefore the DB contains an inconsistent object as it has the new pcpuset field from 1.6 but the nova_object.version is still set to 1.4. It turned out that this can cause that an old compute node having ovo 1.4 code will not request a back levelling of the ovo even if it is already data migrated to 1.6 causing data loss from the compute perspective. Also if the compute saves the object back to DB then the data loss becomes persistent. Related-Bug: #2097359 Change-Id: I76ee9d59abc39e29c54be7217491e911b88a0621 (cherry picked from commit ae2f9bd) (cherry picked from commit 82b5bd6) (cherry picked from commit a3d3b89) (cherry picked from commit db9a810)
The data migration of InstanceNUMACell 1.4 to 1.5 only moved the data to the new pcpuset field but does not update the ovo version string of the object in the DB. This resulted in an 1.6 data with an 1.4 version string in the DB which subsequently causes that an old compute running 1.4 ovo version will think it got an old ovo even though the data is already in the new format. This leads to instance lifecycle errors and if the nova-compute saves the instance then it also leads to permanent data loss. So this change modified the data migration to also update the ovo version in the DB so that the version string in the DB matches the schema the data uses in the DB. Related-Bug: #2097359 Change-Id: I9a99f10526f8e466ac04b035121b24be70a23dae (cherry picked from commit 643a6a8) (cherry picked from commit 7ff108b) (cherry picked from commit f6be113) (cherry picked from commit 2ffc00c)
The data migration of InstanceNUMACell 1.4 to 1.5 only moved the data to the new pcpuset field but does not update the ovo version string of the object in the DB. The previous patch added the missing version update logic. However it only fixes the issue if the data is not already "half" migrated to the new structure. So this patch adds logic to also do the right thing if the wrong data migration already happened. At the end the solution needs to consider multiple scenarios: * data is never migrated to the new schema so the new code needs to migrate it and update the version string to match the new schema. (done by the previous patch) * data is half migrated by the buggy code and the new code need to finish the migration by stamping the version in the DB. * data is half migrated and then further modified to use the new 1.6 feature cpu_policy mixed. * data version is older in the DB than we can meaningfully upgrade Closes-Bug: #2097359 Change-Id: I10ecfa7841b15637dea3e4736e90faa5f33ddff3 (cherry picked from commit 9507b7b) (cherry picked from commit c008126) (cherry picked from commit 882c5eb) (cherry picked from commit e3db83c)
Change I60d6f04d374e9ede5895a43b7a75e955b0fea3c5 added tpool.Proxy
wrapping to the listDevices() and listAllDevices() methods but
introduced a regression for listDevices() that led to an error in
update_available_resource():
TypeError: virNodeDeviceLookupByName() argument 2 must be str or
None, not Proxy
The error was not caught by unit or functional testing because those
test environments intentionally do not import the libvirt Python
module -- so mocked code in the LibvirtFixture runs instead. Also, the
update_available_resource() method has a 'except Exception:' at the end
which logs an error but does not re-raise. So it would not cause a
functional test to fail.
This adds a functional test to reproduce the bug and adds a keyword arg
to the test _run_periodics() method to specify whether it should raise
an exception if an error is logged.
Related-Bug: #2098892
Change-Id: I3a3dda57f2181b24bd6589ac7bb8160014ab2396
(cherry picked from commit 3cf6667)
(cherry picked from commit 173defb)
(cherry picked from commit f516a0c)
(cherry picked from commit c9bc74e)
This a partial revert of change
I60d6f04d374e9ede5895a43b7a75e955b0fea3c5 which added tpool.Proxy
wrapping to the listDevices() and listAllDevices() methods.
The regression was caught during downstream testing with vGPUs and the
update_available_resource() periodic task was failing with:
TypeError: virNodeDeviceLookupByName() argument 2 must be str or
None, not Proxy
It turns out that while the listAllDevices() method returns a list of
virNodeDevice objects [1], the listDevices() method returns a list of
string names [2] and is generated from the corresponding function in C
[3].
The error was not caught by unit or functional testing because those
test environments intentionally do not import the libvirt Python
module -- so mocked code in the LibvirtFixture runs instead. Also, the
update_available_resource() method has a 'except Exception:' at the end
which logs an error but does not re-raise. So it would not cause a
functional test to fail.
This reverts the change that caused the regression, updates potentially
confusing docstrings, adds type annotations to the methods that use
listDevices(), and moves the nodeDeviceLookupByName type checking into
the LibvirtFixture.
Closes-Bug: #2098892
[1] https://github.com/libvirt/libvirt-python/blob/408815a/libvirt-override-virConnect.py#L520-L524
[2] https://github.com/libvirt/libvirt-python/blob/408815a/libvirt-override-api.xml#L448-L453
[3] https://libvirt.org/html/libvirt-libvirt-nodedev.html#virNodeListDevices
Change-Id: Ib5befdd3c13367daa208ff969f66cba693ae2c76
(cherry picked from commit 2c07aa0)
(cherry picked from commit c4f4ae7)
(cherry picked from commit b8bfa1e)
(cherry picked from commit c446736)
non-SLURP branches are EOL'd in case they reach their end of maintained phase. This could produce a situation when a patch is merged in a non-SLURP branch that was deleted in the meantime and it's further backports fail on gate with backport validator as the hash of the non-SLURP version of the patch is not on any branch. This patch fixes the above issue as follows: in case a hash is not found on any branch, then it checks if it can be found under any *-eol tag and only fails if there is not found either. Change-Id: I56705bce8ee4354cd5cb1577a520c2d1c525f57b (cherry picked from commit e383b46) (cherry picked from commit 8b0ae72) (cherry picked from commit 88e49dd) (cherry picked from commit db438e5)
The method get_root_info sometimes receives a BlockDeviceMapping object, which lacks a copy method. The previous code assumed root_bdm was always an instance of DriverBlockDevice, a subclass of dict that supports the copy() method. However, during testing, it was discovered that root_bdm could also be a BlockDeviceMapping object, which does not have a copy method. To address this, the change replaces the copy() call with copy.deepcopy() according to the suggestion in the comment [1], which works for both BlockDeviceMapping and DriverBlockDevice instances. The deepcopy method is supported because oslo.versionedobjects implements the __deepcopy__ method. This change ensures the function handles both object types correctly, preventing the AttributeError observed during testing. [1] https://review.opendev.org/c/openstack/nova/+/909611/4/nova/virt/libvirt/blockinfo.py Change-Id: I9432718586855ff57e8e6a5cae064e0685dd01e8 (cherry picked from commit 065bf99) Signed-off-by: Zhang Hua <[email protected]> (cherry picked from commit 9ff4953) (cherry picked from commit 608a73e)
…to unmaintained/2023.1
… unmaintained/2023.1
This change updates bdm for swap in finish_resize and revert_resize functionality. Change also adds supporting: - unit and functional tests - docs: releasenote Closes-Bug: #1552777 Change-Id: Ic8aaa0728a43936cd4c6e1ed590e01ba8f0fbf5b (cherry picked from commit 15dccae) (cherry picked from commit 3d3ada4) (cherry picked from commit 39714c2) Signed-off-by: Elod Illes <[email protected]>
github-actions
bot
added
automated
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains a snapshot of 2023.1 from upstream unmaintained/2023.1.