KVM memballooning requires free page reporting and autodeflate #11932

bradh352 · 2025-10-29T15:01:44Z

Description

As per #11930 KVM's memory ballooning does not auto-inflate and auto-deflate without the Free Page Reporting and autodeflate attributes on the memballoon configuration.

This is a change similar to OpenStack's NOVA:
openstack/nova@cd401c5

Basically if memballooning is enabled for KVM, these features are also always enabled. Having memory ballooning on in cloudstack otherwise does nothing as there is no tooling to inflate or deflate the balloon.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Not yet tested, hoping CI/CD runs through some self-tests, then I'll try a test deployment.

How did you try to break this feature and the system with this change?

N/A

plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java

DaanHoogland · 2025-10-29T16:49:29Z

@blueorangutan package

blueorangutan · 2025-10-29T16:50:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-10-29T16:55:37Z

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 17.56%. Comparing base (8089d32) to head (f43cdf9).
⚠️ Report is 160 commits behind head on main.

Files with missing lines	Patch %	Lines
...om/cloud/hypervisor/kvm/resource/LibvirtVMDef.java	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11932      +/-   ##
============================================
+ Coverage     17.36%   17.56%   +0.20%     
- Complexity    15245    15541     +296     
============================================
  Files          5888     5909      +21     
  Lines        525831   529061    +3230     
  Branches      64183    64618     +435     
============================================
+ Hits          91298    92924    +1626     
- Misses       424227   425683    +1456     
- Partials      10306    10454     +148

Flag	Coverage Δ
uitests	`3.58% <ø> (-0.05%)`	⬇️
unittests	`18.63% <75.00%> (+0.22%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bradh352 · 2025-10-29T17:59:06Z

pretty sure the simulator failure is unrelated to my changes

blueorangutan · 2025-10-29T18:05:40Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15585

weizhouapache

code lgtm, not tested

thanks @bradh352

DaanHoogland · 2025-10-30T07:43:08Z

@blueorangutan test keepEnv

blueorangutan · 2025-10-30T07:44:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-31T03:04:24Z

[SF] Trillian test result (tid-14747)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 65004 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11932-t14747-kvm-ol8.zip
Smoke tests completed. 148 look OK, 0 have errors, 1 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
all_test_human_readable_logs	`Skipped`	---	test_human_readable_logs.py

bradh352 · 2025-10-31T12:28:58Z

@weizhouapache looks like the smoke test passed

weizhouapache · 2025-10-31T12:56:17Z

@weizhouapache looks like the smoke test passed

good @bradh352
now let's wait someone to test it

bradh352 · 2025-10-31T13:01:21Z

@weizhouapache is there a package artifact generated by this I can download and use to test? I'm still getting familiar with cloudstack, I'd rather not figure out how to build .deb packages for it if I don't have to :)

weizhouapache · 2025-10-31T13:07:23Z

@weizhouapache is there a package artifact generated by this I can download and use to test? I'm still getting familiar with cloudstack, I'd rather not figure out how to build .deb packages for it if I don't have to :)

we do not provide the packages publicly, as the cloudstack-management-xx.deb package is huge

bernardodemarco · 2025-10-31T14:10:14Z

plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java

+            memBalloonBuilder.append("<memballoon model='" + memBalloonModel + "'");
+            /* Version integer format: major * 1,000,000 + minor * 1,000 + release.
+             * Require: libvirt 6.9.0, qemu 5.1.0 */
+            if (memBalloonModel != MemBalloonModel.NONE && s_qemuVersion >= 5001000 && s_libvirtVersion >= 6009000) {


@bradh352, thanks for the PR, code looks good. Just one minor suggestion, could you extract this comment into a method's Javadoc string? For instance:

cloudstack/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java

Lines 2178 to 2183 in 3d6cafe

/**

* Updates the parent on the {@code checkpointXml} to {@code snapshotParent}. If {@code snapshotParent} is null, removes the parent.

* @param checkpointXml the checkpoint XML to be updated

* @param snapshotParent the snapshot parent. Inform null if no parent.

* */

private void updateParent(String snapshotParent, Document checkpointXml, XPath xPath) throws XPathExpressionException {

@bernardodemarco I'm unfortunately not familiar with the specifics of javadoc so its not clear to me how to apply this to something other than a method itself. I can update it if you let me know how it needs to be formatted.

bernardodemarco · 2025-10-31T14:26:46Z

I'd rather not figure out how to build .deb packages for it if I don't have to :)

@bradh352, I personally use this DEB builder: https://github.com/scclouds/cloudstack-deb-builder.

By default, it does not populate the cloudstack-management DEB with the system VM template and, therefore, the resulting package is more lightweight. However, the ACS environment must have the correct system VM template configured before installing the PR's packages.

To build the packages, go to the Apache CloudStack source code root directory and execute:

docker run -v <code-path>:/mnt/build/cloudstack -v ~/.m2:/root/.m2 -e "USER_ID=$(id -u)" -e "USER_GID=$(id -g)" -e "ACS_BUILD_OPTS=-T <number-of-threads> -DskipTests" scclouds/cloudstack-deb-builder:ubuntu2004-jdk11-python3

Where:

<code-path> is the path of the Apache CloudStack source code directory; and,
<number-of-threads> is the number of threads that will be used to build the artifacts.

For example:

docker run -v ~/code/cloudstack:/mnt/build/cloudstack -v ~/.m2:/root/.m2 -e "USER_ID=$(id -u)" -e "USER_GID=$(id -g)" -e "ACS_BUILD_OPTS=-T 4 -DskipTests" scclouds/cloudstack-deb-builder:ubuntu2004-jdk11-python3

It may take a little bit longer for the first build process to finish, due to the download process of some dependencies. After it is finished, the DEB packages will be located inside the Apache CloudStack source code directory, at the /dist/debbuild/DEBS path.

Hope it helps!

bradh352 · 2025-12-01T23:43:55Z

I'd rather not figure out how to build .deb packages for it if I don't have to :)

@bradh352, I personally use this DEB builder: https://github.com/scclouds/cloudstack-deb-builder.

By default, it does not populate the cloudstack-management DEB with the system VM template and, therefore, the resulting package is more lightweight. However, the ACS environment must have the correct system VM template configured before installing the PR's packages.

To build the packages, go to the Apache CloudStack source code root directory and execute:
docker run -v <code-path>:/mnt/build/cloudstack -v ~/.m2:/root/.m2 -e "USER_ID=$(id -u)" -e "USER_GID=$(id -g)" -e "ACS_BUILD_OPTS=-T <number-of-threads> -DskipTests" scclouds/cloudstack-deb-builder:ubuntu2004-jdk11-python3
Where:

<code-path> is the path of the Apache CloudStack source code directory; and,

<number-of-threads> is the number of threads that will be used to build the artifacts.

For example:
docker run -v ~/code/cloudstack:/mnt/build/cloudstack -v ~/.m2:/root/.m2 -e "USER_ID=$(id -u)" -e "USER_GID=$(id -g)" -e "ACS_BUILD_OPTS=-T 4 -DskipTests" scclouds/cloudstack-deb-builder:ubuntu2004-jdk11-python3
It may take a little bit longer for the first build process to finish, due to the download process of some dependencies. After it is finished, the DEB packages will be located inside the Apache CloudStack source code directory, at the /dist/debbuild/DEBS path.

Hope it helps!

Thanks, I'm going to test this out tomorrow!

bhouse-nexthop · 2025-12-02T16:17:56Z

@bernardodemarco I did get .deb files generated using your method above like ./dist/debbuild/DEBS/cloudstack-agent_4.22.1.0-SNAPSHOT~focal_all.deb But given that the image is ubuntu 20.04 is there any problem trying to actually install and this on ubuntu 24.04? I didn't see a docker template for ubuntu 24.04 available.

bradh352 · 2025-12-03T13:02:10Z

Ok, I have a custom built 4.22 with this PR installed in my test environment.

virsh dumpxml of a running vm shows:

    <memballoon model='virtio' autodeflate='on' freePageReporting='on'>
      <stats period='10'/>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>

And in turn the generated qemu command line has this:

-device {"driver":"virtio-balloon-pci","id":"balloon0","deflate-on-oom":true,"free-page-reporting":true,"bus":"pci.0","addr":"0x6"}

VMs start up just fine. I'm going to now try to allocate a bunch of memory within a series of VMs and release it and inspect if the host is able to reclaim the memory.

bradh352 · 2025-12-03T14:31:50Z

Well, honestly the first test was enough to convince me it is working well:

Fresh VM (wihtin VM):

root@test1:~# free
               total        used        free      shared  buff/cache   available
Mem:        16369284      617328    15567232       11004      474740    15751956
Swap:              0           0           0

In Hypervisor:

root@node1:~# ps -o pid,rss,vsz,command -p 1417359
    PID   RSS    VSZ COMMAND
1417359 1408680 19634724 /usr/bin/qemu-system-x86_64 -name guest=i-2-304-VM,...

In VM, allocate a tmpfs mount and a large file:

root@test1:~# mount -t tmpfs tmpfs /mnt
root@test1:~# dd if=/dev/zero of=/mnt/bigfile bs=1M count=10000
dd: error writing '/mnt/bigfile': No space left on device
7993+0 records in
7992+0 records out
8381071360 bytes (8.4 GB, 7.8 GiB) copied, 3.95783 s, 2.1 GB/s
root@test1:~# free
               total        used        free      shared  buff/cache   available
Mem:        16369284     8805920     7369168     8195644     8678156     7563364
Swap:              0           0           0

In Hypervisor:

root@node1:~# ps -o pid,rss,vsz,command -p 1417359
    PID   RSS    VSZ COMMAND
1417359 9782676 19615192 /usr/bin/qemu-system-x86_64 -name guest=i-2-304-VM,...

Cleanup:

root@test1:~# rm -f /mnt/bigfile && umount /mnt
root@test1:~# free
               total        used        free      shared  buff/cache   available
Mem:        16369284      599236    15467600       11004      594464    15770048
Swap:              0           0           0

In Hypervisor:

root@node1:~# ps -o pid,rss,vsz,command -p 1417359
    PID   RSS    VSZ COMMAND
1417359 1523988 19610052 /usr/bin/qemu-system-x86_64 -name guest=i-2-304-VM,...

rajujith · 2025-12-04T04:50:26Z

@blueorangutan package

blueorangutan · 2025-12-04T04:52:03Z

@rajujith a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-04T06:21:09Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15913

use Free Page Reporting

d81c695

boring-cyborg bot added the component:kvm label Oct 29, 2025

bradh352 mentioned this pull request Oct 29, 2025

KVM Memballoon add support for Free Page Reporting and autodeflate #11930

Open

weizhouapache reviewed Oct 29, 2025

View reviewed changes

plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java Outdated Show resolved Hide resolved

bradh352 added 3 commits October 29, 2025 17:56

add qemu and libvirt version checks

c57377a

add test for old and new libvirt/qemu

f6a92da

revert back to older versions

f43cdf9

weizhouapache approved these changes Oct 30, 2025

View reviewed changes

DaanHoogland added the status:needs-testing label Oct 30, 2025

bernardodemarco reviewed Oct 31, 2025

View reviewed changes

DaanHoogland changed the base branch from main to 4.22 November 5, 2025 15:23

bhouse-nexthop mentioned this pull request Dec 4, 2025

mem.overprovisioning not working as expected. #7529

Closed

	/**
	* Updates the parent on the {@code checkpointXml} to {@code snapshotParent}. If {@code snapshotParent} is null, removes the parent.
	* @param checkpointXml the checkpoint XML to be updated
	* @param snapshotParent the snapshot parent. Inform null if no parent.
	* */
	private void updateParent(String snapshotParent, Document checkpointXml, XPath xPath) throws XPathExpressionException {

KVM memballooning requires free page reporting and autodeflate #11932

Are you sure you want to change the base?

KVM memballooning requires free page reporting and autodeflate #11932

Conversation

bradh352 commented Oct 29, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

Uh oh!

DaanHoogland commented Oct 29, 2025

Uh oh!

blueorangutan commented Oct 29, 2025

Uh oh!

codecov bot commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bradh352 commented Oct 29, 2025

Uh oh!

blueorangutan commented Oct 29, 2025

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Oct 30, 2025

Uh oh!

blueorangutan commented Oct 30, 2025

Uh oh!

blueorangutan commented Oct 31, 2025

Uh oh!

bradh352 commented Oct 31, 2025

Uh oh!

weizhouapache commented Oct 31, 2025

Uh oh!

bradh352 commented Oct 31, 2025

Uh oh!

weizhouapache commented Oct 31, 2025

Uh oh!

bernardodemarco Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

bradh352 Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

bernardodemarco commented Oct 31, 2025

Uh oh!

bradh352 commented Dec 1, 2025

Uh oh!

bhouse-nexthop commented Dec 2, 2025

Uh oh!

bradh352 commented Dec 3, 2025

Uh oh!

bradh352 commented Dec 3, 2025

Uh oh!

rajujith commented Dec 4, 2025

Uh oh!

blueorangutan commented Dec 4, 2025

Uh oh!

blueorangutan commented Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov bot commented Oct 29, 2025 •

edited

Loading