Skip to content

Commit 120b1a2

Browse files
committed
CA-423172: Xen uses ~294 pages/vCPU, not 256
Measured the actual increase in host memory usage when increasing the number of vCPUs on a VM from 1 to 64: ``` vcpu,memory_overhead_pages,coeff 1,264,264 2,558,279 3,776,258.667 4,1032,258 5,1350,270 6,1614,269 7,1878,268.286 8,2056,257 9,2406,267.333 10,2670,267 11,2934,266.727 12,3198,266.5 13,3462,266.308 14,3726,266.143 15,3990,266 16,4254,265.875 17,4518,265.765 18,4782,265.667 19,5046,265.579 20,5310,265.5 21,5574,265.429 22,5838,265.364 23,6102,265.304 24,6366,265.25 25,6630,265.2 26,6894,265.154 27,7158,265.111 28,7422,265.071 29,7686,265.034 30,7952,265.067 31,8216,265.032 32,8480,265 33,8744,264.97 34,9009,264.971 35,9276,265.029 36,9543,265.083 37,9810,265.135 38,10076,265.158 39,10340,265.128 40,10604,265.1 41,10869,265.098 42,11133,265.071 43,11397,265.047 44,11662,265.045 45,11925,265 46,12191,265.022 47,12454,264.979 0,30,0 1,294,294 2,558,279 3,822,274 4,1086,271.5 5,1350,270 6,1614,269 7,1878,268.286 8,2142,267.75 9,2406,267.333 10,2670,267 11,2934,266.727 12,3198,266.5 13,3462,266.308 14,3726,266.143 15,3990,266 16,4254,265.875 17,4518,265.765 18,4782,265.667 19,5046,265.579 20,5310,265.5 21,5574,265.429 22,5838,265.364 23,6102,265.304 24,6366,265.25 25,6630,265.2 26,6894,265.154 27,7158,265.111 28,7422,265.071 29,7686,265.034 30,7952,265.067 31,8216,265.032 32,8480,265 33,8744,264.97 34,9011,265.029 35,9278,265.086 36,9546,265.167 37,9811,265.162 38,10076,265.158 39,10340,265.128 40,10603,265.075 41,10869,265.098 42,11132,265.048 43,11397,265.047 44,11663,265.068 45,11925,265 46,12191,265.022 47,12456,265.021 [INFO]VM memory_overhead_pages = ... + vcpu * 294 =~ ... + vcpu * 294 ``` We already allocate 256 pages/vcpu as part of shadow, so we need an extra 294-256=38 pages/vcpu. This can lead to internal errors raised by xenguest, or NOT_ENOUGH_FREE_MEMORY errors raised by xenopsd, after `assert_can_boot_here` has already replied yes, even when booting VMs sequentially. It could also lead XAPI to choose the wrong host to evacuate a VM too, which could lead to RPU migration failures. This is a pre-existing bug, affecting both the versions of Xen in XS8 and XS9. Cannot allocate this from shadow, because otherwise the memory usage would never converge (Xen doesn't allocate these from shadow). On another host the measured overhead is less, take the maximum for now: ``` [INFO]VM memory_overhead_pages = ... + vcpu * 264.067 =~ ... + vcpu * 265 ``` Also the amount of shadow memory reserved is nearly twice as much as needed, especially that shadow is compiled out of Xen, but overestimates are OK, and we might fix that separately. Signed-off-by: Edwin Török <edwin.torok@citrix.com>
1 parent 4edb0d7 commit 120b1a2

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

ocaml/xapi-idl/memory/memory.ml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,8 @@ module Memory_model (D : MEMORY_MODEL_DATA) = struct
218218
D.extra_internal_mib
219219
+++ D.extra_external_mib
220220
+++ shadow_mib static_max_mib vcpu_count multiplier
221+
(* CA-423172: Xen uses some extra memory/vCPU *)
222+
+++ mib_of_pages_used (38L *** Int64.of_int vcpu_count)
221223
+++ D.shim_mib static_max_mib
222224

223225
let footprint_mib target_mib static_max_mib vcpu_count multiplier =

0 commit comments

Comments
 (0)