Skip to content

Conversation

PlaidCat
Copy link
Collaborator

@PlaidCat PlaidCat commented Jan 31, 2025

Update process (This kernel CentOS base for 4.18.0-553)

  • Kernel History Rebuild Process for all src.rpms hosted by RESF
  • Create sig-cloud-8/4.18.0-553.36.1.el8_10 branch
  • Check if any maintained code is included in the new el release.
  • Cherry-pick all code from previous branch into new branch (skipping unneeded code)
    • Fix conflicts as they arise
  • Build and Test

Removed Patches

  • None

BUILD

[maple@r8-sigcloud-builder kernel-src-tree]$ ../kernel-src-tree-tools/kernel_build.sh
/mnt/code/kernel-src-tree
no .config file found, moving on
[TIMER]{MRPROPER}: 0s
x86_64 architecture detected, copying config
'configs/kernel-x86_64.config' -> '.config'
Setting Local Version for build
CONFIG_LOCALVERSION="-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10"
Making olddefconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  YACC    scripts/kconfig/zconf.tab.c
  LEX     scripts/kconfig/zconf.lex.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --olddefconfig Kconfig
#
# configuration written to .config
#
Starting Build
scripts/kconfig/conf  --syncconfig Kconfig
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h

[SNIP]

  LD [M]  sound/x86/snd-hdmi-lpe-audio.ko
  LD [M]  sound/xen/snd_xen_front.ko
  LD [M]  virt/lib/irqbypass.ko
[TIMER]{BUILD}: 1791s
Making Modules
  INSTALL arch/x86/crypto/blowfish-x86_64.ko
  INSTALL arch/x86/crypto/camellia-aesni-avx-x86_64.ko
  INSTALL arch/x86/crypto/camellia-aesni-avx2.ko

[SNIP]

  INSTALL sound/xen/snd_xen_front.ko
  INSTALL virt/lib/irqbypass.ko
  DEPMOD  4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+
[TIMER]{MODULES}: 40s
Making Install
sh ./arch/x86/boot/install.sh 4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+ arch/x86/boot/bzImage \
	System.map "/boot"
[TIMER]{INSTALL}: 20s
Checking kABI
kABI check passed
Setting Default Kernel to /boot/vmlinuz-4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+ and Index to 1
The default is /boot/loader/entries/8b9f9fc76d3e4375b3234b6633e9504d-4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+.conf with index 1 and kernel /boot/vmlinuz-4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+
The default is /boot/loader/entries/8b9f9fc76d3e4375b3234b6633e9504d-4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+.conf with index 1 and kernel /boot/vmlinuz-4.18.0-_jmaple__sig-cloud-8_4.18.0-553.36.1.el8_10+
Generating grub configuration file ...
done
Hopefully Grub2.0 took everything ... rebooting after time metrices
[TIMER]{MRPROPER}: 0s
[TIMER]{BUILD}: 1791s
[TIMER]{MODULES}: 40s
[TIMER]{INSTALL}: 20s
[TIMER]{TOTAL} 1855s

LOGIN

[maple@r8-sigcloud-builder code]$ uname -a
Linux r8-sigcloud-builder 4.18.0-sig-cloud-8_4.18.0-553.36.1.el8_10+ #1 SMP Tue Jan 28 23:42:07 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Testing

Screenshot 2025-01-31 at 4 02 20 PM

gvrose8192 and others added 6 commits January 28, 2025 18:13
jira LE-1482
commit 415d832
upstream-diff - Conflicts around use of arch_atomic vs atomic - there
are no arch_atomic* ops in Rocky 8.x

    These operations are documented as always ordered in
    include/asm-generic/bitops/instrumented-atomic.h, and producer-consumer
    type use cases where one side needs to ensure a flag is left pending
    after some shared data was updated rely on this ordering, even in the
    failure case.

    This is the case with the workqueue code, which currently suffers from a
    reproducible ordering violation on Apple M1 platforms (which are
    notoriously out-of-order) that ends up causing the TTY layer to fail to
    deliver data to userspace properly under the right conditions.  This
    change fixes that bug.

    Change the documentation to restrict the "no order on failure" story to
    the _lock() variant (for which it makes sense), and remove the
    early-exit from the generic implementation, which is what causes the
    missing barrier semantics in that case.  Without this, the remaining
    atomic op is fully ordered (including on ARM64 LSE, as of recent
    versions of the architecture spec).

    Suggested-by: Linus Torvalds <[email protected]>
    Cc: [email protected]
    Fixes: e986a0d ("locking/atomics, asm-generic/bitops/atomic.h: Rewrite using atomic_*() APIs")
    Fixes: 61e0239 ("locking/atomic/bitops: Document and clarify ordering semantics for failed test_and_{}_bit()")
    Signed-off-by: Hector Martin <[email protected]>
    Acked-by: Will Deacon <[email protected]>
    Reviewed-by: Arnd Bergmann <[email protected]>
    Signed-off-by: Linus Torvalds <[email protected]>
    (cherry picked from commit 415d832)

Signed-off-by: Greg Rose <[email protected]>
Signed-off-by: Jonathan Maple <[email protected]>
…tead of a two-phase approach

jira roc-2673
commit fbf6449

Instead of setting x86_virt_bits to a possibly-correct value and then
correcting it later, do all the necessary checks before setting it.

At this point, the #VC handler references boot_cpu_data.x86_virt_bits,
and in the previous version, it would be triggered by the CPUIDs between
the point at which it is set to 48 and when it is set to the correct
value.

    Suggested-by: Dave Hansen <[email protected]>
    Signed-off-by: Adam Dunlap <[email protected]>
    Signed-off-by: Ingo Molnar <[email protected]>
    Tested-by: Jacob Xu <[email protected]>
    Link: https://lore.kernel.org/r/[email protected]

Signed-off-by: Ronnie Sahlberg <[email protected]>
Signed-off-by: Jonathan Maple <[email protected]>
jira roc-2673
commit 3e32552

c->x86_cache_alignment is initialized from c->x86_clflush_size.
However, commit fbf6449 moved c->x86_clflush_size initialization
to later in boot without moving the c->x86_cache_alignment assignment:

  fbf6449 ("x86/sev-es: Set x86_virt_bits to the correct value straight away, instead of a two-phase approach")

This presumably left c->x86_cache_alignment set to zero for longer
than it should be.

The result was an oops on 32-bit kernels while accessing a pointer
at 0x20.  The 0x20 came from accessing a structure member at offset
0x10 (buffer->cpumask) from a ZERO_SIZE_PTR=0x10.  kmalloc() can
evidently return ZERO_SIZE_PTR when it's given 0 as its alignment
requirement.

Move the c->x86_cache_alignment initialization to be after
c->x86_clflush_size has an actual value.

    Fixes: fbf6449 ("x86/sev-es: Set x86_virt_bits to the correct value straight away, instead of a two-phase approach")
    Signed-off-by: Dave Hansen <[email protected]>
    Signed-off-by: Ingo Molnar <[email protected]>
    Tested-by: Nathan Chancellor <[email protected]>
    Link: https://lore.kernel.org/r/[email protected]
    (cherry picked from commit 3e32552)
Signed-off-by: Ronnie Sahlberg <[email protected]>

Signed-off-by: Jonathan Maple <[email protected]>
jira LE-2183
bug-fix x86/sev-es: Set x86_virt_bits
commit-author Paolo Bonzini <[email protected]>
commit 9a45819

In commit fbf6449 ("x86/sev-es: Set x86_virt_bits to the correct
value straight away, instead of a two-phase approach"), the initialization
of c->x86_phys_bits was moved after this_cpu->c_early_init(c).  This is
incorrect because early_init_amd() expected to be able to reduce the
value according to the contents of CPUID leaf 0x8000001f.

Fortunately, the bug was negated by init_amd()'s call to early_init_amd(),
which does reduce x86_phys_bits in the end.  However, this is very
late in the boot process and, most notably, the wrong value is used for
x86_phys_bits when setting up MTRRs.

To fix this, call get_cpu_address_sizes() as soon as X86_FEATURE_CPUID is
set/cleared, and c->extended_cpuid_level is retrieved.

Fixes: fbf6449 ("x86/sev-es: Set x86_virt_bits to the correct value straight away, instead of a two-phase approach")
	Signed-off-by: Paolo Bonzini <[email protected]>
	Signed-off-by: Dave Hansen <[email protected]>
	Cc:[email protected]
Link: https://lore.kernel.org/all/20240131230902.1867092-2-pbonzini%40redhat.com
(cherry picked from commit 9a45819)
	Signed-off-by: Jonathan Maple <[email protected]>
Signed-off-by: Jonathan Maple <[email protected]>
…sizes()

jira LE-2183
bug-fix-prereq x86/sev-es: Set x86_virt_bits
commit-author Borislav Petkov (AMD) <[email protected]>
commit 95bfb35

Drop 'vp_bits_from_cpuid' as it is not really needed.

No functional changes.

	Signed-off-by: Borislav Petkov (AMD) <[email protected]>
	Signed-off-by: Ingo Molnar <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
(cherry picked from commit 95bfb35)
	Signed-off-by: Jonathan Maple <[email protected]>
Signed-off-by: Jonathan Maple <[email protected]>
jira LE-2183
bug-fix x86/sev-es: Set x86_virt_bits
commit-author Dave Hansen <[email protected]>
commit 2a38e4c

tl;dr: CPUs with CPUID.80000008H but without CPUID.01H:EDX[CLFSH]
will end up reporting cache_line_size()==0 and bad things happen.
Fill in a default on those to avoid the problem.

Long Story:

The kernel dies a horrible death if c->x86_cache_alignment (aka.
cache_line_size() is 0.  Normally, this value is populated from
c->x86_clflush_size.

Right now the code is set up to get c->x86_clflush_size from two
places.  First, modern CPUs get it from CPUID.  Old CPUs that don't
have leaf 0x80000008 (or CPUID at all) just get some sane defaults
from the kernel in get_cpu_address_sizes().

The vast majority of CPUs that have leaf 0x80000008 also get
->x86_clflush_size from CPUID.  But there are oddballs.

Intel Quark CPUs[1] and others[2] have leaf 0x80000008 but don't set
CPUID.01H:EDX[CLFSH], so they skip over filling in ->x86_clflush_size:

	cpuid(0x00000001, &tfms, &misc, &junk, &cap0);
	if (cap0 & (1<<19))
		c->x86_clflush_size = ((misc >> 8) & 0xff) * 8;

So they: land in get_cpu_address_sizes() and see that CPUID has level
0x80000008 and jump into the side of the if() that does not fill in
c->x86_clflush_size.  That assigns a 0 to c->x86_cache_alignment, and
hilarity ensues in code like:

        buffer = kzalloc(ALIGN(sizeof(*buffer), cache_line_size()),
                         GFP_KERNEL);

To fix this, always provide a sane value for ->x86_clflush_size.

Big thanks to Andy Shevchenko for finding and reporting this and also
providing a first pass at a fix. But his fix was only partial and only
worked on the Quark CPUs.  It would not, for instance, have worked on
the QEMU config.

1. https://raw.githubusercontent.com/InstLatx64/InstLatx64/master/GenuineIntel/GenuineIntel0000590_Clanton_03_CPUID.txt
2. You can also get this behavior if you use "-cpu 486,+clzero"
   in QEMU.

[ dhansen: remove 'vp_bits_from_cpuid' reference in changelog
	   because bpetkov brutally murdered it recently. ]

Fixes: fbf6449 ("x86/sev-es: Set x86_virt_bits to the correct value straight away, instead of a two-phase approach")
	Reported-by: Andy Shevchenko <[email protected]>
	Signed-off-by: Dave Hansen <[email protected]>
	Tested-by: Andy Shevchenko <[email protected]>
	Tested-by: Jörn Heusipp <[email protected]>
	Cc: [email protected]
Link: https://lore.kernel.org/all/[email protected]/
Link: https://lore.kernel.org/lkml/[email protected]/
Link: https://lore.kernel.org/all/20240517200534.8EC5F33E%40davehans-spike.ostc.intel.com
(cherry picked from commit 2a38e4c)
	Signed-off-by: Jonathan Maple <[email protected]>
Signed-off-by: Jonathan Maple <[email protected]>
Copy link

@gvrose8192 gvrose8192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builds and installs for me + passes kABI check. The patches are already well identified and in use for previous releases.
Thanks!

Copy link
Collaborator

@bmastbergen bmastbergen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥌

@PlaidCat PlaidCat merged commit b2097ac into sig-cloud-8/4.18.0-553.36.1.el8_10 Feb 3, 2025
2 checks passed
@PlaidCat PlaidCat deleted the {jmaple}_sig-cloud-8/4.18.0-553.36.1.el8_10 branch February 3, 2025 17:15
github-actions bot pushed a commit that referenced this pull request Oct 5, 2025
When running with transparent huge pages and CONFIG_NVHE_EL2_DEBUG then
the debug checking in assert_host_shared_guest() fails on the launch of an
np-guest. This WARN_ON() causes a panic and generates the stack below.

In __pkvm_host_relax_perms_guest() the debug checking assumes the mapping
is a single page but it may be a block map. Update the checking so that
the size is not checked and just assumes the correct size.

While we're here make the same fix in __pkvm_host_mkyoung_guest().

  Info: # lkvm run -k /share/arch/arm64/boot/Image -m 704 -c 8 --name guest-128
  Info: Removed ghost socket file "/.lkvm//guest-128.sock".
[ 1406.521757] kvm [141]: nVHE hyp BUG at: arch/arm64/kvm/hyp/nvhe/mem_protect.c:1088!
[ 1406.521804] kvm [141]: nVHE call trace:
[ 1406.521828] kvm [141]:  [<ffff8000811676b4>] __kvm_nvhe_hyp_panic+0xb4/0xe8
[ 1406.521946] kvm [141]:  [<ffff80008116d12c>] __kvm_nvhe_assert_host_shared_guest+0xb0/0x10c
[ 1406.522049] kvm [141]:  [<ffff80008116f068>] __kvm_nvhe___pkvm_host_relax_perms_guest+0x48/0x104
[ 1406.522157] kvm [141]:  [<ffff800081169df8>] __kvm_nvhe_handle___pkvm_host_relax_perms_guest+0x64/0x7c
[ 1406.522250] kvm [141]:  [<ffff800081169f0c>] __kvm_nvhe_handle_trap+0x8c/0x1a8
[ 1406.522333] kvm [141]:  [<ffff8000811680fc>] __kvm_nvhe___skip_pauth_save+0x4/0x4
[ 1406.522454] kvm [141]: ---[ end nVHE call trace ]---
[ 1406.522477] kvm [141]: Hyp Offset: 0xfffece8013600000
[ 1406.522554] Kernel panic - not syncing: HYP panic:
[ 1406.522554] PS:834003c9 PC:0000b1806db6d170 ESR:00000000f2000800
[ 1406.522554] FAR:ffff8000804be420 HPFAR:0000000000804be0 PAR:0000000000000000
[ 1406.522554] VCPU:0000000000000000
[ 1406.523337] CPU: 3 UID: 0 PID: 141 Comm: kvm-vcpu-0 Not tainted 6.16.0-rc7 #97 PREEMPT
[ 1406.523485] Hardware name: FVP Base RevC (DT)
[ 1406.523566] Call trace:
[ 1406.523629]  show_stack+0x18/0x24 (C)
[ 1406.523753]  dump_stack_lvl+0xd4/0x108
[ 1406.523899]  dump_stack+0x18/0x24
[ 1406.524040]  panic+0x3d8/0x448
[ 1406.524184]  nvhe_hyp_panic_handler+0x10c/0x23c
[ 1406.524325]  kvm_handle_guest_abort+0x68c/0x109c
[ 1406.524500]  handle_exit+0x60/0x17c
[ 1406.524630]  kvm_arch_vcpu_ioctl_run+0x2e0/0x8c0
[ 1406.524794]  kvm_vcpu_ioctl+0x1a8/0x9cc
[ 1406.524919]  __arm64_sys_ioctl+0xac/0x104
[ 1406.525067]  invoke_syscall+0x48/0x10c
[ 1406.525189]  el0_svc_common.constprop.0+0x40/0xe0
[ 1406.525322]  do_el0_svc+0x1c/0x28
[ 1406.525441]  el0_svc+0x38/0x120
[ 1406.525588]  el0t_64_sync_handler+0x10c/0x138
[ 1406.525750]  el0t_64_sync+0x1ac/0x1b0
[ 1406.525876] SMP: stopping secondary CPUs
[ 1406.525965] Kernel Offset: disabled
[ 1406.526032] CPU features: 0x0000,00000080,8e134ca1,9446773f
[ 1406.526130] Memory Limit: none
[ 1406.959099] ---[ end Kernel panic - not syncing: HYP panic:
[ 1406.959099] PS:834003c9 PC:0000b1806db6d170 ESR:00000000f2000800
[ 1406.959099] FAR:ffff8000804be420 HPFAR:0000000000804be0 PAR:0000000000000000
[ 1406.959099] VCPU:0000000000000000 ]

Signed-off-by: Ben Horgan <[email protected]>
Fixes: f28f1d0 ("KVM: arm64: Add a range to __pkvm_host_unshare_guest()")
Cc: Vincent Donnefort <[email protected]>
Cc: Quentin Perret <[email protected]>
Cc: Ryan Roberts <[email protected]>
Cc: [email protected]
Reviewed-by: Vincent Donnefort <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants