Skip to content

Commit 0238d3c

Browse files
torvaldstorvalds
committed
Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon: "The changes are a real mixed bag this time around. The only scary looking one from the diffstat is the uapi change to asm-generic/mman-common.h, but this has been acked by Arnd and is actually just adding a pair of comments in an attempt to prevent allocation of some PROT values which tend to get used for arch-specific purposes. We'll be using them for Branch Target Identification (a CFI-like hardening feature), which is currently under review on the mailing list. New architecture features: - Support for Armv8.5 E0PD, which benefits KASLR in the same way as KPTI but without the overhead. This allows KPTI to be disabled on CPUs that are not affected by Meltdown, even is KASLR is enabled. - Initial support for the Armv8.5 RNG instructions, which claim to provide access to a high bandwidth, cryptographically secure hardware random number generator. As well as exposing these to userspace, we also use them as part of the KASLR seed and to seed the crng once all CPUs have come online. - Advertise a bunch of new instructions to userspace, including support for Data Gathering Hint, Matrix Multiply and 16-bit floating point. Kexec: - Cleanups in preparation for relocating with the MMU enabled - Support for loading crash dump kernels with kexec_file_load() Perf and PMU drivers: - Cleanups and non-critical fixes for a couple of system PMU drivers FPU-less (aka broken) CPU support: - Considerable fixes to support CPUs without the FP/SIMD extensions, including their presence in heterogeneous systems. Good luck finding a 64-bit userspace that handles this. Modern assembly function annotations: - Start migrating our use of ENTRY() and ENDPROC() over to the new-fangled SYM_{CODE,FUNC}_{START,END} macros, which are intended to aid debuggers Kbuild: - Cleanup detection of LSE support in the assembler by introducing 'as-instr' - Remove compressed Image files when building clean targets IP checksumming: - Implement optimised IPv4 checksumming routine when hardware offload is not in use. An IPv6 version is in the works, pending testing. Hardware errata: - Work around Cortex-A55 erratum #1530923 Shadow call stack: - Work around some issues with Clang's integrated assembler not liking our perfectly reasonable assembly code - Avoid allocating the X18 register, so that it can be used to hold the shadow call stack pointer in future ACPI: - Fix ID count checking in IORT code. This may regress broken firmware that happened to work with the old implementation, in which case we'll have to revert it and try something else - Fix DAIF corruption on return from GHES handler with pseudo-NMIs Miscellaneous: - Whitelist some CPUs that are unaffected by Spectre-v2 - Reduce frequency of ASID rollover when KPTI is compiled in but inactive - Reserve a couple of arch-specific PROT flags that are already used by Sparc and PowerPC and are planned for later use with BTI on arm64 - Preparatory cleanup of our entry assembly code in preparation for moving more of it into C later on - Refactoring and cleanup" * tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (73 commits) arm64: acpi: fix DAIF manipulation with pNMI arm64: kconfig: Fix alignment of E0PD help text arm64: Use v8.5-RNG entropy for KASLR seed arm64: Implement archrandom.h for ARMv8.5-RNG arm64: kbuild: remove compressed images on 'make ARCH=arm64 (dist)clean' arm64: entry: Avoid empty alternatives entries arm64: Kconfig: select HAVE_FUTEX_CMPXCHG arm64: csum: Fix pathological zero-length calls arm64: entry: cleanup sp_el0 manipulation arm64: entry: cleanup el0 svc handler naming arm64: entry: mark all entry code as notrace arm64: assembler: remove smp_dmb macro arm64: assembler: remove inherit_daif macro ACPI/IORT: Fix 'Number of IDs' handling in iort_id_map() mm: Reserve asm-generic prot flags 0x10 and 0x20 for arch use arm64: Use macros instead of hard-coded constants for MAIR_EL1 arm64: Add KRYO{3,4}XX CPU cores to spectre-v2 safe list arm64: kernel: avoid x18 in __cpu_soft_restart arm64: kvm: stop treating register x18 as caller save arm64/lib: copy_page: avoid x18 register in assembler code ...
2 parents d5226fa + e533dbe commit 0238d3c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

94 files changed

+1315
-515
lines changed

‎Documentation/arm64/cpu-feature-registers.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,8 @@ infrastructure:
117117
+------------------------------+---------+---------+
118118
| Name | bits | visible |
119119
+------------------------------+---------+---------+
120+
| RNDR | [63-60] | y |
121+
+------------------------------+---------+---------+
120122
| TS | [55-52] | y |
121123
+------------------------------+---------+---------+
122124
| FHM | [51-48] | y |
@@ -200,6 +202,12 @@ infrastructure:
200202
+------------------------------+---------+---------+
201203
| Name | bits | visible |
202204
+------------------------------+---------+---------+
205+
| I8MM | [55-52] | y |
206+
+------------------------------+---------+---------+
207+
| DGH | [51-48] | y |
208+
+------------------------------+---------+---------+
209+
| BF16 | [47-44] | y |
210+
+------------------------------+---------+---------+
203211
| SB | [39-36] | y |
204212
+------------------------------+---------+---------+
205213
| FRINTTS | [35-32] | y |
@@ -234,10 +242,18 @@ infrastructure:
234242
+------------------------------+---------+---------+
235243
| Name | bits | visible |
236244
+------------------------------+---------+---------+
245+
| F64MM | [59-56] | y |
246+
+------------------------------+---------+---------+
247+
| F32MM | [55-52] | y |
248+
+------------------------------+---------+---------+
249+
| I8MM | [47-44] | y |
250+
+------------------------------+---------+---------+
237251
| SM4 | [43-40] | y |
238252
+------------------------------+---------+---------+
239253
| SHA3 | [35-32] | y |
240254
+------------------------------+---------+---------+
255+
| BF16 | [23-20] | y |
256+
+------------------------------+---------+---------+
241257
| BitPerm | [19-16] | y |
242258
+------------------------------+---------+---------+
243259
| AES | [7-4] | y |

‎Documentation/arm64/elf_hwcaps.rst

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,37 @@ HWCAP2_FRINT
204204

205205
Functionality implied by ID_AA64ISAR1_EL1.FRINTTS == 0b0001.
206206

207+
HWCAP2_SVEI8MM
208+
209+
Functionality implied by ID_AA64ZFR0_EL1.I8MM == 0b0001.
210+
211+
HWCAP2_SVEF32MM
212+
213+
Functionality implied by ID_AA64ZFR0_EL1.F32MM == 0b0001.
214+
215+
HWCAP2_SVEF64MM
216+
217+
Functionality implied by ID_AA64ZFR0_EL1.F64MM == 0b0001.
218+
219+
HWCAP2_SVEBF16
220+
221+
Functionality implied by ID_AA64ZFR0_EL1.BF16 == 0b0001.
222+
223+
HWCAP2_I8MM
224+
225+
Functionality implied by ID_AA64ISAR1_EL1.I8MM == 0b0001.
226+
227+
HWCAP2_BF16
228+
229+
Functionality implied by ID_AA64ISAR1_EL1.BF16 == 0b0001.
230+
231+
HWCAP2_DGH
232+
233+
Functionality implied by ID_AA64ISAR1_EL1.DGH == 0b0001.
234+
235+
HWCAP2_RNG
236+
237+
Functionality implied by ID_AA64ISAR0_EL1.RNDR == 0b0001.
207238

208239
4. Unused AT_HWCAP bits
209240
-----------------------

‎Documentation/arm64/silicon-errata.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,8 @@ stable kernels.
8888
+----------------+-----------------+-----------------+-----------------------------+
8989
| ARM | Cortex-A76 | #1463225 | ARM64_ERRATUM_1463225 |
9090
+----------------+-----------------+-----------------+-----------------------------+
91+
| ARM | Cortex-A55 | #1530923 | ARM64_ERRATUM_1530923 |
92+
+----------------+-----------------+-----------------+-----------------------------+
9193
| ARM | Neoverse-N1 | #1188873,1418040| ARM64_ERRATUM_1418040 |
9294
+----------------+-----------------+-----------------+-----------------------------+
9395
| ARM | Neoverse-N1 | #1349291 | N/A |

‎arch/arm64/Kconfig

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ config ARM64
162162
select HAVE_PERF_USER_STACK_DUMP
163163
select HAVE_REGS_AND_STACK_ACCESS_API
164164
select HAVE_FUNCTION_ARG_ACCESS_API
165+
select HAVE_FUTEX_CMPXCHG if FUTEX
165166
select HAVE_RCU_TABLE_FREE
166167
select HAVE_RSEQ
167168
select HAVE_STACKPROTECTOR
@@ -302,6 +303,9 @@ config ARCH_SUPPORTS_UPROBES
302303
config ARCH_PROC_KCORE_TEXT
303304
def_bool y
304305

306+
config BROKEN_GAS_INST
307+
def_bool !$(as-instr,1:\n.inst 0\n.rept . - 1b\n\nnop\n.endr\n)
308+
305309
config KASAN_SHADOW_OFFSET
306310
hex
307311
depends on KASAN
@@ -515,9 +519,13 @@ config ARM64_ERRATUM_1418040
515519

516520
If unsure, say Y.
517521

522+
config ARM64_WORKAROUND_SPECULATIVE_AT_VHE
523+
bool
524+
518525
config ARM64_ERRATUM_1165522
519526
bool "Cortex-A76: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
520527
default y
528+
select ARM64_WORKAROUND_SPECULATIVE_AT_VHE
521529
help
522530
This option adds a workaround for ARM Cortex-A76 erratum 1165522.
523531

@@ -527,6 +535,19 @@ config ARM64_ERRATUM_1165522
527535

528536
If unsure, say Y.
529537

538+
config ARM64_ERRATUM_1530923
539+
bool "Cortex-A55: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
540+
default y
541+
select ARM64_WORKAROUND_SPECULATIVE_AT_VHE
542+
help
543+
This option adds a workaround for ARM Cortex-A55 erratum 1530923.
544+
545+
Affected Cortex-A55 cores (r0p0, r0p1, r1p0, r2p0) could end-up with
546+
corrupted TLBs by speculating an AT instruction during a guest
547+
context switch.
548+
549+
If unsure, say Y.
550+
530551
config ARM64_ERRATUM_1286807
531552
bool "Cortex-A76: Modification of the translation table for a virtual address might lead to read-after-read ordering violation"
532553
default y
@@ -543,9 +564,13 @@ config ARM64_ERRATUM_1286807
543564
invalidated has been observed by other observers. The
544565
workaround repeats the TLBI+DSB operation.
545566

567+
config ARM64_WORKAROUND_SPECULATIVE_AT_NVHE
568+
bool
569+
546570
config ARM64_ERRATUM_1319367
547571
bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
548572
default y
573+
select ARM64_WORKAROUND_SPECULATIVE_AT_NVHE
549574
help
550575
This option adds work arounds for ARM Cortex-A57 erratum 1319537
551576
and A72 erratum 1319367
@@ -1364,6 +1389,11 @@ config ARM64_PAN
13641389
instruction if the cpu does not implement the feature.
13651390

13661391
config ARM64_LSE_ATOMICS
1392+
bool
1393+
default ARM64_USE_LSE_ATOMICS
1394+
depends on $(as-instr,.arch_extension lse)
1395+
1396+
config ARM64_USE_LSE_ATOMICS
13671397
bool "Atomic instructions"
13681398
depends on JUMP_LABEL
13691399
default y
@@ -1485,6 +1515,30 @@ config ARM64_PTR_AUTH
14851515

14861516
endmenu
14871517

1518+
menu "ARMv8.5 architectural features"
1519+
1520+
config ARM64_E0PD
1521+
bool "Enable support for E0PD"
1522+
default y
1523+
help
1524+
E0PD (part of the ARMv8.5 extensions) allows us to ensure
1525+
that EL0 accesses made via TTBR1 always fault in constant time,
1526+
providing similar benefits to KASLR as those provided by KPTI, but
1527+
with lower overhead and without disrupting legitimate access to
1528+
kernel memory such as SPE.
1529+
1530+
This option enables E0PD for TTBR1 where available.
1531+
1532+
config ARCH_RANDOM
1533+
bool "Enable support for random number generation"
1534+
default y
1535+
help
1536+
Random number generation (part of the ARMv8.5 Extensions)
1537+
provides a high bandwidth, cryptographically secure
1538+
hardware random number generator.
1539+
1540+
endmenu
1541+
14881542
config ARM64_SVE
14891543
bool "ARM Scalable Vector Extension support"
14901544
default y
@@ -1545,7 +1599,7 @@ config ARM64_MODULE_PLTS
15451599

15461600
config ARM64_PSEUDO_NMI
15471601
bool "Support for NMI-like interrupts"
1548-
select CONFIG_ARM_GIC_V3
1602+
select ARM_GIC_V3
15491603
help
15501604
Adds support for mimicking Non-Maskable Interrupts through the use of
15511605
GIC interrupt priority. This support requires version 3 or later of

‎arch/arm64/Makefile

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,8 @@ LDFLAGS_vmlinux += --fix-cortex-a53-843419
3030
endif
3131
endif
3232

33-
# Check for binutils support for specific extensions
34-
lseinstr := $(call as-instr,.arch_extension lse,-DCONFIG_AS_LSE=1)
35-
36-
ifeq ($(CONFIG_ARM64_LSE_ATOMICS), y)
37-
ifeq ($(lseinstr),)
33+
ifeq ($(CONFIG_ARM64_USE_LSE_ATOMICS), y)
34+
ifneq ($(CONFIG_ARM64_LSE_ATOMICS), y)
3835
$(warning LSE atomics not supported by binutils)
3936
endif
4037
endif
@@ -45,19 +42,15 @@ cc_has_k_constraint := $(call try-run,echo \
4542
return 0; \
4643
}' | $(CC) -S -x c -o "$$TMP" -,,-DCONFIG_CC_HAS_K_CONSTRAINT=1)
4744

48-
ifeq ($(CONFIG_ARM64), y)
49-
brokengasinst := $(call as-instr,1:\n.inst 0\n.rept . - 1b\n\nnop\n.endr\n,,-DCONFIG_BROKEN_GAS_INST=1)
50-
51-
ifneq ($(brokengasinst),)
45+
ifeq ($(CONFIG_BROKEN_GAS_INST),y)
5246
$(warning Detected assembler with broken .inst; disassembly will be unreliable)
53-
endif
5447
endif
5548

56-
KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) \
49+
KBUILD_CFLAGS += -mgeneral-regs-only \
5750
$(compat_vdso) $(cc_has_k_constraint)
5851
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
5952
KBUILD_CFLAGS += $(call cc-disable-warning, psabi)
60-
KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
53+
KBUILD_AFLAGS += $(compat_vdso)
6154

6255
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
6356
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)

‎arch/arm64/boot/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
1818

19-
targets := Image Image.gz
19+
targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo
2020

2121
$(obj)/Image: vmlinux FORCE
2222
$(call if_changed,objcopy)

‎arch/arm64/include/asm/alternative.h

Lines changed: 21 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,16 @@ void apply_alternatives_module(void *start, size_t length);
3535
static inline void apply_alternatives_module(void *start, size_t length) { }
3636
#endif
3737

38-
#define ALTINSTR_ENTRY(feature,cb) \
38+
#define ALTINSTR_ENTRY(feature) \
3939
" .word 661b - .\n" /* label */ \
40-
" .if " __stringify(cb) " == 0\n" \
4140
" .word 663f - .\n" /* new instruction */ \
42-
" .else\n" \
41+
" .hword " __stringify(feature) "\n" /* feature bit */ \
42+
" .byte 662b-661b\n" /* source len */ \
43+
" .byte 664f-663f\n" /* replacement len */
44+
45+
#define ALTINSTR_ENTRY_CB(feature, cb) \
46+
" .word 661b - .\n" /* label */ \
4347
" .word " __stringify(cb) "- .\n" /* callback */ \
44-
" .endif\n" \
4548
" .hword " __stringify(feature) "\n" /* feature bit */ \
4649
" .byte 662b-661b\n" /* source len */ \
4750
" .byte 664f-663f\n" /* replacement len */
@@ -62,33 +65,40 @@ static inline void apply_alternatives_module(void *start, size_t length) { }
6265
*
6366
* Alternatives with callbacks do not generate replacement instructions.
6467
*/
65-
#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled, cb) \
68+
#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled) \
6669
".if "__stringify(cfg_enabled)" == 1\n" \
6770
"661:\n\t" \
6871
oldinstr "\n" \
6972
"662:\n" \
7073
".pushsection .altinstructions,\"a\"\n" \
71-
ALTINSTR_ENTRY(feature,cb) \
74+
ALTINSTR_ENTRY(feature) \
7275
".popsection\n" \
73-
" .if " __stringify(cb) " == 0\n" \
7476
".pushsection .altinstr_replacement, \"a\"\n" \
7577
"663:\n\t" \
7678
newinstr "\n" \
7779
"664:\n\t" \
7880
".popsection\n\t" \
7981
".org . - (664b-663b) + (662b-661b)\n\t" \
8082
".org . - (662b-661b) + (664b-663b)\n" \
81-
".else\n\t" \
83+
".endif\n"
84+
85+
#define __ALTERNATIVE_CFG_CB(oldinstr, feature, cfg_enabled, cb) \
86+
".if "__stringify(cfg_enabled)" == 1\n" \
87+
"661:\n\t" \
88+
oldinstr "\n" \
89+
"662:\n" \
90+
".pushsection .altinstructions,\"a\"\n" \
91+
ALTINSTR_ENTRY_CB(feature, cb) \
92+
".popsection\n" \
8293
"663:\n\t" \
8394
"664:\n\t" \
84-
".endif\n" \
8595
".endif\n"
8696

8797
#define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \
88-
__ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg), 0)
98+
__ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg))
8999

90100
#define ALTERNATIVE_CB(oldinstr, cb) \
91-
__ALTERNATIVE_CFG(oldinstr, "NOT_AN_INSTRUCTION", ARM64_CB_PATCH, 1, cb)
101+
__ALTERNATIVE_CFG_CB(oldinstr, ARM64_CB_PATCH, 1, cb)
92102
#else
93103

94104
#include <asm/assembler.h>

‎arch/arm64/include/asm/archrandom.h

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
/* SPDX-License-Identifier: GPL-2.0 */
2+
#ifndef _ASM_ARCHRANDOM_H
3+
#define _ASM_ARCHRANDOM_H
4+
5+
#ifdef CONFIG_ARCH_RANDOM
6+
7+
#include <linux/random.h>
8+
#include <asm/cpufeature.h>
9+
10+
static inline bool __arm64_rndr(unsigned long *v)
11+
{
12+
bool ok;
13+
14+
/*
15+
* Reads of RNDR set PSTATE.NZCV to 0b0000 on success,
16+
* and set PSTATE.NZCV to 0b0100 otherwise.
17+
*/
18+
asm volatile(
19+
__mrs_s("%0", SYS_RNDR_EL0) "\n"
20+
" cset %w1, ne\n"
21+
: "=r" (*v), "=r" (ok)
22+
:
23+
: "cc");
24+
25+
return ok;
26+
}
27+
28+
static inline bool __must_check arch_get_random_long(unsigned long *v)
29+
{
30+
return false;
31+
}
32+
33+
static inline bool __must_check arch_get_random_int(unsigned int *v)
34+
{
35+
return false;
36+
}
37+
38+
static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
39+
{
40+
/*
41+
* Only support the generic interface after we have detected
42+
* the system wide capability, avoiding complexity with the
43+
* cpufeature code and with potential scheduling between CPUs
44+
* with and without the feature.
45+
*/
46+
if (!cpus_have_const_cap(ARM64_HAS_RNG))
47+
return false;
48+
49+
return __arm64_rndr(v);
50+
}
51+
52+
53+
static inline bool __must_check arch_get_random_seed_int(unsigned int *v)
54+
{
55+
unsigned long val;
56+
bool ok = arch_get_random_seed_long(&val);
57+
58+
*v = val;
59+
return ok;
60+
}
61+
62+
static inline bool __init __early_cpu_has_rndr(void)
63+
{
64+
/* Open code as we run prior to the first call to cpufeature. */
65+
unsigned long ftr = read_sysreg_s(SYS_ID_AA64ISAR0_EL1);
66+
return (ftr >> ID_AA64ISAR0_RNDR_SHIFT) & 0xf;
67+
}
68+
69+
#else
70+
71+
static inline bool __arm64_rndr(unsigned long *v) { return false; }
72+
static inline bool __init __early_cpu_has_rndr(void) { return false; }
73+
74+
#endif /* CONFIG_ARCH_RANDOM */
75+
#endif /* _ASM_ARCHRANDOM_H */

0 commit comments

Comments
 (0)