RISCV-V RVV Extension Support in Zephyr

[akifejaz]

Describe the bug

I'm trying to test vector instructions, but it seems to give me error saying zephyr is unable to recognize the vector instructions.
west build -p always -b qemu_riscv64 samples/rvv_demo

To Reproduce

Steps to reproduce the behavior:

1. git checkout afce532
2. cp -r samples/hello_world/ samples/rvv_demo
3. add below code in samples/rvv_demo/src/main.c

#include <stdio.h>
int main(void)
{
    // vector instructions 
    uint64_t addr = 256;
     __asm__ volatile (
    "li    x1, 0x200 \n\t"
    "csrs  mstatus, x1\n\t"
    "li    x1, 0x800005000 \n\t"
    "vsetvli x0,%0,e8,m1,ta,mu \n\t"
    "vle32.v v0, (x1)\n\t"
    :: "r"(addr)
    );

	return 0;
}

4. proj.config as below

CONFIG_BOOT_BANNER=n
CONFIG_GNU_C_EXTENSIONS=y
CONFIG_GEN_IRQ_VECTOR_TABLE=y
CONFIG_GEN_SW_ISR_TABLE=y
CONFIG_RISCV_VECTORED_MODE=y

5. west build -p always -b qemu_riscv64 samples/rvv_demo

Is there something I'm missing in configuration? I'm not sure but what I could find I tried to enable like mainly CONFIG_RISCV_VECTORED_MODE.
Is there no RISCV-V (vector instructions) support in Zephyr or in Zephyr-SDK ? Since I tried the same for scalar instructions that works fine.

Please help me in this regard, like if its enabled is there anything else I've to configured.
Best,
Akif

[kartben]

I'm guessing it's just an issue with qemu specifically, and the way we bootstrap it.
https://lists.nongnu.org/archive/html/qemu-riscv/2024-06/msg00270.html

I would try adding the v=true flag here, maybe?

zephyr/boards/qemu/riscv64/board.cmake

Line 6 in 184c0f9

set(QEMU_CPU_TYPE_${ARCH} riscv64)

[kartben]

Oh I guess I should have actually read the logs first 😅 didn't realize it was a build issue, sorry

[fkokosinski]

Hey @akifejaz, thanks for reporting this.

CONFIG_RISCV_VECTORED_MODE

Note that this does not refer to the "V" vector extension, but to the interrupt vector (mtvec). You can see the supported ISA extensions in arch/riscv/Kconfig.isa (RISCV_ISA_EXT_* options).

Right now the RISC-V port in Zephyr does not support the "V" extension, but the binutils version that Zephyr SDK uses seems to include the support for them :https://github.com/zephyrproject-rtos/binutils-gdb/blob/zephyr-binutils-2_38/opcodes/riscv-opc.c#L966.

This means that you could get the "V" extension working by adding an appropriate ISA extension Kconfig option, selecting it in your SoC, and changing this file a bit: https://github.com/zephyrproject-rtos/zephyr/blob/main/cmake/compiler/gcc/target_riscv.cmake.

[akifejaz]

This means that you could get the "V" extension working by adding an appropriate ISA extension Kconfig option, selecting it in your SoC, and changing this file a bit: https://github.com/zephyrproject-rtos/zephyr/blob/main/cmake/compiler/gcc/target_riscv.cmake.

@fkokosinski Thanks for these details. I've tried this and it seems to compile the test file with vector instructions. But I'm stuck at something related to picolibc.

ninja: Entering directory `./build'
[1/94] Preparing syscall dependency handling

[9/94] Generating include/generated/zephyr/version.h
-- Zephyr version: 4.0.0-rc1 (/projects/hydra/t_aejaz/zephyr), build: a97c5e271054
[89/94] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /projects/hydra/t_aejaz/zephyr/samples/xcelerium/basic_demo/build/zephyr/zephyr_pre0.map 
: && /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/riscv64-zephyr-elf-gcc  -gdwarf-4 -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/riscv/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/projects/hydra/t_aejaz/zephyr/samples/xcelerium/basic_demo/build/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/riscv/core/libarch__riscv__core.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/drivers/timer/libdrivers__timer.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L/projects/hydra/t_aejaz/zephyr/samples/xcelerium/basic_demo/build/zephyr  zephyr/arch/common/libisr_tables.a  -mcmodel=medany  -mabi=lp64  -march=rv64imacv_zicsr_zifencei  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -specs=picolibc.specs  -DPICOLIBC_LONG_LONG_PRINTF_SCANF -L"/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0" -lc -lgcc && cd /projects/hydra/t_aejaz/zephyr/samples/xcelerium/basic_demo/build/zephyr && /usr/local/lib64/python3.7/site-packages/cmake/data/bin/cmake -E true
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_errno_errno.c.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: failed to merge target specific data of file /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_errno_errno.c.o)
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_string_strnlen.c.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: failed to merge target specific data of file /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_string_strnlen.c.o)
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_tinystdio_vflprintf.c.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: failed to merge target specific data of file /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_tinystdio_vflprintf.c.o)
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_tinystdio_vflscanf.c.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: failed to merge target specific data of file /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(libc_tinystdio_vflscanf.c.o)
/projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/riscv64-zephyr-elf/12.2.0/../../../../riscv64-zephyr-elf/bin/ld.bfd: /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a(memcpy-asm.S.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
  

I understand the issue somewhat like when I checked /projects/hydra/t_aejaz/zephyr-sdk-0.16.8/riscv64-zephyr-elf/bin/../lib/gcc/../../picolibc/riscv64-zephyr-elf/lib/./libc.a file it seems to be ELF 32 b file. and thats why its unable to link it with ELF 64b.

But I'm confused, why this can happen by only adding the V extension?
I only changed the ISA from -march=rv64imac_zicsr_zifencei to -march=rv64imacv_zicsr_zifencei if I remove the v it links fine this picolibc.

I've also tried to disable this by CONFIG_PICOLIBC_SOURCE but it didnt help. Can you please help what I might be doing wrong here?

Best,

[fkokosinski]

@akifejaz looks like a toolchain issue. I applied the following patch:

diff --git a/cmake/compiler/gcc/target_riscv.cmake b/cmake/compiler/gcc/target_riscv.cmake
index a297a772886..6dac4797a7c 100644
--- a/cmake/compiler/gcc/target_riscv.cmake
+++ b/cmake/compiler/gcc/target_riscv.cmake
@@ -45,6 +45,8 @@ if(CONFIG_RISCV_ISA_EXT_C)
     string(CONCAT riscv_march ${riscv_march} "c")
 endif()
 
+string(CONCAT riscv_march ${riscv_march} "v")
+
 if(CONFIG_RISCV_ISA_EXT_ZICSR)
     string(CONCAT riscv_march ${riscv_march} "_zicsr")
 endif()
@@ -90,4 +92,4 @@ set(LLEXT_APPEND_FLAGS
   -mabi=${riscv_mabi}
   -march=${riscv_march}
   -mno-relax
-)
\ No newline at end of file
+)
diff --git a/samples/hello_world/src/main.c b/samples/hello_world/src/main.c
index c550ab461cb..0ce8c1e5e82 100644
--- a/samples/hello_world/src/main.c
+++ b/samples/hello_world/src/main.c
@@ -9,6 +9,15 @@
 int main(void)
 {
 	printf("Hello World! %s\n", CONFIG_BOARD_TARGET);
-
+	// vector instructions
+	uint64_t addr = 256;
+	__asm__ volatile (
+			"li    x1, 0x200 \n\t"
+			"csrs  mstatus, x1\n\t"
+			"li    x1, 0x800005000 \n\t"
+			"vsetvli x0,%0,e8,m1,ta,mu \n\t"
+			"vle32.v v0, (x1)\n\t"
+			:: "r"(addr)
+			);
 	return 0;
 }

and builidng for qemu_riscv32 works just fine:

west build -p -b qemu_riscv32 samples/hello_world

Make sure that the march and the mabi are set to correct values when you're invoking GCC.

[akifejaz]

and builidng for qemu_riscv32 works just fine:

Hi, I'm working on RV64, can you try qemu_riscv64 once please?

west build -p -b qemu_riscv64 samples/hello_world

[fkokosinski]

and builidng for qemu_riscv32 works just fine:

Hi, I'm working on RV64, can you try qemu_riscv64 once please?

west build -p -b qemu_riscv64 samples/hello_world

Yeah, I get the same error. It looks like providing gcc with the v extension in the ISA string causes it to switch to ilp32 ABI, which then (understandably) causes link errors down the line, because it the one GCC uses to default to.

If you y-select CONFIG_MINIMAL_LIBC then you still get libgcc-related errors, because again—it defaults to mabi=ilp32.

You can get around that by changing the default libgcc.a to the one you want to use, i.e.:

cd zephyr-sdk-0.16.8/riscv64-zephyr-elf/lib/gcc/riscv64-zephyr-elf/12.2.0 # on any other path where you have zephyr sdk
cp libgcc.a libgcc_bak.a
cp ./rv64imac_zicsr_zifencei_zba_zbb_zbc_zbs/lp64/medany/libgcc.a libgcc.a

And now you can do:

west build -p -b qemu_riscv64 samples/hello_world -- -DCONFIG_MINIMAL_LIBC=y

This is obviously a hack, and you should probably implement proper support for the V extension in Zephyr instead, and then contribute it in a PR :)

[haijun20222]

Hello, @fkokosinski as you described above, I am using the latest official riscv64-zephyr-elf toolchain to compile riscv64 CPUs with vector support and encountered compilation issues. I need to contribute new code to Zephyr, but there are problems compiling riscv64 with riscv vector support. Currently, I have the following questions:
1，Which RISC-V CPU types is the riscv64-zephyr-elf toolchain applicable to, and what is its application scope? In other words, what is the purpose of this toolchain's design?
2，Will the v extension be continued for riscv64 in the future? If not, do I need to provide my own toolchain support when contributing code?
thanks very much.