在 Arm 平台上启动 Virtio

[luodeb]

Clone代码

新建一个 clone.sh 文件

#!/bin/bash

BRANCH="-b debin/virtio_dev"
VMM_BRANCH_0="-b debin/shell_0x400"
VMM_BRANCH_1="-b debin/shell_0x800"

mkdir -p crates

git clone $BRANCH https://github.com/arceos-hypervisor/axvisor.git 

cd axvisor

# 克隆arceos主仓库
git clone $VMM_BRANCH_0 https://github.com/arceos-hypervisor/arceos.git ../arceos_0
git clone $VMM_BRANCH_1 https://github.com/arceos-hypervisor/arceos.git ../arceos_1

# 克隆其他仓库到crates目录
REPOS=(
    "arceos"
    "arm_vcpu"
    "axaddrspace"
    "axdevice"
    "axdevice_crates"
    "axvcpu"
    "axvirtio"
    "axvisor_api"
    "axvm"
    "x86_vcpu"
)

for repo in "${REPOS[@]}"; do
    git clone $BRANCH "https://github.com/arceos-hypervisor/${repo}.git" "../crates/${repo}"
done

echo "所有仓库克隆完成！"

cd .. && mkdir .vscode

cat > .vscode/settings.json << 'EOF'
{
    "rust-analyzer.cargo.target": "aarch64-unknown-none-softfloat",
    "rust-analyzer.check.allTargets": false,
    "rust-analyzer.cargo.features": ["irq", "hv"],
    "rust-analyzer.cargo.extraEnv": {
        "RUSTFLAGS": "--cfg platform_family=\"aarch64-qemu-virt\"",
        "AX_CONFIG_PATH": "${workspaceFolder}/axvisor/.axconfig.toml"
    }
}
EOF

编译Arceos

编译VM0

首先需要编译第一个 vm，启动地址是在 0x4020_0000

cd arceos_0/
make ARCH=aarch64 A=examples/shell BUS=mmio

编译 VM1，启动地址是在 0x8020_0000，这两个 Arcos 只有启动地址不一致，其他的都是一致的

cd arceos_1/
make ARCH=aarch64 A=examples/shell BUS=mmio

启动AxVisor

# 创建磁盘文件
make disk_img
# 启动AxVisor
make ACCEL=n ARCH=aarch64 LOG=debug VM_CONFIGS=configs/vms/arceos-aarch64.toml:configs/vms/arceos-aarch64_2.toml FEATURES=hv APP_FEATURES=fs SMP=2 MEM=4G SECOND_SERIAL=y run

# 打开第二个终端
telnet localhost 4321

运行结果

VM0

       d8888                            .d88888b.   .d8888b.
      d88888                           d88P" "Y88b d88P  Y88b
     d88P888                           888     888 Y88b.
    d88P 888 888d888  .d8888b  .d88b.  888     888  "Y888b.
   d88P  888 888P"   d88P"    d8P  Y8b 888     888     "Y88b.
  d88P   888 88[8       0.365320 1:4888       8888axv8888 888     888is       "8or::hal:91] 88
 d8888888888 888IRQ handler 26
     Y88b.    Y8b.     Y88[  0.366033b. .d8 18P Y8:9 axv8b  disor::vm88m::vcP
d8pus:8P  28  3]  88888 VM[2] VC  pu   [0] waiting fo"Yr 888runn8Ping  
"Y8888   "Y888"   m[  0."366433 Y8881:8P"
9 axvisor::vmm::vcpus:286] 
VM[2] VCpu[0] running...
rch [m= aarch64
platform = aarch64-qemu-virt
target = aarch64-unknown-none-softfloat
build_mode = release
log_level = warn
smp = 1

[  0.386910 1:9 arm_vcpu::context_frame:106] Try to set zero register at index [31] as 1
[  0.387665 0:8 arm_vcpu::context_frame:106] Try to set zero register at index [31] as 1
[  0.388835 0:8 axvirtio_blk::mmio::device:623] VirtIO device driver OK
[  0.388870 1:9 axvirtio_blk::mmio::device:623] VirtIO device driver OK
[  0.458149 0 fatfs::dir:139] Is a directory
[  0.627210 0 fatfs::dir:139] Is a directory
[  0.905075 0 fatfs::dir:139] Is a directory
[  1.290565 0 fatfs::dir:139] Is a directory
Available commands:
  cat
  cd
  echo
  exit
  help
  ls
  mkdir
  pwd
  rm
  uname
arceos:/$

VM1

telnet localhost 4321
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

       d8888                            .d88888b.   .d8888b.
      d88888                           d88P" "Y88b d88P  Y88b
     d88P888                           888     888 Y88b.
    d88P 888 888d888  .d8888b  .d88b.  888     888  "Y888b.
   d88P  888 888P"   d88P"    d8P  Y8b 888     888     "Y88b.
  d88P   888 888     888      88888888 888     888       "888
 d8888888888 888     Y88b.    Y8b.     Y88b. .d88P Y88b  d88P
d88P     888 888      "Y8888P  "Y8888   "Y88888P"   "Y8888P"

arch = aarch64
platform = aarch64-qemu-virt
target = aarch64-unknown-none-softfloat
build_mode = release
log_level = warn
smp = 1

[  0.461203 0 fatfs::dir:139] Is a directory
[  0.635225 0 fatfs::dir:139] Is a directory
[  0.920567 0 fatfs::dir:139] Is a directory
[  1.314218 0 fatfs::dir:139] Is a directory
Available commands:
  cat
  cd
  echo
  exit
  help
  ls
  mkdir
  pwd
  rm
  uname
arceos:/$

[luodeb]

性能测试

测试环境

• Guest：https://github.com/rslabbox/arm_tiny/tree/sample_virtio
• 平台：Qemu nvme/fat32
• Virtio-block

测试结果

单Guest + Qemu

 Timer frequency: 62500000 Hz
 Testing 10000 sectors starting from sector 1000
 === Write Performance Test ===
 Write Results:
 - Sectors written: 10000/10000
 - Total time: 428 ms
 - Data written: 5120000 bytes
 - Write speed: 11 MB/s
 - Per-operation timing:
   * Min: 0 ms
   * Max: 0 ms
   * Avg: 0 ms
 === Read Performance Test ===
 Read Results:
 - Sectors read: 10000/10000
 - Data verified: 10000/10000
 - Total time: 537 ms (including verification)
 - Data read: 5120000 bytes
 - Read speed: 9 MB/s
 - Per-operation timing (read only):
   * Min: 0 ms
   * Max: 0 ms
   * Avg: 0 ms

Qemu + AxVisor + 单Guest + nvme-backend

Timer frequency: 62500000 Hz
Testing 10000 sectors starting from sector 1000
=== Write Performance Test ===
Write Results:
- Sectors written: 10000/10000
- Total time: 746 ms
- Data written: 5120000 bytes
- Write speed: 6 MB/s
- Per-operation timing:
  * Min: 0 ms
  * Max: 0 ms
  * Avg: 0 ms
=== Read Performance Test ===
Read Results:
- Sectors read: 10000/10000
- Data verified: 10000/10000
- Total time: 798 ms (including verification)
- Data read: 5120000 bytes
- Read speed: 6 MB/s
- Per-operation timing (read only):
  * Min: 0 ms
  * Max: 0 ms
  * Avg: 0 ms

Qemu + AxVisor + 单Guest + file-backend

 Timer frequency: 62500000 Hz
 Testing 100 sectors starting from sector 1000
 === Write Performance Test ===
 Write Results:
 - Sectors written: 100/100
 - Total time: 4255 ms
 - Data written: 51200 bytes
 - Write speed: 0 MB/s
 - Per-operation timing:
   * Min: 0 ms
   * Max: 45 ms
   * Avg: 42 ms
 === Read Performance Test ===
 Read Results:
 - Sectors read: 100/100
 - Data verified: 100/100
 - Total time: 4401 ms (including verification)
 - Data read: 51200 bytes
 - Read speed: 0 MB/s
 - Per-operation timing (read only):
   * Min: 41 ms
   * Max: 85 ms
   * Avg: 44 ms

结论

要点

• AxVisor + nvme-backend 相比 Qemu 直挂，吞吐下降：写约 -45%，读约 -33%；平均每扇区时延增加至约 1.7–1.9 倍。
• AxVisor + file-backend 性能严重退化：吞吐仅约 0.011 MiB/s，较基线慢 ~800–950 倍；单次操作时延达 41–85 ms，明显呈现同步/小颗粒 I/O 的高开销特征。

吞吐汇总

场景	后端	扇区数	数据量	写用时(ms)	写速(报告MB/s / 计算MiB/s)	读用时(ms)	读速(报告MB/s / 计算MiB/s)
单Guest + Qemu	virtio-block	10000	5,120,000 B (4.88 MiB)	428	11 / 11.41	537	9 / 9.09
Qemu + AxVisor + 单Guest	nvme-backend	10000	5,120,000 B (4.88 MiB)	746	6 / 6.55	798	6 / 6.12
Qemu + AxVisor + 单Guest	file-backend	100	51,200 B (0.0488 MiB)	4255	~0 / 0.0115	4401	~0 / 0.0111

时延与 per-op 统计

场景	平均每扇区写时延(ms)	平均每扇区读时延(ms)	单次操作计时-写(min/avg/max, ms)	单次操作计时-读(min/avg/max, ms)
单Guest + Qemu	0.0428	0.0537	0 / 0 / 0	0 / 0 / 0
Qemu + AxVisor + 单Guest（nvme）	0.0746	0.0798	0 / 0 / 0	0 / 0 / 0
Qemu + AxVisor + 单Guest（file）	42.55	44.01	0 / 42 / 45	41 / 44 / 85

相对基线（单Guest + Qemu）的变化

• AxVisor + nvme-backend
  • 吞吐：写 -45%（11 → 6 MB/s），读 -33%（9 → 6 MB/s）
  • 平均时延：写 +74%（0.043 → 0.075 ms），读 +49%（0.054 → 0.080 ms）
• AxVisor + file-backend（仅 100 扇区）
  • 吞吐：写约 -99.9%（11 → 0.0115 MiB/s ≈ 959× 变慢），读约 -99.9%（9 → 0.0111 MiB/s ≈ 818× 变慢）
  • 平均时延：写/读约 +1000×（~0.043/0.054 ms → ~42/44 ms）

原因分析

• 单次操作路径极大的缩短
• 在 file-backend 上：Guest virtio 请求 → Hypervisor → fs操作 → virtio请求 → QEMU → 虚拟Nvme磁盘
• 在 nvme-backend 上：Guest virtio 请求 → Hypervisor → 虚拟Nvme磁盘

还存在的问题

• Nvme 驱动只能在 alloc-slab 上启动，具体原因不明
• Arceos 在 nvme-backend 上的 Virtio 驱动无法工作，可能还是内存分配器的原因