Does the special vdev improve Read/Write performance ?

[yuanfangcc]

1. I uesd eight HDD(sdb - sdi) create a pool.(raid0)
   zpool create mypool sdb sdc sdd sde sdf sdg sdh sdi
2. I uesd eight HDD(sdb - sdi) create a pool and one SSD(sdj) be a special vdev add to the pool.
   zpool create mypool sdb sdc sdd sde sdf sdg sdh sdi special sdj

I think the pool read and write performance is higher in the second case than in the first.
But performance doesn't actually improve that much.

I don't know why that is.Thank you in advance for your answer.

The tool I use to measure performance is Vdbench.The Vdbench configuration file is as follows:
fsd=default,width=8,depth=3,files=20,distribution=all,size=8KB,openflags=(o_dsync,o_rsync) fsd=fsd1,anchor=/mypool/test1 fwd=fwd1,fsd=fsd*,rdpct=0,xfersize=8KB,fileio=random,fileselect=random,threads=50 Rd=run1,fwd=fwd*,fwdrate=max,format=restart,elapsed=60,interval=1,warmup=10

The file system creation command is as follows:
zfs create -o recordsize=8KB -o atime=off -o logbias=throughput -o sync=always -o acltype=posixacl -o xattr=sa -o mountpoint=/mnt/mnt1 mypool/test1

Different block sizes and file numbers are tested. Why does special vdev not improve read/write performance?

[jxdking]

Arc may cache metadata anyway. Try to set primarycache=none, and test again.

[yuanfangcc]

Thank you for your answer.

[amotin]

I am not sure primarycache=none is a very good idea, it is too synthetic. But flushing the caches by exporting/importing the pool before the test would make sense. And the test should be short enough to complete before all metadata will be cached. With default indirect block size of 128KB with ~1024 pointers per indirect block you need to read only about 0.1% of data randomly to already cache all the metadata in ARC.

You'd probably need more of different metadata types, such as tons of small files, directories, xattrs, etc to make a bigger benefits from special vdev. Also special vdev is almost mandatory in case regular vdevs unable to store small blocks, such as DRAID, or may be some RAIDZ2/3 with large ashift, etc.

[yuanfangcc]

Thank you for your answer.