Don't remove Data Disks from Solaris Zpools if performance is important

Solaris 11.4 delivers a new feature to remove data disks from existing zpools.
zpool remove myzpool <disk>

We used this feature a few times without problems on test environments.

But it has performance impact if the removed disk had data on it.
Especially if read performance is important. We know customers with oracle databases
where latency around 1ms is expected. After removing a disk from a
large zpool the performance was terrible and the only solution was to
re-create the zpool.

Important to understand there is an expected performance impact while the disk is removing.
Sure. The data needs to be copied to the remaining disks.
But even after the remove there can be a major performance impact when the data (from the removed disk) must be read, because the data copy added additional internal redirections.

The recommendation is to use this feature only after an accidentally add
of a disk to a wrong zpool. there is no performance impact if the
removed disk has no data.

Find the details of this recommendation in the Solaris 11.4 ZFS Manual:
https://docs.oracle.com/cd/E37838_01/html/E61017/remove-devices.html

 

To avoid such troubles we disabled the dataset -c remdisk feature for data disks
by default in VDCF Version 8.1.8

More about our VDCF Solaris Management product can be found on
https://www.jomasoft.ch/vdcf/