Showing posts with label compression. Show all posts
Showing posts with label compression. Show all posts

23 May 2022

Solaris 11.4 SRU45 (May 2022) - zstd compression utility

Solaris 11.4.45 includes now the zstd compression utility.

-bash-5.1$ zstd -V
*** zstd command line interface 64-bits v1.5.0, by Yann Collet ***

To compare I have a Oracle DB19c home as a tar file.
I executed this tests on a SPARC S7 LDom with 1 core

# ls -lh 19c.tar
-rw-r--r--   1 root     root       6.66G May 23 17:36 19c.tar

zstd does a higher compression compared to lz4, but takes
more time.

# time zstd 19c.tar -o 19c.tar.zstd
19c.tar              : 39.24%   (7153274368 => 2806612298 bytes, 19c.tar.zstd)

real    3m26.408s
user    3m29.804s
sys     0m6.698s


# time lz4 19c.tar
Compressed filename will be : 19c.tar.lz4
Compressed 7153274368 bytes into 3629706666 bytes ==> 50.74%

real    2m35.092s
user    2m25.227s
sys     0m6.852s

# ls -lh 19c.tar.lz4 19c.tar.zstd
-rw-r--r--   1 root     root       3.38G May 23 17:36 19c.tar.lz4
-rw-r--r--   1 root     root       2.61G May 23 18:14 19c.tar.zstd

With decompression zstd is a bit faster than lz4.

# time unlz4 19c.tar.lz4
Decoding file 19c.tar
19c.tar.lz4          : decoded 7153274368 bytes

real    2m9.913s
user    1m23.008s
sys     0m8.339s


# time unzstd 19c.tar.zstd
19c.tar.zstd        : 7153274368 bytes

real    1m52.263s
user    1m1.836s
sys     0m9.236s


07 April 2017

Is there a performance impact when using Solaris ZFS lz4 compression?

Starting with Solaris 11.3 ZFS supports lz4 compression. Lets verify the impact to performance if we enable lz4 compression with 2 concrete sample files.
First a zip file containing Solaris 11 SRU Updates and second a simple text logfile.

We disable the ZFS Cache to see the impact of I/O and compression
# zfs set primarycache=metadata v0123_db/source
# zfs set primarycache=metadata compressed/fs
# zfs set primarycache=metadata uncompressed/fs


Test 1 - zipped file

# time cp p25604852_1100_Solaris86-64_1of4.zip /uncompressed

real    1m27.571s
user    0m0.002s
sys     0m4.361s

-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME             PROPERTY       VALUE  SOURCE
uncompressed/fs  compression    off    inherited from uncompressed
uncompressed/fs  compressratio  1.00x  -
uncompressed/fs  used           1.35G  -


# time cp p25604852_1100_Solaris86-64_1of4.zip /compressed

real    1m27.427s
user    0m0.002s
sys     0m4.408s

-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME           PROPERTY       VALUE  SOURCE
compressed/fs  compression    lz4    inherited from compressed
compressed/fs  compressratio  1.00x  -
compressed/fs  used           1.34G  -

We see the same duration, no performance loss and because the file is zipped
nearly no space savings.



Test 2 - Log file with Text

# time cp framework.log /uncompressed/

real    0m24.608s
user    0m0.001s
sys     0m1.241s

-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME             PROPERTY       VALUE  SOURCE
uncompressed/fs  compression    off    inherited from uncompressed
uncompressed/fs  compressratio  1.00x  -
uncompressed/fs  used           390M   -


# time cp framework.log /compressed/

real    0m24.495s
user    0m0.001s
sys     0m1.260s

-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME           PROPERTY       VALUE  SOURCE
compressed/fs  compression    lz4    inherited from compressed
compressed/fs  compressratio  6.37x  -
compressed/fs  used           61.4M  -

Good compression (6x). We save 330MB of disk space here.
No impact to duration. The SPARC S7 core is fast enough.


And now Read Performance

# time cp /compressed/framework.log /tmp; time cp /uncompressed/framework.log /tmp

real    0m17.415s
user    0m0.001s
sys     0m1.354s

real    0m24.479s
user    0m0.001s
sys     0m1.389s

Better results from compressed filesystem. CPU decompression is faster than doing I/O. Need to read 6x the data from uncompressed zfs filesystem.


Summary
With above samples we don't see negative impact when enabling lz4 compression. If you use compressable text files you save lots of disk space while gaining read performance. We start using lz4 on our ZPOOLs by default now.