Solaris 11.4.45 includes now the zstd compression utility.
-bash-5.1$ zstd -V
*** zstd command line interface 64-bits v1.5.0, by Yann Collet ***
To compare I have a Oracle DB19c home as a tar file.
I executed this tests on a SPARC S7 LDom with 1 core
# ls -lh 19c.tar
-rw-r--r-- 1 root root 6.66G May 23 17:36 19c.tar
zstd does a higher compression compared to lz4, but takes
more time.
# time zstd 19c.tar -o 19c.tar.zstd
19c.tar : 39.24% (7153274368 => 2806612298 bytes, 19c.tar.zstd)
real 3m26.408s
user 3m29.804s
sys 0m6.698s
# time lz4 19c.tar
Compressed filename will be : 19c.tar.lz4
Compressed 7153274368 bytes into 3629706666 bytes ==> 50.74%
real 2m35.092s
user 2m25.227s
sys 0m6.852s
# ls -lh 19c.tar.lz4 19c.tar.zstd
-rw-r--r-- 1 root root 3.38G May 23 17:36 19c.tar.lz4
-rw-r--r-- 1 root root 2.61G May 23 18:14 19c.tar.zstd
With decompression zstd is a bit faster than lz4.
# time unlz4 19c.tar.lz4
Decoding file 19c.tar
19c.tar.lz4 : decoded 7153274368 bytes
real 2m9.913s
user 1m23.008s
sys 0m8.339s
# time unzstd 19c.tar.zstd
19c.tar.zstd : 7153274368 bytes
real 1m52.263s
user 1m1.836s
sys 0m9.236s
Showing posts with label lz4. Show all posts
Showing posts with label lz4. Show all posts
23 May 2022
Solaris 11.4 SRU45 (May 2022) - zstd compression utility
07 April 2017
Is there a performance impact when using Solaris ZFS lz4 compression?
Starting with Solaris 11.3 ZFS supports lz4 compression. Lets verify the impact to performance if we enable lz4 compression with 2 concrete sample files.
First a zip file containing Solaris 11 SRU Updates and second a simple text logfile.
We disable the ZFS Cache to see the impact of I/O and compression
# zfs set primarycache=metadata v0123_db/source
# zfs set primarycache=metadata compressed/fs
# zfs set primarycache=metadata uncompressed/fs
Test 1 - zipped file
# time cp p25604852_1100_Solaris86-64_1of4.zip /uncompressed
real 1m27.571s
user 0m0.002s
sys 0m4.361s
-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME PROPERTY VALUE SOURCE
uncompressed/fs compression off inherited from uncompressed
uncompressed/fs compressratio 1.00x -
uncompressed/fs used 1.35G -
# time cp p25604852_1100_Solaris86-64_1of4.zip /compressed
real 1m27.427s
user 0m0.002s
sys 0m4.408s
-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME PROPERTY VALUE SOURCE
compressed/fs compression lz4 inherited from compressed
compressed/fs compressratio 1.00x -
compressed/fs used 1.34G -
We see the same duration, no performance loss and because the file is zipped
nearly no space savings.
Test 2 - Log file with Text
# time cp framework.log /uncompressed/
real 0m24.608s
user 0m0.001s
sys 0m1.241s
-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME PROPERTY VALUE SOURCE
uncompressed/fs compression off inherited from uncompressed
uncompressed/fs compressratio 1.00x -
uncompressed/fs used 390M -
# time cp framework.log /compressed/
real 0m24.495s
user 0m0.001s
sys 0m1.260s
-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME PROPERTY VALUE SOURCE
compressed/fs compression lz4 inherited from compressed
compressed/fs compressratio 6.37x -
compressed/fs used 61.4M -
Good compression (6x). We save 330MB of disk space here.
No impact to duration. The SPARC S7 core is fast enough.
And now Read Performance
# time cp /compressed/framework.log /tmp; time cp /uncompressed/framework.log /tmp
real 0m17.415s
user 0m0.001s
sys 0m1.354s
real 0m24.479s
user 0m0.001s
sys 0m1.389s
Better results from compressed filesystem. CPU decompression is faster than doing I/O. Need to read 6x the data from uncompressed zfs filesystem.
Summary
With above samples we don't see negative impact when enabling lz4 compression. If you use compressable text files you save lots of disk space while gaining read performance. We start using lz4 on our ZPOOLs by default now.
First a zip file containing Solaris 11 SRU Updates and second a simple text logfile.
We disable the ZFS Cache to see the impact of I/O and compression
# zfs set primarycache=metadata v0123_db/source
# zfs set primarycache=metadata compressed/fs
# zfs set primarycache=metadata uncompressed/fs
Test 1 - zipped file
# time cp p25604852_1100_Solaris86-64_1of4.zip /uncompressed
real 1m27.571s
user 0m0.002s
sys 0m4.361s
-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME PROPERTY VALUE SOURCE
uncompressed/fs compression off inherited from uncompressed
uncompressed/fs compressratio 1.00x -
uncompressed/fs used 1.35G -
# time cp p25604852_1100_Solaris86-64_1of4.zip /compressed
real 1m27.427s
user 0m0.002s
sys 0m4.408s
-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME PROPERTY VALUE SOURCE
compressed/fs compression lz4 inherited from compressed
compressed/fs compressratio 1.00x -
compressed/fs used 1.34G -
We see the same duration, no performance loss and because the file is zipped
nearly no space savings.
Test 2 - Log file with Text
# time cp framework.log /uncompressed/
real 0m24.608s
user 0m0.001s
sys 0m1.241s
-bash-4.4$ zfs get compression,compressratio,used uncompressed/fs
NAME PROPERTY VALUE SOURCE
uncompressed/fs compression off inherited from uncompressed
uncompressed/fs compressratio 1.00x -
uncompressed/fs used 390M -
# time cp framework.log /compressed/
real 0m24.495s
user 0m0.001s
sys 0m1.260s
-bash-4.4$ zfs get compression,compressratio,used compressed/fs
NAME PROPERTY VALUE SOURCE
compressed/fs compression lz4 inherited from compressed
compressed/fs compressratio 6.37x -
compressed/fs used 61.4M -
Good compression (6x). We save 330MB of disk space here.
No impact to duration. The SPARC S7 core is fast enough.
And now Read Performance
# time cp /compressed/framework.log /tmp; time cp /uncompressed/framework.log /tmp
real 0m17.415s
user 0m0.001s
sys 0m1.354s
real 0m24.479s
user 0m0.001s
sys 0m1.389s
Better results from compressed filesystem. CPU decompression is faster than doing I/O. Need to read 6x the data from uncompressed zfs filesystem.
Summary
With above samples we don't see negative impact when enabling lz4 compression. If you use compressable text files you save lots of disk space while gaining read performance. We start using lz4 on our ZPOOLs by default now.
Subscribe to:
Posts (Atom)