Way Faster VPS backups https://www.softaculous.com/board/index.php?tid=14659 <![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46694 @softacoulus: please add the ability to specify the compressor/decompressor that should be used for backups so we do no longer need to do it at OS level.
]]>
Mon, 22 Jul 2019 09:26:25 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46694
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46687
Quote From : wolke July 18, 2019, 10:11 am
There might be an even better alternative than lzop, specially when it comes to cpu time. I´am not into academic overoptimization but it seems that´s worth testing under reallife conditions.

Source: https://hannuhartikainen.fi/blog/everyday-compression-tool/

Abstract:
tool    filesize      time  CPU time
pigz      35125348    1.46    8.82
lbzip2  20301117      5.33  32.16
pixz    19333820      34.90  211.77
pzstd  39684543      0.45  2.51



Great find, this is really impressive:

Using Lzop:

Code
[root@node02]# time /bin/dd if=/dev/kvmvg/backup-v1002 bs=24M iflag=direct | /bin/lzop -2 > /backup/backup-v1002-lzop.tar.gz
512+0 records in
512+0 records out
12884901888 bytes (13 GB) copied, 53.2806 s, 242 MB/s

real    0m53.527s
user    0m21.149s
sys    0m28.381s


Using pzstd:

Code
[root@node02]# whereis pzstd
pzstd: /usr/bin/pzstd /usr/share/man/man1/pzstd.1.gz
[root@node02 tmp]# time /bin/dd if=/dev/kvmvg/backup-v1002 bs=24M iflag=direct | /usr/bin/pzstd -2 > /backup/backup-v1002.tar.gz
512+0 records in
512+0 records out
12884901888 bytes (13 GB) copied, 20.8995 s, 617 MB/s

real    0m23.406s
user    0m51.984s
sys    0m25.450s


Whats interesting is that the filesize with lzop is 5.1GB and with pzstd it's 3.8GB]]>
Mon, 22 Jul 2019 05:09:07 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46687
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46647
Source: https://hannuhartikainen.fi/blog/everyday-compression-tool/

Abstract:
tool    filesize      time  CPU time
pigz      35125348    1.46    8.82
lbzip2  20301117      5.33  32.16
pixz    19333820      34.90  211.77
pzstd  39684543      0.45  2.51

]]>
Thu, 18 Jul 2019 10:11:55 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46647
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46643
Thanks for the heads up on this.

@Softaculous - please can you allow us to specify the location to lzop so we don't need to do the symlink.

]]>
Thu, 18 Jul 2019 08:46:30 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=2#p46643
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p46640
We tested the replacement and it seemed to work perfectly but then...
We had several servers running into kernel panic after a kernel update took place and the servers had been rebooted.

The console message was:

Quote
kernel panic not syncing vfs unable to mount root fs on unknown-block"


Here is why:
When unpacking the kernel image, lzop will be used also (because of the changed symlink) but fails with several errors:

Quote
cpio - premature end of archive

Unable to decompress /boot/initramfs-3-10[....] - unknown format


and on cloudlinux server also:

Quote
Error communicating with CLN - Bad request

lzop can´t decompress several archives and thus the kernel update (exactly the initramfs generation) fails.

This happened exactly for kernel 3.10.0-962.3.2.lve1.5.25.12.el7.x86_64 on Centos7/CloudLInux

This really nasty bug has been detected by incredible CloudLinux support staff (in 3rd level support). They removed the /usr/bin/pigz symlink to lzop and initramfs generated smoothly and the server could reboot into the new kernel.
We will stay with pigz binary therefore.


]]>
Thu, 18 Jul 2019 08:14:37 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p46640
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45952
Quote From : wolke May 21, 2019, 9:20 am
Thanks for sharing this. I wonder what will happen on OS updates. Did you ever got trouble with that?


Absolutely no problems at all, works perfect for backups and restores and to get around updates, you just add pigz to your package managers excludes so it won't update pigz and overwrite the symlink.]]>
Tue, 21 May 2019 10:50:27 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45952
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45949 ]]> Tue, 21 May 2019 09:20:07 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45949 <![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45945
The option flags for pigz and lz4 are identical, so it's a complete drop in replacement.

With compression, the backups are pretty fast (390MB/sec on the lowest IO class) and there is little to no load on the servers when backups are done.
]]>
Tue, 21 May 2019 03:51:15 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45945
<![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45943 Mon, 20 May 2019 21:34:09 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45943 <![CDATA[Way Faster VPS backups]]> https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45942 Mon, 20 May 2019 21:32:40 GMT https://www.softaculous.com/board/index.php?tid=14659&tpg=1#p45942