Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace lrzip with zstd #389

Open
AlexDaniel opened this issue Nov 5, 2023 · 1 comment
Open

Replace lrzip with zstd #389

AlexDaniel opened this issue Nov 5, 2023 · 1 comment
Labels
requires more builds Issues that would need more versions to be built whateverable Issues affecting the bot framework (and therefore all of the bots)

Comments

@AlexDaniel
Copy link
Member

See #23 (comment) for some context. When lrzip was introduced, zstd did not have a long-range mode, so lrzip was the only choice to achieve what we needed. Today the situation is different, and we can use zstd for everything.

@AlexDaniel AlexDaniel added whateverable Issues affecting the bot framework (and therefore all of the bots) requires more builds Issues that would need more versions to be built labels Nov 5, 2023
@AlexDaniel
Copy link
Member Author

Actually, because of facebook/zstd#3062, it's not a win-win situation.

For 20 rakudo builds the results are like this:

Command Compression time File size Decompression time
lrzip -q -L 9 ≈26s ≈8M ≈5.3s
zstd -19 --long=31 ≈97s ≈12M ≈0.8s
zstd --long=31 ≈3.7s ≈16M ≈0.8s

It's amazing what zstd can do at just double the size. However, CPU cycles are cheap, so I don't mind if the server uses the idle time to achieve a slightly higher compression rate. That said, I wish zstd was able to reach lrzip's compression ratio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
requires more builds Issues that would need more versions to be built whateverable Issues affecting the bot framework (and therefore all of the bots)
Projects
None yet
Development

No branches or pull requests

1 participant