On Jan 25, 2010, at 8:22 PM, Tatu Saloranta wrote:
> On Mon, Jan 25, 2010 at 12:54 AM, Paul Sandoz <Paul.Sandoz_at_sun.com>
> wrote:
>>
>> On Jan 21, 2010, at 9:38 PM, Tatu Saloranta wrote:
>>
> ...
>>> I have a related question: would it be easy to add support for
>>> alternate compression codecs?
>>> Results I have seen with LZF (used by H2 DB, added to Voldemort) are
>>> very encouraging,
>>
>> Any pointers to results?
>
> Unfortunately I have not had time to clean things up. After creating
> simple codec for Voldemort, I was hoping to create solid Japex-based
> test to get reproducible numbers. So far I only have pile of
> micro-benchmarks, comparing gzip and lzf. Problem is just not having
> (had) time to do it yet.
> So it is still on my backlog. I will send an update here if and when I
> get there.
>
> For what it's worth my observation was that whereas gzip can add 2-3x
> overhead (more so for writing), lzf can actually have overhead more in
> line with data binding (as in +50%), and it is especially more
> efficient for writing.
> Compression rate is lower (it does not do huffman-coding part), more
> prominently so for most compressible contents (instead of getting,
> say, 93% compression for very repetitive fluffy xml, you'd get 84% or
> so).
>
OK. This kind of reminds me of the various compression tradeoffs, size/
performance, when enabled for ZFS (which can support LZJB or GZIP).
Paul.