-------- Original Message --------
Subject: RE: A quick prototyping of ByteArray interface
Date: Tue, 29 Apr 2008 08:06:32 -0700
From: Outside - Karl's ACM <kpauls_at_acm.org>
Reply-To: dev_at_mina.apache.org
To: 'dev_at_mina.apache.org' <dev_at_mina.apache.org>
> From: trustin_at_gmail.com [mailto:trustin_at_gmail.com]
>
> So.. the introduction of CompositeByteArray makes direct exposure of NIO
> ByteBuffers somewhat difficult, although workarounds (disconnected
> buffer by on-demand/manual copy or returns an array of buffers) are
> available.
>
> WDYT?
IMO the most essential use of multiple buffers is in the Scatter/Gather
interfaces. So at least an array returning method is necessary.
I'm also quite interested in the zero-copy front. Grizzly and MINA
currently differ in this regard. I think that Grizzly without some
modification has no support of zero-copy making it unsuitable for high
volume data, for example something like a pure Java logging server with
some kind of analysis/load to consume the data. I can begin testing this
use case in MINA, but I have some ideas to make the memory utilization
more manageable.
In this use case some slices of buffers could persist for longer than
others. One approach to minimize copies and allow more memory to be
reclaimed would include evaluating allocating smaller buffers (perhaps
512 bytes or smaller, varied as part of the experiment) and persisting
these until the data has been evaluated. These buffers could be
periodically tested for their waste and copied to free up the waste.
However, to keep reads efficient I would want to experiment with the
GatheringByteChannel methods.
For Grizzly I think this means creating a new ProtocolFilter and
extending some part of the entire filter chain to support it. For MINA I
will have to do a little more reading. Grizzly will also be relying
entirely on the garbage collector to reclaim buffers and I may have to
craft a class to track buffer splits, and by then I might as well
implement a buffer pool.
-karl