On 23 June 2012 03:13, Scott Ferguson <ferg_at_caucho.com> wrote:
> On 06/22/2012 04:29 PM, Danny Coward wrote:
>
>
> Issues covered in v002
>
> * Message representation: The API presents messages as String, byte[] and
> Iterators thereof. No nio classes, no IO streams. The goal of this approach
> was to allow buffered messages (for convenience) and chunked messages (for
> efficiency) without limiting the implementation choices IO/NIO/combination
> of implementors of the API.
>
>
> Without streams, it's not a serious proposal.
>
Well streams are only one way to solve the issue of large messages....
however I agree that we do need to skin that cat and the proposed iterator
is not a nice solution.
The consumer should be able to consume a message in little chunks - but the
size of those chunks should be unrelated to the frame size (which should be
unknown to the application). Streams allows the consumer to read 1 byte
at a time or n bytes, so they are good for applying a memory ceiling to the
application.
However, stream can also suffer from their blocking API nature as threads
are needed to read from the streams.
So we will need callbacks to allow asynchronous consumption - these
callbacks can says that there is data available to be read from a stream -
or it is possible to bypass the stream and the callbacks can deliver the
content in consumable sized chunks - but importantly this chunk size should
be unrelated to (or at least decoupled from) the received frame size -
instead they are related to how much data the implementation/application is
prepared to handle in a single operation.
Streams with async callbacks are probably the style of interface that will
give least surprise , but they are not the only solution.
cheers
--
Greg Wilkins <gregw_at_intalio.com>
www.webtide.com
Developer advice, services and support
from the Jetty & CometD experts.