jfarcand@dev.java.net wrote:
https://grizzly.dev.java.net/issues/show_bug.cgi?id=84
User jfarcand changed the following:
What |Old value |New value
================================================================================
Assigned to|issues@grizzly |oleksiys
--------------------------------------------------------------------------------
Target milestone|milestone 1 |1.8.0
--------------------------------------------------------------------------------
------- Additional comments from jfarcand@dev.java.net Thu May 22 16:39:59 +0000 2008 -------
Let's see if we can have something for 1.8.0
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@grizzly.dev.java.net
For additional commands, e-mail: issues-help@grizzly.dev.java.net
I thought this is an interesting bug (or perhaps RFE?).
I've been doing some work for parallel marshaling
that may be relevant here.
Part of the infrastructure behind my project is two classes:
ByteBufferWriter and
ByteBufferReader. Ignoring many details, ByteBufferWriter looks like:
public class ByteBufferWriter {
public interface BufferHandler {
/** Dispose of the current ByteBuffer and return a new one.
*/
EmergeBuffer overflow( EmergeBuffer current ) ;
/** Dispose of current buffer. Called
* when closing a ByteBufferWriter.
*/
void close( EmergeBuffer current ) ;
}
ByteBufferWriter( BufferHandler handler, EmergeBuffer buffer ) { ... }
// The rest of the API has void putXXX( XXX data ) calls for the various
// primitive data types, plus putXXXArray( XXX[] data ) for arrays of primitives.
// I also have a VarOctet type that we can ignore for now.
...
}
The EmergeBuffer is a wrapper around a byte[]-backed indirect
ByteBuffer (I have no intention
of using direct ByteBuffers, because Charlie's work should make them
irrelevant at some point).
Its interface is:
public class EmergeBuffer {
/** Allocate a buffer that has room for dataSize data, and also
* may prepend up to headerSize bytes before the start of the data.
*/
public EmergeBuffer( int headerSize, int dataSize ) { ... }
public ByteBuffer buffer() { ... }
/** Reset the buffer for reading. Moves ByteBuffer position back to 0.
*/
public ByteBuffer reset() { ... }
/** Prepend data from header.position() to header.limit() to the
* current buffer. This will change the object instance returned by buffer()!
* @throws IllegalArgumentException if header.remaining() > headerSize.
*/
public ByteBuffer prepend( ByteBuffer header ) { ... }
}
EmergeBuffer is implemented in such a way that the maximum amount of
data
copied is headerSize (it uses two internal ByteBuffers, one that hides
the space
reserved for the header, and a second one that slices the first after
the header,
so the position 0 of the second (which is returned by the buffer()
call) is
position headerSize of the first.
I created EmergeBuffer to solve an annoying problem in the
ByteBufferReader:
dealing with split primitive types. This occurs when writing
primitives to a
stream, but the underlying transport delivers the data at arbitrary
boundaries,
possibly in the middle of a primitve. Since the maximum primitive size
is 8 bytes,
that is the most data that is every copied on a read.
Getting back to the ByteBufferWriter, it's operation is quite simple:
whenever a
write method is called, a check is first made to see if enough space is
left in the
current buffer (in the case of an array, all the data that fits is
written to the
current buffer). When no more data can be written, but we still have
data to
write, the overflow method on the handler is called, allowing the
handler to
do whatever it needs to do with the buffer. This could include:
- write the data to a file
- internally queue it in a pipe of some sort
- write the data to a communication channel of some sort
This allows hiding all of the ByteBuffer management behind a stream API
implemented on top of the ByteBufferWriter.
I should probably define an interface that includes just the write
operations
in this case. java.io.DataOutput is somewhat similar, but includes
higher-level
details like writeUTF and lacks methods for arrays of primitives (which
are important
for efficiency reasons). org.omg.CORBA.DataOutputStream includes
arrays of primitives,
but because it's part of CORBA, it includes CORBA types not relevant to
other applications,
and has IDL-style method names
ByteBufferReader looks like:
public class ByteBufferReader {
/** Create a new ByteBufferReader.
* @param timeout Time in milliseconds to wait for more data.
* @throws TimeoutException when a read operation times out waiting for more data.
*/
public ByteBufferReader( long timeout ) { ... }
/** Allocate a buffer that reserves a large enough header to handle the largest
* primitive type.
*/
public static EmergeBuffer allocate( int size ) { ... }
/** Add more data for the reader thread to read.
*/
public void dataReceived( EmergeBuffer bb ) { ... }
// The rest of the API contains getXXX( XXX arg ) and getXXXArray( XXX[] arg )
// methods similar to the write case. Note that we assume that the length of an
// array is known, so that the correct array size can be allocated before the
// getXXXArray call. For example, many protocols may marshal the array length
// in some form before the array contents.
}
A ByteBufferReader could be attached to a connection, so that buffers
are appended
to the ByteBufferReader as they are received. As in the writer case,
ByteBufferReader
should implement an appropriate API.
One thing missing from the ByteBufferReader API is a handler with a
dispose( EmergeBuffer ) method
for handling buffers that we have finished using. This doesn't matter
much for my
current implementation, but would be important if the underlying buffer
management
is using direct ByteBuffers.
Currently these classes are fairly well tested, in the case where a
writer writes to
a pipe from which a reader reads synchronously. There are some places
that need
to be addressed in the code to make it properly MT safe.
Ken.