Jeanfrancois Arcand wrote:
Salut,
Oleksiy Stashok wrote:
Hello Ken,
I have also posted a blog entry about this
at
http://blogs.sun.com/ejcorba/entry/bytebufferstreams_in_grizzly
great! thank you!
It seems that I cannot commit to trunk/www,
so the README.html for the
package is not viewable without downloading it first. It would be
useful
to fix that.
Now you have Content Developer role, so can try it again :)
As for the implementation, IMHO, It looks very clear!
The first idea I have... May be it makes sense to separate
ByteBufferStreams API from Allocator API?
+1 on my side as well!
A+
-- Jeanfrancois
So they will be independent.
It would be great to have Slabs/SlabsPool as
one of possible Grizzly 2.0 MemoryManager implementations. This could
be really interesting to have MemoryManager, which have smart
ByteBuffer pooling implementation.
ByteBufferStreams API could be interesting to use as common streaming
API, which will be possible to use with any kind of buffers:
ByteBuffer, byte[]...
What do you think?
I'm looking forward to start use that in Grizzly 2.0! :)
The key requirement for the streams is that they use BufferWrapper as
their
representation of an allocated buffer. Their are a couple of reasons
for this:
- When a stream is done with a buffer, it needs to get rid of it.
This is the BufferWrapper.dispose call, which relies on the
BufferWrapper knowing the Allocator used to allocate it.
- The Reader relies on the BufferWrapper.prepend method to handle
split primitives. prepend works by reserving a small amount of space
at the start of the buffer (8 bytes normally) which is large enough to
handle the largest primitive (long or double) when split between two
buffers.
These requirements come from the Reader: the Writer has no significant
requirements on the
BufferWrapper.
The BufferWrapper knows both the allocator and the slab used to
allocate it. Both are needed in
the BufferWrapper.trim() and BufferWrapper.dispose() methods:
- trim is used to allocate a buffer as large as the free space in a
slab for reading, so that very large reads can be made efficiently.
After reading the data, trim() is called in order to reduce the space
needed to the actual data read. This works best in the case where a
slab is used by a single reader thread, which is the expected mode of
use.
- dispose is used to release a buffer once the stream is done with
it (directly from the Reader, most likely as part of the BufferHandler
in the Writer). Here both the allocator and the slab are needed: the
slab tracks whether or not all of its allocated space has been freed,
and the allocator (in the pool case by delegating to the SlabPoolImpl)
tracks empty, partial and full slabs for recycling memory.
So to separate these:
- It's easy (and sensible) to use the Allocator without the
streams. This is probably a good idea for simple HTTP and raw memory
access cases, where decoding incoming data as binary data is not
required.
- The streams require the allocator as discussed above. If other
memory management schemes are contemplated beyond what I have written,
it would be best to express those in the Allocator/Slab framework in
order to be able to use the streams.
- If you want re-arrange the packages, I'd recommend something like
(package names in bold italics):
- streams (or perhaps a different package name)
- Reader
- Writer
- StreamFactory
- impl
- buffers (or perhaps a different package name)
- Allocator
- AllocatorFactory
- BufferWrapper
- impl
- AllocatorBase
- AllocatorImpl
- AllocatorPoolImpl
- Slab
- SlabPoolImpl
Re-factoring the package names this way is pretty easy in
netbeans. Should these packages be directly under
org.glassfish.grizzly, or is there another intermediate package after
grizzly?
Ken.