dev@grizzly.java.net

Re: SlabMemoryFilter

From: John ROM <snake-john_at_gmx.de>
Date: Tue, 06 Jan 2009 14:04:38 +0100

Hi Alexey,
se inline
> Hi John,
>
> >>
> >> Hi John,
> >>
> >> I had some time today to take a look at Slabs memory management
> >> code in Grizzly 2.0. IMHO it looks very good, thank you!
> >> Have one idea about SlabMemoryFilter... IMHO we can avoid to use
> >> such a filter, but correct me if I'm wrong. The idea is to move the
> >> logic from this Filter to SlabMemoryManager(Base). So, when you'll
> >> call SlabMemoryManager.allocate(...) - it will check current thread
> >> and associated slab allocator. So we can avoid of using additional
> >> Filter.
> >> What do you think?
> > Hmm I am not too sure.
> >
> > I 'm ok with the allocate(..) proposal.
> > I would create a ThreadAttachedMemoryManger which delegates to
> > SlabMemoryManager.allocate(...)
> > because I would like to keep the dependency on WorkerThread out of
> > SlabMemoryManager (nicer for testing).
> Ok, +1.
>
> > Unfortunately SlabMemoryFilter main purpose is to efficiently read
> > in bytes .
> > 1) Therefore it allocates a very big ByteBuffer.
> > 2) Invokes connection.read()
> > 3) It then trims the very big ByteBuffer which actually gives
> > SlabMemoryManager the unused space back.
> > 4) It sets the "filled in ByteBuffer" as the contexts's message.
> >
> But it is possible to do in general way, not just with slabs, right?
> (We have trim() method declared in common Buffer interface).
>
> > SlabMemoryManager relies on the further processing by StreamReader
> > to be given back the used memory (Bytebuffer)
> Is it possible to avoid such a dependancy, so SlabMemoryManager could
> be used in all usecases, not just Streams?
there is no problem. you can do that right now.
it was only meant as a first usage case

>
>
> > This is different to Grizzly's 2.0 default memory processing which
> > is done in TCPNIOTransportFilter.
> > For example there is no ByteBuffer associated with each Thread nor
> > is there any storing of ByteBuffer in the connection.
> Ok, we can remove ByteBuffer<->Thread association once
> SlabMemoryManager will become default memory manager.
Well if we are brave we should do this as soon as possible ((-:

>
> As for storing ByteBuffer in a connection. Think there is not such a
> usecase for Slabs, because you mentioned it works just with Streams,
> right?
> But what if we will want to make it general default memory manager for
> Grizzly? In this case, IMHO, we may need to store some Buffers in a
> connections, because of incomplete parsing or so...
right! It wasn't clear to me that you intended to go so far ((-:
yes you would have a list with "ready to be parsed buffers" stored in the connection.

When using Slab-Memory-Management you have the choice of 2 different
implementations :
SlabMemoryManagerImpl using garbage collection or SlabPoolMemoryManagerImpl using a pool.

SlabMemoryManagerImpl is in some respect similar to ByteBufferViewManager which is the
current default MemoryManager. This switch might lead to the creation of more
ByteBuffers instances (these instances are only slices) since we
cannot reuse a ByteBuffer-slice stored in WorkerThread anymore.
So we might loose some memory effiency here?

But then if memory effiency is important you could use SlabPoolMemoryManagerImpl.
The drawback here is that users have to make sure that space is allways given back to the pool by
calling ByteBufferWrapper.dispose(). Otherwise you really get in trouble ((-:



>
> > How was your holiday?
> Oh, as usual. Nice and short :))
> What about you?
also very nice ((-: And the good news is that in 10 days I have another holiday haha ((-.

Many Greetings
John

-- 
Sensationsangebot verlängert: GMX FreeDSL - Telefonanschluss + DSL 
für nur 16,37 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K1308T4569a