dev@grizzly.java.net

Re: SlabMemoryFilter

From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
Date: Tue, 06 Jan 2009 11:42:59 -0500

Salut,

catching up...

Oleksiy Stashok wrote:
> Hi John,
>
>>>
>>>> Unfortunately SlabMemoryFilter main purpose is to efficiently read
>>>> in bytes .
>>>> 1) Therefore it allocates a very big ByteBuffer.
>>>> 2) Invokes connection.read()
>>>> 3) It then trims the very big ByteBuffer which actually gives
>>>> SlabMemoryManager the unused space back.
>>>> 4) It sets the "filled in ByteBuffer" as the contexts's message.
>>>>
>>> But it is possible to do in general way, not just with slabs, right?
>>> (We have trim() method declared in common Buffer interface).

Just curious, how many instance of big byte buffer we will have? One per
thread or one per manager?


>>>
>>>> SlabMemoryManager relies on the further processing by StreamReader
>>>> to be given back the used memory (Bytebuffer)
>>> Is it possible to avoid such a dependancy, so SlabMemoryManager could
>>> be used in all usecases, not just Streams?
>> there is no problem. you can do that right now.
>> it was only meant as a first usage case
> Ok, that's good.
>
>
>>>> This is different to Grizzly's 2.0 default memory processing which
>>>> is done in TCPNIOTransportFilter.
>>>> For example there is no ByteBuffer associated with each Thread nor
>>>> is there any storing of ByteBuffer in the connection.
>>> Ok, we can remove ByteBuffer<->Thread association once
>>> SlabMemoryManager will become default memory manager.
>> Well if we are brave we should do this as soon as possible ((-:
> Agree. Once you'll say it's ready :)
>
>
>>>
>>> As for storing ByteBuffer in a connection. Think there is not such a
>>> usecase for Slabs, because you mentioned it works just with Streams,
>>> right?
>>> But what if we will want to make it general default memory manager for
>>> Grizzly? In this case, IMHO, we may need to store some Buffers in a
>>> connections, because of incomplete parsing or so...
>> right! It wasn't clear to me that you intended to go so far ((-:
>> yes you would have a list with "ready to be parsed buffers" stored in
>> the connection.

Are those bb the bug one described above, or view to a big bb? Seems
dangerous to manipulate such big bb, specially if you think about
implementing an HTTP server which support POST with a large body.



>>
>> When using Slab-Memory-Management you have the choice of 2 different
>> implementations :
>> SlabMemoryManagerImpl using garbage collection or
>> SlabPoolMemoryManagerImpl using a pool.

+1 I use grizzly on iPhone and I need to reduce gc utilization :-)

>>
>>
>> SlabMemoryManagerImpl is in some respect similar to
>> ByteBufferViewManager which is the
>> current default MemoryManager. This switch might lead to the creation
>> of more
>> ByteBuffers instances (these instances are only slices) since we
>> cannot reuse a ByteBuffer-slice stored in WorkerThread anymore.
>> So we might loose some memory effiency here?
> May be it's not critical. We will check this using perf. tests.

I would think a property should not be bad as well.


>
>
>> But then if memory effiency is important you could use
>> SlabPoolMemoryManagerImpl.
>> The drawback here is that users have to make sure that space is
>> allways given back to the pool by
>> calling ByteBufferWrapper.dispose(). Otherwise you really get in
>> trouble ((-:
> First of all we can try to hide Buffers as much as possible. We will
> have streams, right? :)
> On other hand it could be good to avoid having troubles, if Buffer is
> not getting released. Can we just forget it and let GC process it?

Not sure I like it, but I think Netty is successfully doing that. But I
would think re-using is better IMO, specially if the buffer is larger.

A+

-- Jeanfrancois


>
> Thank you.
>
> WBR,
> Alexey.
>
>>>> How was your holiday?
>>> Oh, as usual. Nice and short :))
>>> What about you?
>> also very nice ((-: And the good news is that in 10 days I have
>> another holiday haha ((-.
>>
>> Many Greetings
>> John
>>
>> --
>> Sensationsangebot verlängert: GMX FreeDSL - Telefonanschluss + DSL
>> für nur 16,37 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K1308T4569a
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>