dev@shoal.java.net

Re: [Shoal-Dev] Re: question concerning abstract transport branch MessageImpl.MAX_TOTAL_MESSAGE_LENGTH

From: Joseph Fialli <joe.fialli_at_oracle.com>
Date: Wed, 12 May 2010 11:50:02 -0400

Hi Bongjae,

Comments inline below.

On 5/11/10 10:13 PM, Bongjae Chang wrote:
> Hi Joe,
>
> I saw that max message size is set to 128k from branch rev. 863.
>
>
When Alexey rewrote GrizzlyMessageProtocolParser to allow it to work
across buffers, I was able to increase the
max message size from 8K to a much higher size. Either this will be
configurable in future or will be increased, seems
large enough for time being. For the TCP side, we are definitely not
restricted to a message size that is smaller than
the buffering size.
> But, I think that BlockingIOMulticastSender.java's DEFAULT_MULTICAST_PACKET_SIZE(16k bytes) should also be increased.
>
>
I agree. However, we should probably not require MULTICAST_PACKET_SIZE
to have to be same size as max message size.

Perhaps bump the DEFAULT size to 64k. The default in
GrizzlyNetworkManager config of MULTICAST_PACKET_SIZE is 64K,
unsure if that is being used or DEFAULT in BlockingIOMulticastSender.

When broadcasting a message over UDP, if message size is larger than
configured MULTICAST_PACKET_SIZE,
default to using synchronous virtual TCP broadcast to send message.

I can do this or you can, if you would like to.

There is a test script runbroadcastmessage.sh that I will be using to
verify testing of the issue you reported is fixed.
I did confirm IOException trying to do a UDP broadcast send of a message
larger than 64K in current source code base.

-Joe

> For example,
> Now user can broadcast a large multicast packet(over 16k) to the group by GroupHandle#sendMessage(), then the packet won't be able to be parsed correctly because BlockingIOMulticastSender.java's multicast socket will read incomplete DatagramPacket(max 16k bytes) once.
>
> What do you think?
>
> Regards,
> Bongjae Chang
>
>
> ----- Original Message -----
> From: "Bongjae Chang"<bongjae.chang_at_gmail.com>
> To:<joe.fialli_at_oracle.com>
> Cc:<dev_at_shoal.dev.java.net>
> Sent: Wednesday, March 10, 2010 1:53 PM
> Subject: Re: question concerning abstract transport branch MessageImpl.MAX_TOTAL_MESSAGE_LENGTH
>
>
>
>> Hi Joe,
>>
>> I agree that the size is small.
>>
>> Actually, I couldn't determine the default size(8k was maybe temporary size). :-)
>>
>> So I just set the default to Socket's send(receive) buffer size.
>>
>> i.g. Socket#getSendBufferSize(), Socket#getReceiveBufferSize(), DatagramSocket#getReceiveBufferSize(), DatagramSocket#getSendBufferSize() and ServerSocket#getReceiveBufferSize() were 8k.
>>
>> So I also think that you had better change the current constant value.
>>
>> But you should consider the default allocation size of DatagramPacket in BlockingIOMulticastSender.java. The default allocation size is about 16k.
>>
>> So I think that maybe you should also change the default allocation size in BlockingIOMulticastSender.java if you will change MessageImpl.MAX_TOTAL_MESSAGE_LENGTH.
>>
>> In addition, I think that it is also better that BlockingIOMulticastSender.java considers MulticastSocket#setSendBufferSize() or #setReceiveBufferSize() APIs.(But I don't know how it has the advantage of efficiency because I didn't have tests of those APIs).
>>
>> Thanks.
>>
>>
>> Regards,
>> Bongjae Chang
>>
>>
>> ----- Original Message -----
>> From: "Joseph Fialli"<joe.fialli_at_oracle.com>
>> To:<bongjae.chang_at_gmail.com>
>> Cc:<dev_at_shoal.dev.java.net>
>> Sent: Wednesday, March 10, 2010 1:10 AM
>> Subject: question concerning abstract transport branch MessageImpl.MAX_TOTAL_MESSAGE_LENGTH
>>
>>
>>
>>> Bongjae,
>>>
>>> We have found that the default MessageImpl.MAX_TOTAL_MESSAGE_LENGTH is
>>> set to a rather modest size of 8K.
>>> This size is small for a particular use case we have in mind. I was
>>> looking to seek your advice on the advantages
>>> of having the default size so small.
>>>
>>> I was considering the following options:
>>>
>>> 1. Change the current constant value from 8192 to 64K. // requiring
>>> the least amount of change by potentially having a negative unforeseen
>>> impact.
>>>
>>> 2. Leave the default as is but make this constant configurable via a
>>> Shoal property that is specified when gms client joins group.
>>>
>>> Before I started working on this change, I wanted to understand the
>>> motivation behind the current value.
>>>
>>> Thanks in advance for your comments.
>>>
>>> -Joe
>>>
>>>
>> >