windshome wrote:
> when a message receive,we read it from a socket channel,but if we think all
> data isn't read completely, we return false ,and we will regist a read event
> in socket selector, we will read from the socketchannel use the buffer in
> old workthread.
>
correct.
>
> in my ProtocolParser I will not clear the buffer before read all data
> completely,so when buffer's postion equals buffer's limit, read from channel
> will return size 0.
>
aha... here comes the question, what do we do when you are expecting
more data and there is no-room (in the ByteBuffer, BB) to fill. This
can happen all the time when the application handles huge number of
bytes.. like 2+Mb data. Every protocol is different. I note that, your
application is having a custom protocol. So, this custom protocol should
have a way to handle the case of chunking / fragmentation. It should
tell the server/client how many bytes are coming still.. and when it's
done sending all.
It's in the protocol.
Say for example, this case, Grizzly's buffer size is 8K. Now, we are
going to handle say, 2Mb from one end to the other (client/server).
Once you read say the first 8K bytes, you need to process them , or
store them, do-whatever, to empty the underlying buffer for the next
read. This is where the parser comes into picture. Ask the following
question.. what would be the optimal size of the buffer, to parse one
complete message. Based on the *optimal size* (during design), you can
set the BB size in Grizzly accordingly, before starting the Controller.
The goal is, as soon as we read "n number of bytes" equivalent to
constitute one single message, we give those read bytes to a parser for
further action. In some application, that action could be masking the
read bytes or alter every eleventh byte ..etc. etc..
Say, for example, some extreme case, your optimal size of BB is 1Mb,
then, that would be a bad design. Why? When doing i/o you want to do it
as fast as as you can considering network bandwidth etc etc.. the
application will run into OOMs and will bailout very fast. Too small
size will make the BB full occasionally and the parser can not parse
completely because, it still needs that one last byte which is expected
be red yet. During those times, you might consider recreating a new BB
(similar to realloc in C) for those the last bits.
So, in a nut shell, it all depends on the Protocol, Parser and
ByteBuffer size.
Good luck :-)
-Harsha
> if I save data in my context and clear the buffer, return false after clear
> buffer,it will correct?
>
> Harsha Godugu wrote:
>
>> windshome wrote:
>>
>>
>>> Oh,I think you are not understand me, my problem is when the bytebuffer is
>>> full,but all my data hasn't read
>>> complete(my data size is 2MB), how can i read all data?
>>>
>>>
>> Hi: Please note that the bytebuffer size that is used for i/o here, in
>> Grizzly, will have a constant threshold. That means, say, if the
>> application can send /receive data (on both server/client side), the
>> size of such a buffer during the process could vary from zero to the
>> threshold. This threshold has got a limit. It can be 8K, 16K or 256k to
>> the max... based on the system limits and application needs. It can not
>> be 8Mb or some gig.. The idea is, the bytebuffer that we use in
>> grizzly (for i/o) is really a BUFFER, meaning a temporary place to read
>> /write to/from a channel. That means, the underlying application needs
>> to store the flushed bits somewhere else. So, as soon as you read say
>> 8K bytes and then if expect more to read, we need store the read bytes
>> somewhere else (in some other object/file/shared memory..etc) and then
>> wait to read the rest of the data until your protocol handler tells you
>> that there are no next messages to parse!
>>
>> hths
>> -Harsha
>>
>>
>>> Jeanfrancois Arcand-2 wrote:
>>>
>>>
>>>
>>>> Salut,
>>>>
>>>> windshome wrote:
>>>>
>>>>
>>>>
>>>>> I process the data to make a digital signature, and return signed data
>>>>>
> to
>
>>>>> client.
>>>>>
>>>>>
>>>>>
>>>> Interesting :-) The number of ByteBuffer created is equal to the number
>>>> of active Threads by default in Grizzly. Right now our default thread
>>>> pool doesn't purge inactive threads, so it may or may not be a problem
>>>> if you need a lot of threads. You might want to replace the default
>>>> thread pool with a one from java.util.concurrent.* that can purge
>>>> inactive thread and their associated byte buffer.
>>>>
>>>> Are you able to determine the size of your expected traffic? If your VM
>>>> is properly tuned (I will let Charlie gives some hints in case you are
>>>> interested) it shouldn't be a problem, assuming you don't need 1000
>>>> threads :-)
>>>>
>>>> A+
>>>>
>>>> -- Jeanfrancois
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Jeanfrancois Arcand-2 wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> sorry for the delay...
>>>>>>
>>>>>> windshome wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I write a server base grizzly,I write a client which send data to
>>>>>>> server
>>>>>>> and
>>>>>>> receive response from server. I send 50 bytes data and 2k data to
>>>>>>> server
>>>>>>> ok,but when I send 20K and 2MB data,server no response.
>>>>>>> I view the code of grizzly,find the ByteBuffer of a workthread capity
>>>>>>> is
>>>>>>> 8192,then I modify it to 81920,then 20k data can receive by server.
>>>>>>>
>>>>>>> If my Server set the init size of ByteBuffer is 81920,I think it would
>>>>>>> use
>>>>>>> to much memory,who can tell me some method,will dynamic just the
>>>>>>> buffer,can
>>>>>>> receive some bytes data ,or some MB data?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> What are you doing with the bytes? Are you saving the bytes somewhere
>>>>>>
> on
>
>>>>>> disk/db (freeing your memory) or you must keep them in memory?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> -- Jeanfrancois
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> My Server's protocol parser code :
>>>>>>>
>>>>>>>
>>>>>>> final ProtocolFilter parserProtocolFilter = new
>>>>>>> ParserProtocolFilter() {
>>>>>>> public ProtocolParser newProtocolParser() {
>>>>>>> return new ProtocolParser() {
>>>>>>> private boolean isExpectingMoreData = false;
>>>>>>> private ByteBuffer byteBuffer;
>>>>>>> private Request message;
>>>>>>>
>>>>>>> public boolean hasMoreBytesToParse() {
>>>>>>> return false;
>>>>>>> }
>>>>>>>
>>>>>>> public boolean isExpectingMoreData() {
>>>>>>> return isExpectingMoreData;
>>>>>>> }
>>>>>>>
>>>>>>> public Object getNextMessage() {
>>>>>>> return message;
>>>>>>> }
>>>>>>>
>>>>>>> public boolean hasNextMessage() {
>>>>>>>
>>>>>>>
>>>>>>> ByteBuffer dup = byteBuffer.duplicate();
>>>>>>> System.out.println( "byteBuffer.position2()="
>>>>>>> +
>>>>>>> byteBuffer.position() );
>>>>>>> if (byteBuffer.position() == 0){
>>>>>>> System.out.println("byteBuffer.position()
>>>>>>> ==
>>>>>>> 0");
>>>>>>> isExpectingMoreData = true;
>>>>>>> return false;
>>>>>>> }
>>>>>>> dup.flip();
>>>>>>>
>>>>>>>
>>>>>>> if( dup.remaining()< 4 ){
>>>>>>> isExpectingMoreData = true;
>>>>>>> return false;
>>>>>>> }
>>>>>>>
>>>>>>> byte[] bs = new byte[4];
>>>>>>> dup.get(bs);
>>>>>>> int len= (bs[0] << 24) + (bs[1] << 16) +
>>>>>>> (bs[2]
>>>>>>> <<
>>>>>>> 8) + (bs[3] << 0);
>>>>>>>
>>>>>>> if( dup.remaining() < len ){
>>>>>>> isExpectingMoreData = true;
>>>>>>> return false;
>>>>>>> }
>>>>>>> byte[] data = new byte[len];
>>>>>>> dup.get( data );
>>>>>>>
>>>>>>> try {
>>>>>>> message = new ByteRequest(data);
>>>>>>> } catch (Exception e) {
>>>>>>> e.printStackTrace();
>>>>>>> message=null;
>>>>>>> return false;
>>>>>>> }
>>>>>>> return true;
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> public void startBuffer(ByteBuffer bb) {
>>>>>>> byteBuffer = bb;
>>>>>>> }
>>>>>>>
>>>>>>> public boolean releaseBuffer() {
>>>>>>> byteBuffer = null;
>>>>>>> message = null;
>>>>>>> return false;
>>>>>>> }
>>>>>>>
>>>>>>> };
>>>>>>> }
>>>>>>> };
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>