users@grizzly.java.net

Re: cutting up logical sets of bytes from data stream? (in 2.0)

From: Oleksiy Stashok <Oleksiy.Stashok_at_Sun.COM>
Date: Wed, 17 Dec 2008 11:52:49 +0100

Hi Emit,

>
> I agree, it's a cool application of annotations... reminds me of
> JPA, which is both good and bad... I think this can/will get
> quite hairy once the developer wants to customize headers and
> data format down to each byte order, nibble and bit.
Agree. I also have some usecase with SOAP/TCP transport for Web
services, which headers are nibble aligned. For such a usecase it will
be required to provide custom Transformer.


> I took a quick look at the preon project mentioned in another
> reply in this thread, and that's the level of complexity that I'm
> talking about... similar free projects like google protobuf that
> deal with (un)marshalling data in a language neutral manner...
> This will need a lot of work to make it robust enough to handle
> any nontrivial protocol... I don't know if it's really worth the
> time when you can refine the low level api first instead :-D
Actually I like the idea of using preon for Grizzly, as really don't
want to spent time on reinventing a wheel. Though still not sure it
confirms to all requirements we have...


> IMO, a common use case of these custom binary protocol decoders
> is to interface with legacy protocols from C apps, which do all
> kinds of silly bit manipulations. Java to Java, there's a lot of
> options already...
I agree, but still didn't see something working with NIO. I could miss
something for sure.


> If I had a message class similar to Alex's GIOPMessage, but my
> bodyLength should be considered a 16 bit unsigned integer
> (unsigned short), you would probably need to add another
> annotation there to reflect that, since in Java I'd most likely
> just stuff it in an int until it's time to go through the
> network. Furthermore what if I didn't want to keep each byte
> separately G,I,O,P,major,minor,flags and instead had Header,
> Version, BitFlags objects, each with their own way of going
> through the stream... and java.util collections... I'm having
> nightmarish flashbacks from dealing with JPA's one to many, many
> to many,etc annotations.
I agree with you. But for easier usecases it may help a lot. For more
difficult - yes it could become easier to do parsing programmatically
with own code, than providing tons of annotations.


> Thanks to Alex's earlier suggestions I got a sort of working grizzly2
> based implementation of my protocol... then I tried mina 2 and was
> shocked that all the plumbing was in place. I cooked up a decoder
> based on their standard CumulativeProtocolDecoder and had it
> working in a few minutes...
>
>
> For example, there's a method in mina's IoBuffer called
> prefixedDataAvailable(x,y), which returns true if there are n
> bytes waiting to be read where n is represented by the next x bytes,
> usually 2 or 4...
>
> I don't know offhand if such a thing exists in grizzly
> (I apologize if it exists), and I admit, it's trivial to implement,
> but small things like that which come up frequently when decoding
> a binary protocol could be either refined or better documented.
No, there is no such feature in Grizzly Buffer.
Though, I think we can add similar feature it to our Streams
implementation.
What do you think, John?


> In the meantime I have two branches in my cvs, one for mina2 and
> one for grizzly2 so I'm just gonna try both back and forth. They
> are actually eerily similar.
Ok. It makes sense.

Thank you very much for feedback!

WBR,
Alexey.

>
>
> -Emit
>
> On Tue, Dec 16, 2008 at 06:38:30PM -0500, Survivant 00 wrote:
>> that's really nice.
>>
>> is it possible to do the same thing with a variable length message ?
>>
>> you could have a annotation for the start (probably not needed) or
>> the end
>> of query.
>>
>> a message is completed when you found the eoq. and if you reach
>> the max
>> buffer size(annotation) you throw a Exception ? (maxBuffersize..)
>>
>> that will save me 200 lines of code for the parser in 1.x .
>>
>>
>>
>> 2008/12/16 Oleksiy Stashok <Oleksiy.Stashok_at_sun.com>
>>
>>>
>>>> Hi Emit,
>>>
>>> coming back to your question, if it's not late...
>>>
>>> What is the best "grizzly 2.0" way to cut up the incoming stream
>>>> into these logical packets of data, hopefully without editting
>>>> the grizzly internal classes? I guess I can just aggregate
>>>> these getMessage Buffers myself after the handleReads but in that
>>>> case I don't see a reason to use this framework with all the
>>>> overhead involved.
>>>>
>>> Yesterday we've added new feature in Grizzly 2.0, which may help
>>> you with
>>> message parsing.
>>> Please take a look here [1]
>>>
>>> Thanks.
>>>
>>> WBR,
>>> Alexey.
>>>
>>> [1]
>>> http://www.nabble.com/Grizzly-2.0%3A-Smart-codec-td21033363.html#a21033363
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>