users@grizzly.java.net

Re: Grizzly production ready? ConcurrentHandler & scenarios

From: felixx <dolaru_at_sympatico.ca>
Date: Tue, 17 Mar 2009 06:30:02 -0700 (PDT)

Hi and thanks for your comments and for 1.9.9
I want you guys to have a big hit at JavaOne so I'll take your word that the
news will be good.
Regarding the number of threads used to execute the handlers, I just feel
that this way we play it safer and a bunch of lazy clients/weird network
condition will not affect everybody. Also I would like the latency to be
almost the same for everybody.
I have 2 scenarios/apps:
1. small payload, under 1k, will try first streaming (I guess this is
lighter for the server) for each client and fall back to long polling if the
client/network doesn't behave;
2. only streaming, payload can be big, 50-100K
There are to many variables so i'll stop here. So short story do you think
having a bigger treads pool would help? 100-200 are not so expensive in
terms of memory & switching and if it helps to play it safer, why not?


Jeanfrancois Arcand-2 wrote:
>
> Salut,
>
> felixx wrote:
>> Salut et merci, I have to say you almost convinced me :-)
>> 1. Can you share some results please, just to get an idea, not details.
>> What
>> I'm trying to achieve is not so big. Let's say 5000 long poll connected
>> clients, 2000 sent messages/s (I.e. 1 msg sent to 2000 clients).
>
> I need to ask my co-speaker as we were planning to do a big hit during
> JavaOne :-). Probably yes :-)
>
>
>> Is it true that their solution is close to what is considered to be added
>> in
>> servlet 3.0?
>
> No, Servlet 3.0 is a complete re-think of Comet. Nothing from
> Grizzly/Jetty/Tomcat has been re-used. The new Async features build on
> top of the preliminary work done on those containers. The main features
> is to be able to suspend/resume the response and to be able to dispatch
> a suspended response to another resources. This is a subset of what
> Grizzly Comet supports (but I'm pretty sure a lot of framework will
> build on top of the 3.0 api).
>
>
>> If detecting when the client is not present or able to receive messages
>> anymore it's based on catching an IOEx when writing the response, based
>> on
>> what I've seen using Tomcat, I'm not sure how well this will work. To be
>> really sure I would rather have a client timeout mechanism on the server
>> to
>> detect disconnected clients (check when the last ping event was
>> successfully
>> sent).
>
> Right, this is how you need to do that right now (except with Grizzly,
> who can detect a client close). With other, you need a thread that ping
> the connection to see if it is still open or not.
>
>
>>
>> 2. I guess I was mislead by seeing in the examples the handler being
>> created
>> in doGet and by the doc saying: " ... Resume the Comet request and remove
>> it
>> from the active CometHandler list. Once resumed, a CometHandler must
>> never
>> manipulate the HttpServletRequest or HttpServletResponse as those object
>> will be recycled and may be re-used to serve another request. If you
>> cache
>> them for later reuse by another thread there is a possibility to
>> introduce
>> corrupted responses next time a request is made. ... "
>> So I should be able to cache it in client's session, attach the new
>> Response
>> in the nxt poll and then add it back to the context?
>
> Yes.
>
> I'll have to check the
>> code to understand how it works. I think it would be useful to be able to
>> add the handler to the context using a custom ID. I'm wondering what
>> happens
>> if I add twice the same handler instance.
>
> The Handler will be invoked many times. But point taken...I will improve
> the javadoc on that front.
>
>>
>> 3. I think this is explained in 2 now.
>>
>> 4. I need to understand how async http write would work. But anyway, just
>> to
>> use a brutal approach, do you see a problem in having a pool of 200-300
>> dispatching threads just to be sure?
>
> It really depend on what the application writes/payload. Can you
> elaborate?
>
> Thanks!
>
> -- Jeanfrancois
>
>
>>
>>
>>
>> Jeanfrancois Arcand-2 wrote:
>>> Salut,
>>>
>>> felixx wrote:
>>>> Hi all,
>>>> I'm trying to decide on a Comet server side solution. My first
>>>> preference
>>>> would be to go GF V3 and Grizzly. However based on the existing issues
>>>> list
>>>> (475, 478, 479) I do have some concerns regarding the
>>>> stability/maturity
>>>> of
>>> 475: bad test (just recently added, OS dependent failure)
>>> 478: Javascript problem (and i'm quite bad with javascript)
>>> 479: Maybe an issue with cometd in GF v3.
>>>
>>> So nothing related to Comet (server side) :-). We lacked internal
>>> testing for awhile (situation fixed now :-)). Grizzly Comet is stable
>>> but true some minor release 1.9.x might introduce minor regressions).
>>>
>>> If your looking at an immature but nice Comet Framework, look at
>>> Atmosphere.dev.java.net (shameless plug as I'm releasing 0.1 today :-))
>>>
>>>
>>>> the Comet implementation. Here are some questions please:
>>>> 1.Do you guys know/used the Grizzly Comet in a real production env. I
>>>> know,
>>>> small examples with a few clients would work but how about having 5000
>>>> clients and 2000 messages/s? Has anyone tested/benchmarked such a
>>>> scenario.
>>> That will be the topic of my javaone session this year :-) ...yes we
>>> have done some test with that but it really depends on the application
>>> itself, long poll vs http streaming, etc. We have two automated
>>> benchmark inside the workspace:
>>>
>>> https://grizzly.dev.java.net/nonav/xref-test/com/sun/grizzly/comet/CometUnitTest.html
>>>
>>> which can always be looked at. Of course they aren't real benchmark with
>>> a real application.
>>>
>>>
>>>> It would be unacceptable for us to have the app hanging or being
>>>> unreliable
>>>> (see the above issues) I've seen some references about GF performance
>>>> but
>>>> I'm referring strictly to the Comet side. Not easy to say it but, and I
>>>> may
>>>> be wrong, it looks like Jetty has been used in such scenarios, worked
>>>> fine
>>>> and seems more mature in this area.
>>> Difficult to convince you here as Jetty was the first one on the market,
>>> and at the time GlassFish was just moving from its "zero" to its "hero"
>>> role :-). But the way Jetty handle suspend/resume cannot scale as every
>>> time you suspend, it throws an exception and your Servlet ends up being
>>> invoked twice. I like Jetty, but the way suspend/resume has been
>>> implemented cannot scale IMO and the logic needed inside a Servlet
>>> (double invokation) make the programming model complicated.
>>>
>>> Not only that, but with Jetty you don't have any support detecting when
>>> clients close connection (which can cause memory leak with some
>>> application because of wasted resources), something grizzly supports.
>>>
>>> Also the way information/message are pushed in Jetty has to be
>>> implemented by the application itself, where in Grizzly you can easily
>>> filter/throttle/push/cluster using the CometContext.notify() and
>>> NotificationHandler. This is less work to do for an application's
>>> developer.
>>>
>>>
>>>> 2.Maybe I'm missing something here but is there a way to avoid
>>>> recreating
>>>> a
>>>> CometHandler every time the same client reconnects for a long pool?
>>> You don't have to recreate. Just re-add it to the list of suspended
>>> response (CometContext.addCometHandler())
>>>
>>> I would
>>>> need a ConcurrentHandler type (with the internal event queue) and seems
>>>> expensive to have for each poll cycle thousands of handlers being
>>>> created
>>>> and then dropped when an event has ocurred. Why not just being able to
>>>> replace the embedded Response and keep the existing handler as long as
>>>> the
>>>> client is present?
>>> CometHandler.attach() is for that. But you still need to invoke
>>> resumeCometHandler followed by addCometHandler()
>>>
>>>
>>>> 3.Here's a scenario I've seen using a different 'long polling'
>>>> solution.
>>>> Due
>>>> to a network condition the server is not able to push the events and
>>>> these
>>>> are accumulating in the handler's queue. (not always the server is
>>>> able
>>>> to
>>>> see right away that the client is not able to receive the events). The
>>>> client has a mechanism to detect if ping events are received and if not
>>>> will
>>>> try again to connect. Now you will end having 2 handler for the same
>>>> client,
>>>> the old one with some events in the queue, also blocking a thread, and
>>>> the
>>>> fresh one. How can we handle this type of situations in? In a different
>>>> system I used just to detect the situation, replace the Response and
>>>> start
>>>> pushing the content of the queue.
>>> why do you think creating CometHandler are expensive? Event if they are
>>> (based on your application), you can alway use the CometHandler.attach()
>>> to recycle them. Did I misunderstood you?
>>>
>>>
>>>> 4.Even with a ConcurrentHandler you may end having a lot of threads
>>>> blocked
>>>> because of slow clients. It looks like if you expect 5000 clients it's
>>>> safer
>>>> to use let's say 400 threads to execute the handler. Any comments on
>>>> this?
>>> It's up to your application to either:
>>>
>>> + enable async http write in Grizzly (so no blocking thread) insid ethe
>>> application (grizzly internal will handle the "blocking" for you
>>> instead).
>>> + define a "strategy" inside NotificationHandler to detect such slow
>>> client and either discard them or park them inside a "queue" for later
>>> write.
>>>
>>> Note that the same issue will arise with Jetty :-)
>>>
>>>
>>>> A lot of questions, I know. Again GF & Grizzly looks very promising,
>>>> great
>>>> work so far, I'm not just sure, and I need your help to convince
>>>> myself,
>>>> it
>>>> is ready for a real life application.
>>> You are welcome. We have multiples implementation inside and outside Sun
>>> that build on top of Comet. One of them is our internal Sun IM
>>> product...so there is a possibility of ~30 000 users at the same time
>>> using it :-)
>>>
>>> Feel free to continue the discussion if I was unclear of if I missed the
>>> point.
>>>
>>> A+
>>>
>>> -- Jeanfrancois
>>>
>>>
>>>
>>>
>>>> Thanks!
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>
>
>

-- 
View this message in context: http://www.nabble.com/Grizzly-production-ready--ConcurrentHandler---scenarios-tp22501469p22558511.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.