Hi John,
John C. Turnbull wrote:
> Hi Jeanfrancois,
>
> OK, firstly I just start GlassFish by using
>
> asadmin start-domain domain1
>
> Is that correct?
Yes.
>
> I am about to rewrite parts of my application so I will wait and see if the
> problem persists after I have the new version working before posting code.
> At the moment, I am much more concerned about the AV and firewall issues I
> refer to in my other post. Do you have any comments on that issue?
I need to read a little but I know there is issue with streaming and
firewall. Will come back on that one.
As for your timeout problem in Grizzly standalone, I think I have found
the problem (will commit the fix now). Give it a try and let me know if
you have a chance.
Thanks!
-- Jeanfrancois
>
> Thanks,
>
> John
>
>> -----Original Message-----
>> From: Jeanfrancois.Arcand_at_Sun.COM [mailto:Jeanfrancois.Arcand_at_Sun.COM]
>> Sent: Tuesday, 29 January 2008 02:15
>> To: users_at_grizzly.dev.java.net
>> Subject: Re: Comet data into the bit bucket
>>
>> Hi John,
>>
>> John C. Turnbull wrote:
>>> Hi Jeanfrancois,
>>>
>>> I will perform a more detailed analysis of the packet transfers but
>> one
>>> thing that I do now know is that the problem does not occur when I
>> run
>>> GlassFish on my local LAN and access it from another machine on the
>> LAN. I
>>> have managed to keep the connection viable for 3 days so far without
>> any
>>> activity so it seems that the problem only occurs when I run the
>> server over
>>> the actual internet. I am not sure what this suggests/indicates.
>> Great analysis :-) That means the problem is with Grizzly only....Can
>> you refresh my mind and explain how you start Grizzly, and what your
>> test is doing? Just a couple of lines to see if I can spot something
>> inside Grizzly code.
>>
>> Thanks!
>>
>> -- Jeanfrancois
>>
>>
>>> Thanks,
>>>
>>> John
>>>
>>>> -----Original Message-----
>>>> From: Jeanfrancois.Arcand_at_Sun.COM
>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
>>>> Sent: Thursday, 24 January 2008 12:33
>>>> To: users_at_grizzly.dev.java.net
>>>> Subject: Re: Comet data into the bit bucket
>>>>
>>>> Hi John,
>>>>
>>>> quite good analysis...
>>>>
>>>> John C. Turnbull wrote:
>>>>> Hi Jeanfrancois,
>>>>>
>>>>> OK, I have a lot of new information that hopefully will allow us to
>>>> finally
>>>>> get to the bottom of this problem.
>>>>>
>>>>> After running packet sniffers on both the server and client and
>> also
>>>> by
>>>>> running the netstat command, I have determined the following
>> things:
>>>>> (1) The exact period of idle time seems to be much more than 10
>>>> minutes and
>>>>> more like an hour although I cannot be certain if this is fixed or
>>>>> pseudo-random.
>>>>> (2) The server is definitely sending data to the IP address of the
>>>> client
>>>>> machine before and after the idle time.
>>>>> (3) The client machine is receiving data from the server's IP
>> address
>>>> before
>>>>> and after the idle time, even though it is not actually getting
>> into
>>>> the
>>>>> applet.
>>>>> (4) The server is using the same DataOutputStream to write the data
>>>> before
>>>>> and after the idle time.
>>>>> (5) Before the idle time, the connections on the client look like
>>>> this:
>>>>> TCP 10.1.1.3:63457 1.2.3.4:ssh ESTABLISHED
>> InHost
>>>>> TCP 10.1.1.3:63940 1.2.3.4:8080 ESTABLISHED
>> InHost
>>>>> Where 1.2.3.4 is the mock address of the server machine. That
>> looks
>>>> fine
>>>>> but I don't know why there's an SSH connection in there (perhaps
>> you
>>>> could
>>>>> explain that).
>>>> No idea for ssh...
>>>>
>>>> Anyway, note that the port in the connection to the server's
>>>>> 8080 port is 63940.
>>>>>
>>>>> (6) Immediately after the idle time, the connections on the client
>>>> look like
>>>>> this:
>>>>>
>>>>> TCP 10.1.1.3:63940 1.2.3.4:8080 ESTABLISHED
>> InHost
>>>>> TCP 10.1.1.3:64892 1.2.3.4:8080 FIN_WAIT_1
>> InHost
>>>>> TCP 10.1.1.3:64898 1.2.3.4:8080 FIN_WAIT_1
>> InHost
>>>>> Note that we still have the original connection but now we have 2
>> new
>>>>> connections in FIN_WAIT_1 state (whatever that means) on different
>>>> ports.
>>>>
>>>> That means two connections have been closed, and the file descriptor
>>>> (socket) will eventually disappear.
>>>>
>>>>> (7) Wireshark reveals that the packets of data arriving on the
>> client
>>>> after
>>>>> the idle time are directed to port 63494 which is not the port of
>> any
>>>> of the
>>>>> above connections. This must be why the applet is not receiving
>> them
>>>> as it
>>>>> is still established on a connection on port 63490.
>>>> But that's strange, because NIO (not Grizzly) cannot change the
>> remote
>>>> port of SocketChannel. That means the connection has probably been
>>>> closed on the client side and reopened (with a new port). What is
>>>> puzzling me is Grizzly will wakes when the connection on the client
>>>> side
>>>> is closed, and call onTerminate().
>>>>
>>>>
>>>>> So there you have it. Everything is functioning correctly except
>>>> that the
>>>>> data after the idle time appears to be sent to the wrong port. I
>>>> can't find
>>>>> any other anomaly.
>>>> Looks like there is a timeout, maybe from the Applet or the browser,
>>>> and
>>>> the connection gets re-openned, hit your Servlet again and comet
>> gets
>>>> re-enabled on that connection (because you are seeing some push).
>> Can
>>>> you grab what is exchanged between the client and the server? I
>> think
>>>> that's the only way we can find something.
>>>>
>>>> Thanks!
>>>>
>>>> -- Jeanfrancois
>>>>
>>>>
>>>>> Any ideas why the destination port is changing after the idle time?
>>>> Can you
>>>>> spot anything else?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> John
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jeanfrancois.Arcand_at_Sun.COM
>>>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
>>>>>> Sent: Wednesday, 23 January 2008 05:17
>>>>>> To: users_at_grizzly.dev.java.net
>>>>>> Subject: Re: Comet data into the bit bucket
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> John C. Turnbull wrote:
>>>>>>> Hi Jeanfrancois,
>>>>>>>
>>>>>>> I have discovered that when the problem occurs, there is no
>>>> interrupt
>>>>>>> detected so it would appear that the connection is still open
>>>> (which
>>>>>>> correlates with the fact that there are no EOF exceptions at
>> either
>>>>>> end).
>>>>>>
>>>>>> Just in case, are you inside a firewall or a proxy? Can it be the
>>>> proxy
>>>>>> that close the connection (I suspect not).
>>>>>>
>>>>>>> Hmm, so where do I go from here? The connection remains open but
>>>> no
>>>>>> data is
>>>>>>> getting through.
>>>>>> From your CometHandler, invoking the
>>>>>> HttpServletResponse.getWriter().write(...) succeed, right? I guess
>>>> you
>>>>>> should install wireshark or ngrep to see if the bytes are sent
>>>>>> properly.
>>>>>> This way we will know if the client or server has problems.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> -- Jeanfrancois
>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Jeanfrancois.Arcand_at_Sun.COM
>>>>>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
>>>>>>>> Sent: Wednesday, 23 January 2008 03:37
>>>>>>>> To: users_at_grizzly.dev.java.net
>>>>>>>> Subject: Re: Comet data into the bit bucket
>>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> John C. Turnbull wrote:
>>>>>>>>> Hi Jeanfrancois,
>>>>>>>>>
>>>>>>>>> Thanks for your reply. Comments inline.
>>>>>>>>>
>>>>>>>>>>> I have Comet working nicely between servlets and applets
>> except
>>>>>> for
>>>>>>>>>> one
>>>>>>>>>>> thing.
>>>>>>>>>> Great!
>>>>>>>>> [JCT] Yes, indeed. I am very happy with Grizzly Comet :-)
>>>>>>>>>
>>>>>>>>>> For some reason, if a connection is left idle for several
>>>>>>>>>>> minutes then the data sent from the servlet seems to
>> disappear
>>>>>> into
>>>>>>>>>> the
>>>>>>>>>>> ether somewhere.
>>>>>>>>>> Do you know how long exactly?
>>>>>>>>> [JCT] Not exactly but it's about 10 minutes.
>>>>>>>>>
>>>>>>>>>> Do you have a test case I can take a look at? Do you know if
>> the
>>>>>>>>>> connection is still active (are you handling the
>>>>>>>>>> onInterrupt/onTerminate
>>>>>>>>>> method of the CometHandler)?
>>>>>>>>> [JCT] It is difficult to provide a test case at this stage but
>> I
>>>>>> can
>>>>>>>> say
>>>>>>>>> that I am not doing anything with onInterrupt or onTerminate
>> yet
>>>> as
>>>>>> I
>>>>>>>> am not
>>>>>>>>> sure how these methods should be used. Is there documentation
>>>> for
>>>>>>>> Comet
>>>>>>>>> somewhere? I mean something that gives a general outline of
>> how
>>>>>>>> everything
>>>>>>>>> hangs together? It's a bit hit-or-miss for me at the moment
>> and
>>>> I
>>>>>> am
>>>>>>>>> finding things out by experimentation.
>>>>>>>> That's the general problem with Grizzly...lack of
>> documentations.
>>>>>> The
>>>>>>>> only place I talk about it is here:
>>>>>>>>
>>>>>>>>
>> http://weblogs.java.net/blog/jfarcand/archive/2006/07/the_grizzly_com.h
>>>>>>>> tml
>>>>>>>>
>> https://grizzly.dev.java.net/nonav/xref/com/sun/grizzly/comet/CometHand
>>>>>>>> ler.html
>>>>>>>>
>>>>>>>> which is far from perfect. Mainly, the onInterrupt will be
>> called
>>>> by
>>>>>>>> Grizzly when the client close the connection or when the timeout
>>>>>>>> expires. The onTerminate will be called when you invoke
>>>>>>>> CometContext.notify(CometEvent.TERMINATE);
>>>>>>>>
>>>>>>>>>>> Any ideas what's actually happening here and where the data
>> is
>>>>>>>> going?
>>>>>>>>>> Difficult to say. Are they any exceptions in the log (I guess
>>>>>> not).
>>>>>>>> Do
>>>>>>>>>> you know if the bytes are sent over the network? A tool like
>>>>>>>> wireshark
>>>>>>>>>> or ngrep could help determining if the server or the client
>> have
>>>>>>>>>> problem.
>>>>>>>>> [JCT] I haven't used a packet sniffer to analyse the network
>> yet
>>>>>> but
>>>>>>>> I do
>>>>>>>>> know that the server "believes" that the data has been
>>>> transmitted
>>>>>> as
>>>>>>>> the
>>>>>>>>> DataOutputStream#size() method is reporting an ever-increasing
>>>>>> number
>>>>>>>> of
>>>>>>>>> transmitted bytes as the data is written and there are no
>>>>>> exceptions.
>>>>>>>>> [JCT] So perhaps the problem has something to do with the fact
>>>> that
>>>>>> I
>>>>>>>> am not
>>>>>>>>> properly handling the interrupt or terminate events in the
>>>>>>>> CometHandler but
>>>>>>>>> I still don't know where the data is ending up. When I get a
>>>>>> chance
>>>>>>>> I will
>>>>>>>>> do some sniffing around for packets and try to see exactly what
>>>> is
>>>>>>>> going on.
>>>>>>>>> [JCT] But, as I said, this is the only outstanding issue I have
>>>>>> with
>>>>>>>> my
>>>>>>>>> usage of Comet.
>>>>>>>> OK let's work on that :-) First thing to try is to output some
>>>> trace
>>>>>>>> inside the onInterrupt to see if that method gets invoked.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> -- Jeanfrancois
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> -JCT
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------
>> --
>>>> --
>>>>>> --
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>>>>> For additional commands, e-mail: users-
>> help_at_grizzly.dev.java.net
>>>>>>>> ----------------------------------------------------------------
>> --
>>>> --
>>>>>> -
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>>> -----------------------------------------------------------------
>> --
>>>> --
>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>>>
>>>>>> ------------------------------------------------------------------
>> --
>>>> -
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>> -------------------------------------------------------------------
>> --
>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>
>>>> --------------------------------------------------------------------
>> -
>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>