Hi Jeanfrancois,
The problem is with UR1 and it has not yet been resolved. After about an
hour, any Comet data sent from the server appears to go nowhere as it is not
received at the applet even though there are no exceptions at either end.
My analysis has revealed that the problem only occurs over the actual
internet (not the LAN) and appears to be related to the server sending
packets to the wrong port.
Thanks,
John
> -----Original Message-----
> From: Jeanfrancois.Arcand_at_Sun.COM [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> Sent: Thursday, 31 January 2008 03:24
> To: users_at_grizzly.dev.java.net
> Subject: Re: Comet data into the bit bucket
>
> Hi John,
>
> John C. Turnbull wrote:
> > Hi Jeanfrancois,
> >
> > I would like to test the change you have made but I am not sure how
> to do
> > it. I have been using GlassFish v2 UR1 and not the standalone
> Grizzly up to
> > this point so I guess I need to try it with the latter now. Is that
> > correct?
>
> Pouua sorry I was completely confused when I asked that :-) Just to
> make
> sure I follow, what was the time out issue you are seeing? Was it with
> v2 or v2 ur1? Is the problem fixed now?
>
> Thanks
>
> -- Jeanfrancois
>
> Does that mean I need to use the Grizzly Web Server instead of
> > GlassFish?
> >
> > I have not used standalone Grizzly yet so I am unsure how to proceed
> and
> > deploy etc.
> >
> > Thanks,
> >
> > John
> >
> >> -----Original Message-----
> >> From: Jeanfrancois.Arcand_at_Sun.COM
> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> >> Sent: Wednesday, 30 January 2008 07:21
> >> To: users_at_grizzly.dev.java.net
> >> Subject: Re: Comet data into the bit bucket
> >>
> >> Hi John,
> >>
> >> John C. Turnbull wrote:
> >>> Hi Jeanfrancois,
> >>>
> >>> OK, firstly I just start GlassFish by using
> >>>
> >>> asadmin start-domain domain1
> >>>
> >>> Is that correct?
> >> Yes.
> >>
> >>> I am about to rewrite parts of my application so I will wait and
> see
> >> if the
> >>> problem persists after I have the new version working before
> posting
> >> code.
> >>> At the moment, I am much more concerned about the AV and firewall
> >> issues I
> >>> refer to in my other post. Do you have any comments on that issue?
> >> I need to read a little but I know there is issue with streaming and
> >> firewall. Will come back on that one.
> >>
> >> As for your timeout problem in Grizzly standalone, I think I have
> found
> >> the problem (will commit the fix now). Give it a try and let me know
> if
> >> you have a chance.
> >>
> >> Thanks!
> >>
> >> -- Jeanfrancois
> >>
> >>
> >>> Thanks,
> >>>
> >>> John
> >>>
> >>>> -----Original Message-----
> >>>> From: Jeanfrancois.Arcand_at_Sun.COM
> >> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> >>>> Sent: Tuesday, 29 January 2008 02:15
> >>>> To: users_at_grizzly.dev.java.net
> >>>> Subject: Re: Comet data into the bit bucket
> >>>>
> >>>> Hi John,
> >>>>
> >>>> John C. Turnbull wrote:
> >>>>> Hi Jeanfrancois,
> >>>>>
> >>>>> I will perform a more detailed analysis of the packet transfers
> but
> >>>> one
> >>>>> thing that I do now know is that the problem does not occur when
> I
> >>>> run
> >>>>> GlassFish on my local LAN and access it from another machine on
> the
> >>>> LAN. I
> >>>>> have managed to keep the connection viable for 3 days so far
> >> without
> >>>> any
> >>>>> activity so it seems that the problem only occurs when I run the
> >>>> server over
> >>>>> the actual internet. I am not sure what this suggests/indicates.
> >>>> Great analysis :-) That means the problem is with Grizzly
> >> only....Can
> >>>> you refresh my mind and explain how you start Grizzly, and what
> your
> >>>> test is doing? Just a couple of lines to see if I can spot
> something
> >>>> inside Grizzly code.
> >>>>
> >>>> Thanks!
> >>>>
> >>>> -- Jeanfrancois
> >>>>
> >>>>
> >>>>> Thanks,
> >>>>>
> >>>>> John
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Jeanfrancois.Arcand_at_Sun.COM
> >>>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> >>>>>> Sent: Thursday, 24 January 2008 12:33
> >>>>>> To: users_at_grizzly.dev.java.net
> >>>>>> Subject: Re: Comet data into the bit bucket
> >>>>>>
> >>>>>> Hi John,
> >>>>>>
> >>>>>> quite good analysis...
> >>>>>>
> >>>>>> John C. Turnbull wrote:
> >>>>>>> Hi Jeanfrancois,
> >>>>>>>
> >>>>>>> OK, I have a lot of new information that hopefully will allow
> us
> >> to
> >>>>>> finally
> >>>>>>> get to the bottom of this problem.
> >>>>>>>
> >>>>>>> After running packet sniffers on both the server and client and
> >>>> also
> >>>>>> by
> >>>>>>> running the netstat command, I have determined the following
> >>>> things:
> >>>>>>> (1) The exact period of idle time seems to be much more than 10
> >>>>>> minutes and
> >>>>>>> more like an hour although I cannot be certain if this is fixed
> >> or
> >>>>>>> pseudo-random.
> >>>>>>> (2) The server is definitely sending data to the IP address of
> >> the
> >>>>>> client
> >>>>>>> machine before and after the idle time.
> >>>>>>> (3) The client machine is receiving data from the server's IP
> >>>> address
> >>>>>> before
> >>>>>>> and after the idle time, even though it is not actually getting
> >>>> into
> >>>>>> the
> >>>>>>> applet.
> >>>>>>> (4) The server is using the same DataOutputStream to write the
> >> data
> >>>>>> before
> >>>>>>> and after the idle time.
> >>>>>>> (5) Before the idle time, the connections on the client look
> like
> >>>>>> this:
> >>>>>>> TCP 10.1.1.3:63457 1.2.3.4:ssh ESTABLISHED
> >>>> InHost
> >>>>>>> TCP 10.1.1.3:63940 1.2.3.4:8080 ESTABLISHED
> >>>> InHost
> >>>>>>> Where 1.2.3.4 is the mock address of the server machine. That
> >>>> looks
> >>>>>> fine
> >>>>>>> but I don't know why there's an SSH connection in there
> (perhaps
> >>>> you
> >>>>>> could
> >>>>>>> explain that).
> >>>>>> No idea for ssh...
> >>>>>>
> >>>>>> Anyway, note that the port in the connection to the server's
> >>>>>>> 8080 port is 63940.
> >>>>>>>
> >>>>>>> (6) Immediately after the idle time, the connections on the
> >> client
> >>>>>> look like
> >>>>>>> this:
> >>>>>>>
> >>>>>>> TCP 10.1.1.3:63940 1.2.3.4:8080 ESTABLISHED
> >>>> InHost
> >>>>>>> TCP 10.1.1.3:64892 1.2.3.4:8080 FIN_WAIT_1
> >>>> InHost
> >>>>>>> TCP 10.1.1.3:64898 1.2.3.4:8080 FIN_WAIT_1
> >>>> InHost
> >>>>>>> Note that we still have the original connection but now we have
> 2
> >>>> new
> >>>>>>> connections in FIN_WAIT_1 state (whatever that means) on
> >> different
> >>>>>> ports.
> >>>>>>
> >>>>>> That means two connections have been closed, and the file
> >> descriptor
> >>>>>> (socket) will eventually disappear.
> >>>>>>
> >>>>>>> (7) Wireshark reveals that the packets of data arriving on the
> >>>> client
> >>>>>> after
> >>>>>>> the idle time are directed to port 63494 which is not the port
> of
> >>>> any
> >>>>>> of the
> >>>>>>> above connections. This must be why the applet is not
> receiving
> >>>> them
> >>>>>> as it
> >>>>>>> is still established on a connection on port 63490.
> >>>>>> But that's strange, because NIO (not Grizzly) cannot change the
> >>>> remote
> >>>>>> port of SocketChannel. That means the connection has probably
> been
> >>>>>> closed on the client side and reopened (with a new port). What
> is
> >>>>>> puzzling me is Grizzly will wakes when the connection on the
> >> client
> >>>>>> side
> >>>>>> is closed, and call onTerminate().
> >>>>>>
> >>>>>>
> >>>>>>> So there you have it. Everything is functioning correctly
> except
> >>>>>> that the
> >>>>>>> data after the idle time appears to be sent to the wrong port.
> I
> >>>>>> can't find
> >>>>>>> any other anomaly.
> >>>>>> Looks like there is a timeout, maybe from the Applet or the
> >> browser,
> >>>>>> and
> >>>>>> the connection gets re-openned, hit your Servlet again and comet
> >>>> gets
> >>>>>> re-enabled on that connection (because you are seeing some
> push).
> >>>> Can
> >>>>>> you grab what is exchanged between the client and the server? I
> >>>> think
> >>>>>> that's the only way we can find something.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> -- Jeanfrancois
> >>>>>>
> >>>>>>
> >>>>>>> Any ideas why the destination port is changing after the idle
> >> time?
> >>>>>> Can you
> >>>>>>> spot anything else?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> John
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Jeanfrancois.Arcand_at_Sun.COM
> >>>>>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> >>>>>>>> Sent: Wednesday, 23 January 2008 05:17
> >>>>>>>> To: users_at_grizzly.dev.java.net
> >>>>>>>> Subject: Re: Comet data into the bit bucket
> >>>>>>>>
> >>>>>>>> Hi John,
> >>>>>>>>
> >>>>>>>> John C. Turnbull wrote:
> >>>>>>>>> Hi Jeanfrancois,
> >>>>>>>>>
> >>>>>>>>> I have discovered that when the problem occurs, there is no
> >>>>>> interrupt
> >>>>>>>>> detected so it would appear that the connection is still open
> >>>>>> (which
> >>>>>>>>> correlates with the fact that there are no EOF exceptions at
> >>>> either
> >>>>>>>> end).
> >>>>>>>>
> >>>>>>>> Just in case, are you inside a firewall or a proxy? Can it be
> >> the
> >>>>>> proxy
> >>>>>>>> that close the connection (I suspect not).
> >>>>>>>>
> >>>>>>>>> Hmm, so where do I go from here? The connection remains open
> >> but
> >>>>>> no
> >>>>>>>> data is
> >>>>>>>>> getting through.
> >>>>>>>> From your CometHandler, invoking the
> >>>>>>>> HttpServletResponse.getWriter().write(...) succeed, right? I
> >> guess
> >>>>>> you
> >>>>>>>> should install wireshark or ngrep to see if the bytes are sent
> >>>>>>>> properly.
> >>>>>>>> This way we will know if the client or server has problems.
> >>>>>>>>
> >>>>>>>> Thanks
> >>>>>>>>
> >>>>>>>> -- Jeanfrancois
> >>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>
> >>>>>>>>> John
> >>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Jeanfrancois.Arcand_at_Sun.COM
> >>>>>>>> [mailto:Jeanfrancois.Arcand_at_Sun.COM]
> >>>>>>>>>> Sent: Wednesday, 23 January 2008 03:37
> >>>>>>>>>> To: users_at_grizzly.dev.java.net
> >>>>>>>>>> Subject: Re: Comet data into the bit bucket
> >>>>>>>>>>
> >>>>>>>>>> Hi John,
> >>>>>>>>>>
> >>>>>>>>>> John C. Turnbull wrote:
> >>>>>>>>>>> Hi Jeanfrancois,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for your reply. Comments inline.
> >>>>>>>>>>>
> >>>>>>>>>>>>> I have Comet working nicely between servlets and applets
> >>>> except
> >>>>>>>> for
> >>>>>>>>>>>> one
> >>>>>>>>>>>>> thing.
> >>>>>>>>>>>> Great!
> >>>>>>>>>>> [JCT] Yes, indeed. I am very happy with Grizzly Comet :-)
> >>>>>>>>>>>
> >>>>>>>>>>>> For some reason, if a connection is left idle for
> several
> >>>>>>>>>>>>> minutes then the data sent from the servlet seems to
> >>>> disappear
> >>>>>>>> into
> >>>>>>>>>>>> the
> >>>>>>>>>>>>> ether somewhere.
> >>>>>>>>>>>> Do you know how long exactly?
> >>>>>>>>>>> [JCT] Not exactly but it's about 10 minutes.
> >>>>>>>>>>>
> >>>>>>>>>>>> Do you have a test case I can take a look at? Do you know
> if
> >>>> the
> >>>>>>>>>>>> connection is still active (are you handling the
> >>>>>>>>>>>> onInterrupt/onTerminate
> >>>>>>>>>>>> method of the CometHandler)?
> >>>>>>>>>>> [JCT] It is difficult to provide a test case at this stage
> >> but
> >>>> I
> >>>>>>>> can
> >>>>>>>>>> say
> >>>>>>>>>>> that I am not doing anything with onInterrupt or
> onTerminate
> >>>> yet
> >>>>>> as
> >>>>>>>> I
> >>>>>>>>>> am not
> >>>>>>>>>>> sure how these methods should be used. Is there
> >> documentation
> >>>>>> for
> >>>>>>>>>> Comet
> >>>>>>>>>>> somewhere? I mean something that gives a general outline
> of
> >>>> how
> >>>>>>>>>> everything
> >>>>>>>>>>> hangs together? It's a bit hit-or-miss for me at the
> moment
> >>>> and
> >>>>>> I
> >>>>>>>> am
> >>>>>>>>>>> finding things out by experimentation.
> >>>>>>>>>> That's the general problem with Grizzly...lack of
> >>>> documentations.
> >>>>>>>> The
> >>>>>>>>>> only place I talk about it is here:
> >>>>>>>>>>
> >>>>>>>>>>
> >>
> http://weblogs.java.net/blog/jfarcand/archive/2006/07/the_grizzly_com.h
> >>>>>>>>>> tml
> >>>>>>>>>>
> >>
> https://grizzly.dev.java.net/nonav/xref/com/sun/grizzly/comet/CometHand
> >>>>>>>>>> ler.html
> >>>>>>>>>>
> >>>>>>>>>> which is far from perfect. Mainly, the onInterrupt will be
> >>>> called
> >>>>>> by
> >>>>>>>>>> Grizzly when the client close the connection or when the
> >> timeout
> >>>>>>>>>> expires. The onTerminate will be called when you invoke
> >>>>>>>>>> CometContext.notify(CometEvent.TERMINATE);
> >>>>>>>>>>
> >>>>>>>>>>>>> Any ideas what's actually happening here and where the
> data
> >>>> is
> >>>>>>>>>> going?
> >>>>>>>>>>>> Difficult to say. Are they any exceptions in the log (I
> >> guess
> >>>>>>>> not).
> >>>>>>>>>> Do
> >>>>>>>>>>>> you know if the bytes are sent over the network? A tool
> like
> >>>>>>>>>> wireshark
> >>>>>>>>>>>> or ngrep could help determining if the server or the
> client
> >>>> have
> >>>>>>>>>>>> problem.
> >>>>>>>>>>> [JCT] I haven't used a packet sniffer to analyse the
> network
> >>>> yet
> >>>>>>>> but
> >>>>>>>>>> I do
> >>>>>>>>>>> know that the server "believes" that the data has been
> >>>>>> transmitted
> >>>>>>>> as
> >>>>>>>>>> the
> >>>>>>>>>>> DataOutputStream#size() method is reporting an ever-
> >> increasing
> >>>>>>>> number
> >>>>>>>>>> of
> >>>>>>>>>>> transmitted bytes as the data is written and there are no
> >>>>>>>> exceptions.
> >>>>>>>>>>> [JCT] So perhaps the problem has something to do with the
> >> fact
> >>>>>> that
> >>>>>>>> I
> >>>>>>>>>> am not
> >>>>>>>>>>> properly handling the interrupt or terminate events in the
> >>>>>>>>>> CometHandler but
> >>>>>>>>>>> I still don't know where the data is ending up. When I get
> a
> >>>>>>>> chance
> >>>>>>>>>> I will
> >>>>>>>>>>> do some sniffing around for packets and try to see exactly
> >> what
> >>>>>> is
> >>>>>>>>>> going on.
> >>>>>>>>>>> [JCT] But, as I said, this is the only outstanding issue I
> >> have
> >>>>>>>> with
> >>>>>>>>>> my
> >>>>>>>>>>> usage of Comet.
> >>>>>>>>>> OK let's work on that :-) First thing to try is to output
> some
> >>>>>> trace
> >>>>>>>>>> inside the onInterrupt to see if that method gets invoked.
> >>>>>>>>>>
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>> -- Jeanfrancois
> >>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>
> >>>>>>>>>>> -JCT
> >>>>>>>>>>>
> >>>>>>>>>>> -----------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>> --
> >>>>>>>> --
> >>>>>>>>>>> To unsubscribe, e-mail: users-
> >> unsubscribe_at_grizzly.dev.java.net
> >>>>>>>>>>> For additional commands, e-mail: users-
> >>>> help_at_grizzly.dev.java.net
> >>>>>>>>>> ------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>> --
> >>>>>>>> -
> >>>>>>>>>> To unsubscribe, e-mail: users-
> unsubscribe_at_grizzly.dev.java.net
> >>>>>>>>>> For additional commands, e-mail: users-
> >> help_at_grizzly.dev.java.net
> >>>>>>>>> -------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>> --
> >>>>>>>>> To unsubscribe, e-mail: users-
> unsubscribe_at_grizzly.dev.java.net
> >>>>>>>>> For additional commands, e-mail: users-
> >> help_at_grizzly.dev.java.net
> >>>>>>>> --------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>> -
> >>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>>>>>>> For additional commands, e-mail: users-
> help_at_grizzly.dev.java.net
> >>>>>>> ---------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>>>>>> For additional commands, e-mail: users-
> help_at_grizzly.dev.java.net
> >>>>>>>
> >>>>>> ----------------------------------------------------------------
> --
> >> --
> >>>> -
> >>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >>>>> -----------------------------------------------------------------
> --
> >> --
> >>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >>>>>
> >>>> ------------------------------------------------------------------
> --
> >> -
> >>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >>> -------------------------------------------------------------------
> --
> >>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >>>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> >> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> > For additional commands, e-mail: users-help_at_grizzly.dev.java.net
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net