Re: Urgent - EJB SSB Timeouts!
>>But you are using an EJB for it anyway, right?
No, we're doing the generation in the timer task on the web tier (I'm using java.util.Timer, spawned by a context listener on startup of the web container)...I didn't bother using an enterprise scheduler as that was a bit of overkill for a simple 20 minute recursive timer thread). It has to be done on the web tier as it generates html files on the local system which get included into a JSP using c:import (we have to generate them as they're waaaaaaay to big and complex to generate dynamically).
The EJB call is just to see if reports need to be generated. It checks some JMS queues and looks up some db tables to see if there is a new bunch of batching data that is ready to be processed and returns either a true or a false...then the calling web tier timer task either does the generation (using more ejb calls to get the data -- it never gets this far) or sleeps for another 20 minutes.
This morning I turned on monitoring on all the tiers just to make sure it wasn't a memory issue -- heaps of free memory, so that's not it.
I did find the following warning in the ejb tier:
[#|2009-06-24T07:35:47.578+1000|WARNING|sun-appserver2.1|javax.jms|_ThreadID=22;_ThreadName=iMQReadChannel-36;_RequestID=5f3d6e80-3240-442e-9e85-ac7c26ddf53f;|[I500]: Caught JVM Exception: java.io.EOFException|#]
The EJB is referencing a queue
@Resource(mappedName="jms/QueueConnectionFactory")
private QueueConnectionFactory queueConnectionFactory;
so I am investigating the theory that the SSB EJBs are being created too fast for the message queue (LOCAL JMS) when the ejb pool resizes...I notice that just before the error occurs (10 minutes or so before) the pool creates a bunch more EJBs and it's the next call that fails. So I've turned down all my values (creating less EJBs on pool resize) to see if that makes any difference. I'll have an answer on that in about 20 minutes after I finishing deploying a new version of the ear and war (with more logging)
Other than that I'm completely stumped :(
--- On Wed, 24/6/09, Marina Vatkina <Marina.Vatkina@Sun.COM> wrote:
> From: Marina Vatkina <Marina.Vatkina@Sun.COM>
> Subject: Re: Urgent - EJB SSB Timeouts!
> To: users@glassfish.dev.java.net
> Received: Wednesday, 24 June, 2009, 9:00 AM
> Adam Jenkins wrote:
> > Yeah, you're correct....it's only one timer, calling
> one SSB method every 20 minutes.
> >
> > I'm using the default number of connections,
> 1024. My thread pool has 50 minimum, initial ejb pool
> size of 5 (max 500, resize 20). So there should be
> ample threads/ejbs
> >
> > No idea why it didn't show up in testing...we recently
> went from v2ur2 up to v2.1...and to be honest, the test
> system never really ran for over an hour at a time :)
> >
> > We use EJB Timer service for all our other timing
> needs and that works fine, but this process specifically
> generates files into the web directory that then are
> referenced from other jsps using c:import...so this is the
> only timer task that has to run on the web tier.
>
> But you are using an EJB for it anyway, right? If yes, why
> can't the EJB Timer
> service do the same job?
>
> If it can't, which JDK api are you using? EJB Timer Service
> reschedules the JDK
> timer task for each call (vs. scheduling the task with
> multiple expirations).
>
> Regards,
> -marina
> >
> > --- On Tue, 23/6/09, glassfish@javadesktop.org
> <glassfish@javadesktop.org>
> wrote:
> >
> >
> >>From: glassfish@javadesktop.org
> <glassfish@javadesktop.org>
> >>Subject: Re: Urgent - EJB SSB Timeouts!
> >>To: users@glassfish.dev.java.net
> >>Received: Tuesday, 23 June, 2009, 7:23 AM
> >>Hi,
> >>
> >>OK, so you say you call the EJB every 20 minutes,
> that
> >>means 3 times in an hour. Do you call only one EJB
> every 20
> >>minutes or several EJBs every 20 minutes?
> >>I am actually just guessing but I think that the
> problem is
> >>not that the name of your EJB is not found but
> maybe rather
> >>that some limit on the naming context lookup object
> was
> >>reached or that maybe there were not any pooled
> EJBs free to
> >>service your request.
> >>Looking at a default domain.xml config of a
> Glassfish I can
> >>see that the max-connections to the ORB are 1024.
> Sounds
> >>high. ;)
> >>Maybe you can give more information how many EJBs
> are
> >>called how often. Can you imagine why this
> behaviour did not
> >>show on the test system?
> >>
> >>Cheers
> >>Chris.
> >>[Message sent by forum member 'chrjohn' (chrjohn)]
> >>
> >>http://forums.java.net/jive/thread.jspa?messageID=352409
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> >>For additional commands, e-mail: users-help@glassfish.dev.java.net
> >>
> >>
> >
> >
> >
> > Access Yahoo!7 Mail on
> your mobile. Anytime. Anywhere.
> > Show me how: http://au.mobile.yahoo.com/mail
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> > For additional commands, e-mail: users-help@glassfish.dev.java.net
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>
Access Yahoo!7 Mail on your mobile. Anytime. Anywhere.
Show me how: http://au.mobile.yahoo.com/mail