users@glassfish.java.net

RE: RE: RE: Re: Interrupted idle thread

From: Martin, Ray <armart3_at_tycho.ncsc.mil>
Date: Wed, 15 Sep 2010 16:16:10 -0400

The test is very repeatable.

I have two cases running right now - one on unpatched 3.0.1 and the
other on patched 3.0.1.

Both cases show that a web service that runs over 15 minutes, glassfish
will wizz all over the parade.

Both cases, right on 15 minutes every time, rip out and do very bad
things. Who ever heard of starting a second process because something
went beyond 15 minutes. If one had a very sensitive process going -
like calculating and writing to a database - oop, yur database is all
screwed up - how nice.

Patched 3.0.1 blows out a WebServiceException and starts a second
instance of the operation. This is the worst that could possibly occur.

Unpatched 3.0.1 only warns of "interrupted idle thread:...", starts
second instance of the operation, but quits the second instance
immediately. On an asynchronous operation, the client believes the
process has completed - this is certainly not a good thing, but at least
it did not corrupt my database.



-----Original Message-----
From: Martin, Ray
Sent: Wednesday, September 15, 2010 2:54 PM
To: users_at_glassfish.dev.java.net
Subject: RE: RE: Re: Interrupted idle thread

I ran the same test on a unpatched version of 3.0.1.
It also ran very badly - but, very differently. It kicked the process
the second time, but immediately quits on the second process.
Also, it appears that the connection is still available between the
client and server - unlike previous version. Before, when getting a
ClientAbortException, the connection was no longer useful.

I think I should go back to an unpatched version of 3.0.1 at a minimum.

-----Original Message-----
From: Martin, Ray
Sent: Wednesday, September 15, 2010 12:58 PM
To: users_at_glassfish.dev.java.net
Subject: RE: Re: Interrupted idle thread

Now using glassfish 3.0.1

Problem remains.

Placed patch jars into glassfish directory.

Still get the "interrupting idle thread...".

Was always using a synchronous client up to this point.

Created an asynchronous client - problem remains.

I needed a more exact replication of the problem.

I used Netbeans 6.9.1.

Created a web service using session bean - the operations are setup and
activate.

In the session bean, activate method, I put the following"

System.out.println("Hello");
int min = 30;
long now = System.currentTimeMillis();
long wait = min*60*1000;
long go = now+wait;
min--;
while(now<go) {
    try {
        Thread.sleep(1000);
    } catch ...
    }
    now = System.currentTimeMillis();
    if(cnt>59) {
System.out.println(min--);
        cnt = 0;
    }
    cnt++;
}

The sys out is written to the glassfish log, of course. When the
Activate SOAP message arrives, Hello appears, one minute later a 29
appears. The sys out keeps counting down until a 16 appears.

Then, WARNING: Interrupting idle Thread: http-thread-pool-8080-(2),
appears.

Then Hello appears again, then a 15, then a 29, 14, 28

Finally, the first minute count goes to 1 (the second count was 16)
Then the log shows:
WARNING: invocation error on ejb endpoint ...

The second count continues to wind down to 1, then the error:
WARNING: invocation error on ejb endpoint ...


Note: I cannot find the " http request-timeout-seconds" in domain.xml

Note: My brother says there is a mechanism to tell the client not to
timeout - he could not remember what it was. Anyone?

Please help!

Thanx,
Ray


-----Original Message-----
From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
Sent: Thursday, September 02, 2010 12:54 PM
To: users_at_glassfish.dev.java.net
Subject: Re: Interrupted idle thread

Martin,

here is one more issue been fixed after 3.0.1 release, might be
related [1].
At the bottom you may find the proposed patch.

WBR,
Alexey.

[1] https://grizzly.dev.java.net/issues/show_bug.cgi?id=862

On Sep 2, 2010, at 14:06 , Martin, Ray wrote:

> Grizzly Framework 1.9.18-o
>
>
>
> -----Original Message-----
> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
> Sent: Wednesday, September 01, 2010 5:08 PM
> To: users_at_glassfish.dev.java.net
> Subject: Re: Interrupted idle thread
>
> Can you pls. check the server.log file, there should be the line like
> "Starting Grizzly Framework 1.9......."
>
> which exactly version you see there?
>
> Thanks.
> Alexey.
>
> On Sep 1, 2010, at 22:53 , Martin, Ray wrote:
>
>> Glassfish V3 also does another querky thing.
>>
>> When the process finishes running - in v2 it would send back the SOAP
>> response.
>> In v3, I get "WARNING: invocation error on ejb endpoint:
>> ConductorWebService at ConductorWebServiceSerice/
>> ConductorWebService :
>> com.ctc.wstx.exc.WstxIOException: null
>>
>> Ya gotta luv v2 - no such hassle :)
>>
>> -----Original Message-----
>> From: Martin, Ray
>> Sent: Wednesday, September 01, 2010 2:59 PM
>> To: users_at_glassfish.dev.java.net
>> Subject: RE: Re: Interrupted idle thread
>>
>> I would be glad to do so - I just have not figured out how just yet.
>>
>> -----Original Message-----
>> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
>> Sent: Wednesday, September 01, 2010 2:57 PM
>> To: users_at_glassfish.dev.java.net
>> Subject: Re: Interrupted idle thread
>>
>> I see.
>> If you can provide some simple testcase to reproduce the issue - I'll
>> try to figure out what's going wrong.
>>
>> WBR,
>> Alexey.
>>
>> On Sep 1, 2010, at 20:30 , Martin, Ray wrote:
>>
>>> Yes, the netbeans auto generation code now uses an AbstractFacade -
>>> it
>>> is gonna take some new understanding.
>>>
>>> V3.0.1 is still broken with respect to JAX-WS. I didn't measure the
>>> time just yet. But, when a process is activated via a JAX-WS
>>> command,
>>> V3 eventually does the "WARNING: Interrupting idle Thread:
>>> http-thread-pool-8080-(2)" - which does not mean a whole lot to me.
>>>
>>> BUT, then V3 does a very bad thing - it acts as though it had
>>> received a
>>> second JAX-WS message and my process is activated the second time -
>>> now
>>> I have two processes pouring junk into my database.
>>>
>>> I MUST go back to v2 - it worked - I have been running and testing
>>> these
>>> routines for months. Switching to V3 has crushed me.
>>>
>>> -----Original Message-----
>>> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
>>> Sent: Wednesday, September 01, 2010 4:47 AM
>>> To: users_at_glassfish.dev.java.net
>>> Subject: Re: Interrupted idle thread
>>>
>>> Hi Martin,
>>>
>>> didn't expect NB 6.9.1 makes something different than 6.8. Hope
>>> you'll
>>> finally make things work with it.
>>>
>>> Alexey.
>>>
>>> On Aug 31, 2010, at 18:29 , Martin, Ray wrote:
>>>
>>>> The javaee-api-6.0.jar answered the redlines in my prior code.
>>>>
>>>> I retested the code generators in netbeans - wow, life is different
>>>> than
>>>> the previous version - looks like I have lots of new learning to
>>>> do. I
>>>> guess this is all called progress.
>>>>
>>>> Tore the databases down in preparation for rebuilding.
>>>>
>>>> Deployed and ran - got an ORA-02049 - this looks like a problem in
>>>> developer space!!! Ball is back in my court.
>>>>
>>>> -----Original Message-----
>>>> From: Martin, Ray
>>>> Sent: Tuesday, August 31, 2010 12:19 PM
>>>> To: users_at_glassfish.dev.java.net
>>>> Subject: RE: RE: Re: Interrupted idle thread
>>>>
>>>> I figured those wonderful code generators in netbeans would tell me
>>>> what/how to do annotations. Wrong - the code generators did not
>>>> write
>>>> code like they use to. Don't know if it is because of the missing
>>>> libraries or if they just don't write code no more???
>>>>
>>>> -----Original Message-----
>>>> From: Martin, Ray
>>>> Sent: Tuesday, August 31, 2010 12:06 PM
>>>> To: users_at_glassfish.dev.java.net
>>>> Subject: RE: Re: Interrupted idle thread
>>>>
>>>> I lucked out - we have netbeans 6.9.1
>>>>
>>>> I have installed.
>>>>
>>>> At admin console, I again setup all my connections pools and jdbc
>>>> resource. Did not yet do my security realm.
>>>>
>>>> Had to reset the server on my projects in netbeans.
>>>>
>>>> Everything is good so far - then oops - where are the EJB
>>>> libraries?
>>>>
>>>> -----Original Message-----
>>>> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
>>>> Sent: Tuesday, August 31, 2010 10:46 AM
>>>> To: users_at_glassfish.dev.java.net
>>>> Subject: Re: Interrupted idle thread
>>>>
>>>> It should work, but in general 3.0.1 is more stable, that's why IMO
>>>> it
>>>> could make sense to try it.
>>>>
>>>> Thanks.
>>>> Alexey.
>>>>
>>>> On Aug 31, 2010, at 16:21 , Martin, Ray wrote:
>>>>
>>>>> Okay - I will make the switch to a newer version ASAP.
>>>>>
>>>>> BUT - to get me back up and running, could I not set the other
>>>>> variable???
>>>>>
>>>>>
>>>>> The link that you provided says:
>>>>> Additional comments
>>>>> "We misinterpret thread-pool configuration elements in domain.xml,
>>>>> we
>>>>> use thread-pool->idle-thread-
>>>>> timeout-seconds to set transaction timeout and http->timeout-
>>>>> seconds
>>>>> to
>>>>> set thread's idle-timeout."
>>>>>
>>>>>
>>>>> So, if I knew the syntax, I could set
>>>>> thread-pool-idle-thread-timeout-seconds
>>>>> Just for enough to get me going - I am in a really bad position at
>>>>> the
>>>>> moment.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
>>>>> Sent: Tuesday, August 31, 2010 9:58 AM
>>>>> To: users_at_glassfish.dev.java.net
>>>>> Subject: Re: Interrupted idle thread
>>>>>
>>>>> Now I see, most probably you're using GF v3, where we had issue
>>>>> [1].
>>>>> Can you pls. switch to GF v3.0.1?
>>>>>
>>>>> I don't remember exactly if NB 6.8 supports GF v3 server out of
>>>>> the
>>>>> box, or you need additional plugin to be installed, but with NB
>>>>> 6.9.1
>>>>> you can use GF 3.0.1 (AFAIR it's one you have out of the box),
>>>>> which
>>>>> should have that issue resolved.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> WBR,
>>>>> Alexey.
>>>>>
>>>>> [1] https://glassfish.dev.java.net/issues/show_bug.cgi?id=11490
>>>>>
>>>>>
>>>>> On Aug 31, 2010, at 14:21 , Martin, Ray wrote:
>>>>>
>>>>>> I am using netbeans6.8 and whatever glassfish comes with it.
>>>>>>
>>>>>> At the glassfish admin console, I press the version button, I get
>>>>>> v3
>>>>>> (no
>>>>>> numbers)
>>>>>>
>>>>>> I am very sorry to say, that I cannot get the descriptor onto the
>>>>>> Internet.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
>>>>>> Sent: Tuesday, August 31, 2010 7:35 AM
>>>>>> To: users_at_glassfish.dev.java.net
>>>>>> Subject: Re: Interrupted idle thread
>>>>>>
>>>>>> Martin,
>>>>>>
>>>>>> just to make sure, are you using the Glassfish 3.0.1 release?
>>>>>> Can you pls. share the domain.xml descriptor?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Alexey.
>>>>>>
>>>>>> On Aug 31, 2010, at 13:28 , Martin, Ray wrote:
>>>>>>
>>>>>>> Mr. Lubke,
>>>>>>>
>>>>>>> I did the following:
>>>>>>> ...
>>>>>>> <http request-timeout-seconds="-1" ...
>>>>>>> ...
>>>>>>>
>>>>>>> I assume that -1 means to disable.
>>>>>>>
>>>>>>> After restarting glassfish and deploying the EJBModule in an
>>>>>>> Enterprise
>>>>>>> app (so it will see the third party jars) - still got the
>>>>>>> There are no highlighted small blocks.>>>>>>>
>>>>>>> But, worse than that is the fact that something causes the SOAP
>>>>>>> message
>>>>>>> to be repeated - then my process is not protected against that
>>>>>>> repeat
>>>>>>> SOAP message again kicks off and stomps all over the database
>>>>>>> records
>>>>>>> that it just wrote before the timeout.
>>>>>>>
>>>>>>> I am going to try an asynchronous client to see if that will
>>>>>>> take
>>>>>>> care
>>>>>>> of the problem.
>>>>>>>
>>>>>>> Thanx for your help.
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Ryan Lubke [mailto:ryan.lubke_at_oracle.com]
>>>>>>> Sent: Monday, August 30, 2010 8:51 PM
>>>>>>> To: Martin, Ray
>>>>>>> Cc: users_at_glassfish.dev.java.net
>>>>>>> Subject: RE: Interrupted idle thread
>>>>>>>
>>>>>>> Hello Martin,
>>>>>>>
>>>>>>> I believe you'll be able to resolve this issue by either
>>>>>>> increasing
>>>>>>> the
>>>>>>> request-timeout-seconds property or disabling
>>>>>>> the feature altogether.
>>>>>>>
>>>>>>> This can be done by backing up your
>>>>>>> GF_HOME/domains/domain1/config/domain.xml and then modify the
>>>>>>> http
>>>>>>> element
>>>>>>> associated with the protocol your listener is using.
>>>>>>>
>>>>>>> For example, if you're using the out-of-the-box configuration,
>>>>>>> you
>>>>>>> can
>>>>>>> change the timeout for requests made to port 8080 like
>>>>>>> so:
>>>>>>>
>>>>>>>
>>>>>>> <protocol name="http-listener-1">
>>>>>>> <http request-timeout-seconds="60" default-virtual-
>>>>>>> server="server"
>>>>>>> max-connections="250" />
>>>>>>> </protocol>
>>>>>>>
>>>>>>> It doesn't appear that this value can currently be set to -1 via
>>>>>>> asadmin
>>>>>>>
>>>>>>> command line, so I'll be looking into resolving
>>>>>>> that shortly.
>>>>>>>
>>>>>>> Could you please try updating the domain.xml using the above xml
>>>>>>> snippet
>>>>>>>
>>>>>>> as a guide and let us know if it resolves your problem?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -rl
>>>>>>>
>>>>>>>
>>>>
> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: users-
>>>>>>> help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: users-
>>>>>> help_at_glassfish.dev.java.net
>>>>>>
>>>>>>
>>>>>>
>>>
---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: users-
>>>>>> help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>>
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>>
> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>
>>>>
>>>>
> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>
>>>>
>>>>
> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>
>>>>
>>>>
> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>
>>>
>>>
---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
For additional commands, e-mail: users-help_at_glassfish.dev.java.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
For additional commands, e-mail: users-help_at_glassfish.dev.java.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
For additional commands, e-mail: users-help_at_glassfish.dev.java.net