dev@glassfish.java.net

Re: glassfish broken after jaxws tests

From: Tom Amiro <Tom.Amiro_at_Sun.COM>
Date: Wed, 07 Mar 2007 09:17:06 -0500

Sreejith, Binod

I've collected more data. Attached is the jstack trace and the domain.xml
when GF is hosed.

Also, here is a diff showing the changes (bold) to the domain.xml, made
by the
SAAJ tests, which seem to trigger the problem.

144c144
< <request-processing header-buffer-length-in-bytes="8192"
initial-thread-count="2" request-timeout-in-seconds="30"
thread-count="20" thread-increment="1"/>
---
 >         <request-processing header-buffer-length-in-bytes="8192" 
*initial-thread-count="5"* request-timeout-in-seconds="30" 
thread-count="20" thread-increment="1"/>
246c246
<       <java-config classpath-suffix="" debug-enabled="false" 
debug-options="-Xdebug 
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" 
env-classpath-ignored="true" java-home="${com.sun.aas.javaRoot}" 
javac-options="-g" rmic-options="-iiop -poa -alwaysgenerate 
-keepgenerated -g" 
system-classpath="${com.sun.aas.installRoot}/lib/appserv-launch.jar">
---
 >       <java-config 
*classpath-suffix="/var/Sun/tests/jaxm-test-gf/lib/jaxmutil.jar:/var/Sun/tests/jaxm-test-gf/lib/reporter.jar:/var/Sun/tests/jaxm-test-gf/lib/commons-logging.jar:"* 
debug-enabled="false" debug-options="-Xdebug 
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" 
env-classpath-ignored="true" java-home="${com.sun.aas.javaRoot}" 
javac-options="-g" rmic-options="-iiop -poa -alwaysgenerate 
-keepgenerated -g" 
system-classpath="${com.sun.aas.installRoot}/lib/appserv-launch.jar">
Tom
Sreejith A K - Sun Microsystems wrote:
> Binod,
>
>
> I am facing a similar issue with glassfish (b37 promoted build)
> - but this is with a small application , not really intensive
>
> Please see attached - the jstack trace
>
>
>
> Thanks
> Sreejith
>
>
> Binod wrote On 03/07/07 08:28,:
>
>> Tom Amiro wrote:
>>
>>> Jon,
>>>
>>> Thanks, that just what I needed. But it says domain.xml is valid.
>>> While at the same, time the browser just hangs on
>>> http://wsqeb6.east:8090, so the server still isn't working.
>>>
>>> One other thing, when I try to stop the domain1, asadmin stop-domain
>>> just hangs. After killing the appserv process, asadmin start-domain
>>> seems to execute OK but the server isn't working.
>>>
>>> Tom
>>>
>> Look like server is doing something really intensive.
>> Can you take a jstack when it appears hanging?
>>
>> - Binod.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>------------------------------------------------------------------------
>
>----------------- t_at_1 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff01e7f8	void Parker::park(int,long long) + 0x358
>0xff0a0550	Unsafe_Park + 0x284
>0xf900bc20	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf900bbc4	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.LockSupport.park() bci:5 line:118 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() bci:0 line:681 (Interpreted frame)
>0xf90057a8	* java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) bci:54 line:837 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) bci:24 line:1144 (Interpreted frame)
>0xf9005764	* java.util.concurrent.CountDownLatch.await() bci:5 line:199 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.waitDone() bci:10 line:367 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.web.EmbeddedWebContainer$WebEngine.startChildren() bci:126 line:447 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase.start() bci:318 line:1220 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.StandardEngine.start() bci:96 line:520 (Interpreted frame)
>0xf9005c2c	* org.apache.catalina.startup.Embedded.start() bci:123 line:916 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.WebContainer.start() bci:234 line:853 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.PEWebContainer.startInstance() bci:12 line:744 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.PEWebContainerLifecycle.onStartup(com.sun.enterprise.server.ServerContext) bci:3 line:71 (Interpreted frame)
>0xf9005c2c	* com.sun.enterprise.server.ApplicationServer.onStartup() bci:36 line:332 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.server.ondemand.OnDemandServer.onStartup() bci:15 line:112 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.server.PEMain.run(java.lang.String) bci:153 line:381 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.server.PEMain.main(java.lang.String[]) bci:332 line:308 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfed0cc40	oopDesc*Reflection::invoke(instanceKlassHandle,methodHandle,Handle,int,objArrayHandle,BasicType,objArrayHandle,int,Thread*) + 0x1520
>0xfedeb498	oopDesc*Reflection::invoke_method(oopDesc*,Handle,objArrayHandle,Thread*) + 0x258
>0xfedeaec4	JVM_InvokeMethod + 0x2dc
>0xfe900358	Java_sun_reflect_NativeMethodAccessorImpl_invoke0 + 0x10
><interpreter> method entry point (kind = native)
>0xf900bbc4	* sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) bci:0 (Interpreted frame)
>0xf9005874	* sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) bci:87 line:39 (Interpreted frame)
>0xf9005874	* sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) bci:6 line:25 (Interpreted frame)
>0xf9005d3c	* java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) bci:111 line:585 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.PELaunch.main(java.lang.String[]) bci:108 line:274 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfedbb0e4	jni_CallStaticVoidMethod + 0x514
>0x000121fc	main + 0x115c
>0x00011088	_start + 0x108
>----------------- t_at_2 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff3285e0	_lwp_cond_timedwait + 0x1c
>0xfecd4cc0	int Monitor::wait(int,long) + 0x308
>0xfecd5874	void VMThread::run() + 0x1b4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_3 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722601 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.Object.wait() bci:2 line:474 (Interpreted frame)
>0xf9005764	* java.lang.ref.Reference$ReferenceHandler.run() bci:46 line:116 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_4 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722601 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame)
>0xf9005874	* java.lang.ref.ReferenceQueue.remove() bci:2 line:132 (Interpreted frame)
>0xf9005874	* java.lang.ref.Finalizer$FinalizerThread.run() bci:3 line:159 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_5 -----------------
>0xff33b5b0	__lwp_park + 0x10
>0xfed3113c	int check_pending_signals(int) + 0x284
>0xfed30c84	void signal_thread_entry(JavaThread*,Thread*) + 0x34
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_6 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecd4e08	int Monitor::wait(int,long) + 0x450
>0xfed31a68	void CompileBroker::compiler_thread_loop() + 0x354
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_7 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecd4b2c	int Monitor::wait(int,long) + 0x174
>0xfed36c4c	void LowMemoryDetector::low_memory_detector_thread_entry(JavaThread*,Thread*) + 0x2f0
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_8 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed3362c	int os_sleep(long long,int) + 0xc8
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfed33350	void WatcherThread::run() + 0xe8
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_9 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff01e7f8	void Parker::park(int,long long) + 0x358
>0xff0a0550	Unsafe_Park + 0x284
>0xf900bc20	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf900bbc4	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.LockSupport.park() bci:5 line:118 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() bci:0 line:681 (Interpreted frame)
>0xf90057a8	* java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) bci:54 line:837 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) bci:24 line:1144 (Interpreted frame)
>0xf9005764	* java.util.concurrent.CountDownLatch.await() bci:5 line:199 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.util.FeatureAvailability.waitForFeature(java.lang.String, java.lang.String) bci:166 line:242 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.management.support.DottedNamesBase$1.doRun() bci:7 line:161 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.runSync() bci:64 line:283 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.run() bci:8 line:317 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.Executors$RunnableAdapter.call() bci:4 line:417 (Interpreted frame)
>0xf9005d3c	* java.util.concurrent.FutureTask$Sync.innerRun() bci:22 line:269 (Interpreted frame)
>0xf9005764	* java.util.concurrent.FutureTask.run() bci:4 line:123 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) bci:44 line:650 (Interpreted frame)
>0xf9005764	* java.util.concurrent.ThreadPoolExecutor$Worker.run() bci:28 line:675 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_10 -----------------
>0xff33b8e0	_so_accept + 0x4
>0xf900bc20	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:-9276523 (Interpreted frame)
>0xf900bbc4	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:0 (Interpreted frame)
>0xf90058b8	* sun.nio.ch.ServerSocketChannelImpl.accept() bci:130 line:145 (Interpreted frame)
>0xf9005874	* sun.nio.ch.ServerSocketAdaptor.accept() bci:39 line:84 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.acceptSocket() bci:4 line:174 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.accept() bci:21 line:154 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocketImpl.accept(java.net.SocketImpl) bci:4 line:161 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.implAccept(java.net.Socket) bci:50 line:450 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.accept() bci:48 line:421 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.tcp.TCPTransport.run() bci:59 line:334 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_11 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188749428 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.Object.wait() bci:2 line:474 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.mainLoop() bci:28 line:483 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.run() bci:1 line:462 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_12 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:197798004 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame)
>0xf9005874	* java.lang.ref.ReferenceQueue.remove() bci:2 line:132 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.ObjectTable$Reaper.run() bci:3 line:336 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_13 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff3285e0	_lwp_cond_timedwait + 0x1c
>0xfecf4b2c	void ObjectMonitor::wait(long long,int,Thread*) + 0x778
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722602 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* sun.misc.GC$Daemon.run() bci:51 line:100 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_14 -----------------
>0xff33beb0	ioctl + 0x8
>----------------- t_at_15 -----------------
>0xff33beb0	ioctl + 0x8
>----------------- t_at_16 -----------------
>0xff33beb0	ioctl + 0x8
>----------------- t_at_18 -----------------
>0xff33beb0	ioctl + 0x8
>----------------- t_at_20 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* sun.rmi.transport.DGCImpl$LeaseChecker.run() bci:3 line:310 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_21 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff01e7f8	void Parker::park(int,long long) + 0x358
>0xff0a0550	Unsafe_Park + 0x284
>0xf900bc20	* sun.misc.Unsafe.park(boolean, long) bci:196153214 (Interpreted frame)
>0xf900bbc4	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.LockSupport.park() bci:5 line:118 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() bci:0 line:681 (Interpreted frame)
>0xf90057a8	* java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) bci:54 line:837 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) bci:24 line:1144 (Interpreted frame)
>0xf9005764	* java.util.concurrent.CountDownLatch.await() bci:5 line:199 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.util.FeatureAvailability.waitForFeature(java.lang.String, java.lang.String) bci:166 line:242 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.management.support.DottedNamesBase$1.doRun() bci:7 line:161 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.runSync() bci:64 line:283 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.run() bci:8 line:317 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.Executors$RunnableAdapter.call() bci:4 line:417 (Interpreted frame)
>0xf9005d3c	* java.util.concurrent.FutureTask$Sync.innerRun() bci:22 line:269 (Interpreted frame)
>0xf9005764	* java.util.concurrent.FutureTask.run() bci:4 line:123 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) bci:44 line:650 (Interpreted frame)
>0xf9005764	* java.util.concurrent.ThreadPoolExecutor$Worker.run() bci:28 line:675 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_22 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.Object.wait() bci:2 line:474 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.mainLoop() bci:28 line:483 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.run() bci:1 line:462 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_23 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff3285e0	_lwp_cond_timedwait + 0x1c
>0xfecf4b2c	void ObjectMonitor::wait(long long,int,Thread*) + 0x778
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722601 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.mainLoop() bci:201 line:509 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.run() bci:1 line:462 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_24 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.Object.wait() bci:2 line:474 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.mainLoop() bci:28 line:483 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.run() bci:1 line:462 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_25 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xfecf490c	void ObjectMonitor::wait(long long,int,Thread*) + 0x558
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.Object.wait() bci:2 line:474 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.mainLoop() bci:28 line:483 (Interpreted frame)
>0xf9005764	* java.util.TimerThread.run() bci:1 line:462 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_26 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff3285e0	_lwp_cond_timedwait + 0x1c
>0xfecf4b2c	void ObjectMonitor::wait(long long,int,Thread*) + 0x778
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722601 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run() bci:95 line:500 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_29 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:791111 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.admin.server.core.channel.RMIClient.run() bci:11 line:141 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_30 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:229836 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.management.support.LoaderBase.mySleep(long) bci:3 line:223 (Interpreted frame)
>0xf90057a8	* com.sun.enterprise.management.support.Loader$DeferredRegistrationThread.run() bci:107 line:376 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_31 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff01e7f8	void Parker::park(int,long long) + 0x358
>0xff0a0550	Unsafe_Park + 0x284
>0xf900bc20	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf900bbc4	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.LockSupport.park() bci:5 line:118 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() bci:41 line:1767 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.ArrayBlockingQueue.take() bci:20 line:341 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.management.support.LoaderRegThread.process() bci:12 line:232 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.management.support.LoaderRegThread.run() bci:13 line:143 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_34 -----------------
>0xff33b8e0	_so_accept + 0x4
>0xf900bc20	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:-9276523 (Interpreted frame)
>0xf900bbc4	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:0 (Interpreted frame)
>0xf90058b8	* sun.nio.ch.ServerSocketChannelImpl.accept() bci:130 line:145 (Interpreted frame)
>0xf9005874	* sun.nio.ch.ServerSocketAdaptor.accept() bci:39 line:84 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.acceptSocket() bci:4 line:174 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.accept() bci:21 line:154 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocketImpl.accept(java.net.SocketImpl) bci:4 line:161 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.implAccept(java.net.Socket) bci:50 line:450 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.accept() bci:48 line:421 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.tcp.TCPTransport.run() bci:59 line:334 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_35 -----------------
>0xff33b8e0	_so_accept + 0x4
>0xf900bc20	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:-9276523 (Interpreted frame)
>0xf900bbc4	* sun.nio.ch.ServerSocketChannelImpl.accept0(java.io.FileDescriptor, java.io.FileDescriptor, java.net.InetSocketAddress[]) bci:0 (Interpreted frame)
>0xf90058b8	* sun.nio.ch.ServerSocketChannelImpl.accept() bci:130 line:145 (Interpreted frame)
>0xf9005874	* sun.nio.ch.ServerSocketAdaptor.accept() bci:39 line:84 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.acceptSocket() bci:4 line:174 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocket.accept() bci:21 line:154 (Interpreted frame)
>0xf9005874	* com.sun.enterprise.server.ss.provider.ASServerSocketImpl.accept(java.net.SocketImpl) bci:4 line:161 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.implAccept(java.net.Socket) bci:50 line:450 (Interpreted frame)
>0xf9005764	* java.net.ServerSocket.accept() bci:48 line:421 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.tcp.TCPTransport.run() bci:59 line:334 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_37 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff3285e0	_lwp_cond_timedwait + 0x1c
>0xfecf4b2c	void ObjectMonitor::wait(long long,int,Thread*) + 0x778
>0xfecf4304	void ObjectSynchronizer::wait(Handle,long long,Thread*) + 0xd8
>0xfecf3f18	JVM_MonitorWait + 0x2f0
>0xf900bc20	* java.lang.Object.wait(long) bci:188722601 (Interpreted frame)
>0xf900bbc4	* java.lang.Object.wait(long) bci:0 (Interpreted frame)
>0xf9005764	* java.lang.ref.ReferenceQueue.remove(long) bci:44 line:116 (Interpreted frame)
>0xf9005874	* sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run() bci:95 line:500 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_41 -----------------
>0xff33c624	___lwp_cond_wait + 0x4
>0xff01e7f8	void Parker::park(int,long long) + 0x358
>0xff0a0550	Unsafe_Park + 0x284
>0xf900bc20	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf900bbc4	* sun.misc.Unsafe.park(boolean, long) bci:0 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.LockSupport.park() bci:5 line:118 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() bci:0 line:681 (Interpreted frame)
>0xf90057a8	* java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(int) bci:67 line:809 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) bci:10 line:1124 (Interpreted frame)
>0xf9005764	* java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock() bci:5 line:423 (Interpreted frame)
>0xf9005c2c	* org.apache.catalina.core.ContainerBase.getLogger() bci:4 line:455 (Interpreted frame)
>0xf9005d3c	* org.apache.catalina.core.ContainerBase.getLogger() bci:67 line:463 (Interpreted frame)
>0xf9005d3c	* org.apache.catalina.core.ContainerBase.getLogger() bci:67 line:463 (Interpreted frame)
>0xf9005874	* org.apache.catalina.core.ApplicationContext.log(java.lang.String, java.lang.Throwable) bci:4 line:805 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ApplicationContextFacade.log(java.lang.String, java.lang.Throwable) bci:50 line:294 (Interpreted frame)
>0xf9005c2c	* org.apache.catalina.core.StandardContext.listenerStart() bci:489 line:4472 (Interpreted frame)
>0xf90057a8	* org.apache.catalina.core.StandardContext.start() bci:1624 line:5120 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.WebModule.start() bci:1 line:304 (Interpreted frame)
>0xf9005c2c	* com.sun.enterprise.web.LifecycleStarter.doRun() bci:4 line:45 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.runSync() bci:64 line:283 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase._submit(com.sun.appserv.management.util.misc.RunnableBase$HowToRun) bci:52 line:155 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.submit() bci:5 line:171 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.VirtualServer.startChildren() bci:54 line:1675 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase.start() bci:318 line:1220 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.StandardHost.start() bci:277 line:938 (Interpreted frame)
>0xf9005c2c	* com.sun.enterprise.web.LifecycleStarter.doRun() bci:4 line:45 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.runSync() bci:64 line:283 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.run() bci:8 line:317 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.Executors$RunnableAdapter.call() bci:4 line:417 (Interpreted frame)
>0xf9005d3c	* java.util.concurrent.FutureTask$Sync.innerRun() bci:22 line:269 (Interpreted frame)
>0xf9005764	* java.util.concurrent.FutureTask.run() bci:4 line:123 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) bci:44 line:650 (Interpreted frame)
>0xf9005764	* java.util.concurrent.ThreadPoolExecutor$Worker.run() bci:28 line:675 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_45 -----------------
>0xf94018c4	0xf94018c4	* com.sun.xml.stream.XMLReaderImpl.next() bci:5 line:557 (Compiled frame; information may be imprecise)
>0xf940161c	0xf940161c	* com.sun.xml.ws.util.xml.XMLStreamReaderFilter.next() bci:4 line:56 (Compiled frame)
>0xf9401000	0xf9401000	* com.sun.xml.ws.streaming.XMLStreamReaderUtil.next(javax.xml.stream.XMLStreamReader) bci:79 line:64 (Compiled frame)
>0xf9408134	0xf9408134	* com.sun.xml.ws.streaming.XMLStreamReaderUtil.nextContent(javax.xml.stream.XMLStreamReader) bci:1 line:85 (Compiled frame)
>0xf90058b8	* com.sun.xml.ws.streaming.XMLStreamReaderUtil.nextElementContent(javax.xml.stream.XMLStreamReader) bci:1 line:75 (Interpreted frame)
>0xf90058b8	* com.sun.xml.ws.server.SDDocumentImpl.create(com.sun.xml.ws.api.server.SDDocumentSource, javax.xml.namespace.QName, javax.xml.namespace.QName) bci:16 line:85 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.server.EndpointFactory.categoriseMetadata(java.util.List, javax.xml.namespace.QName, javax.xml.namespace.QName) bci:49 line:419 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.server.EndpointFactory.createEndpoint(java.lang.Class, boolean, com.sun.xml.ws.api.server.Invoker, javax.xml.namespace.QName, javax.xml.namespace.QName, com.sun.xml.ws.api.server.Container, com.sun.xml.ws.api.WSBinding, com.sun.xml.ws.api.server.SDDocumentSource, java.util.Collection, org.xml.sax.EntityResolver, boolean) bci:210 line:146 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.api.server.WSEndpoint.create(java.lang.Class, boolean, com.sun.xml.ws.api.server.Invoker, javax.xml.namespace.QName, javax.xml.namespace.QName, com.sun.xml.ws.api.server.Container, com.sun.xml.ws.api.WSBinding, com.sun.xml.ws.api.server.SDDocumentSource, java.util.Collection, org.xml.sax.EntityResolver, boolean) bci:18 line:420 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.transport.http.DeploymentDescriptorParser.parseAdapters(javax.xml.stream.XMLStreamReader) bci:327 line:237 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.transport.http.DeploymentDescriptorParser.parse(java.lang.String, java.io.InputStream) bci:24 line:133 (Interpreted frame)
>0xf9005874	* com.sun.xml.ws.transport.http.servlet.WSServletContextListener.contextInitialized(javax.servlet.ServletContextEvent) bci:122 line:97 (Interpreted frame)
>0xf9005c2c	* org.apache.catalina.core.StandardContext.listenerStart() bci:435 line:4468 (Interpreted frame)
>0xf90057a8	* org.apache.catalina.core.StandardContext.start() bci:1624 line:5120 (Interpreted frame)
>0xf9005764	* com.sun.enterprise.web.WebModule.start() bci:1 line:304 (Interpreted frame)
>0xf9005c2c	* com.sun.enterprise.web.LifecycleStarter.doRun() bci:4 line:45 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.runSync() bci:64 line:283 (Interpreted frame)
>0xf9005764	* com.sun.appserv.management.util.misc.RunnableBase.run() bci:8 line:317 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.Executors$RunnableAdapter.call() bci:4 line:417 (Interpreted frame)
>0xf9005d3c	* java.util.concurrent.FutureTask$Sync.innerRun() bci:22 line:269 (Interpreted frame)
>0xf9005764	* java.util.concurrent.FutureTask.run() bci:4 line:123 (Interpreted frame)
>0xf9005c2c	* java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) bci:44 line:650 (Interpreted frame)
>0xf9005764	* java.util.concurrent.ThreadPoolExecutor$Worker.run() bci:28 line:675 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_47 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:23841040 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() bci:22 line:1795 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_48 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:23841040 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() bci:22 line:1795 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_49 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:23841040 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() bci:22 line:1795 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>----------------- t_at_50 -----------------
>0xff33c11c	__pollsys + 0x4
>0xff2d9f28	poll + 0x7c
>0xfed33724	int os_sleep(long long,int) + 0x1c0
>0xfed316cc	int os::sleep(Thread*,long long,int) + 0x284
>0xfedc484c	JVM_Sleep + 0x260
>0xf900bc20	* java.lang.Thread.sleep(long) bci:23841040 (Interpreted frame)
>0xf900bbc4	* java.lang.Thread.sleep(long) bci:0 (Interpreted frame)
>0xf9005764	* org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() bci:22 line:1795 (Interpreted frame)
>0xf9005c2c	* java.lang.Thread.run() bci:11 line:595 (Interpreted frame)
>0xf9000218	<StubRoutines>
>0xfecdc090	void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*) + 0x5b8
>0xfecf38c0	void JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,symbolHandle,symbolHandle,Thread*) + 0x18c
>0xfecf371c	void thread_entry(JavaThread*,Thread*) + 0x12c
>0xfecf35a0	void JavaThread::run() + 0x1f4
>0xff02af08	void*_start(void*) + 0x200
>0xff33b510	_lwp_start
>
>  
>
>------------------------------------------------------------------------
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>  
>