users@glassfish.java.net

glassfish hangs, any opinion of jstack

From: Mladen Adamovic <mladen.adamovic_at_gmail.com>
Date: Sat, 28 Jul 2012 22:43:19 +0200

Hi guys,

I had application running on Glassfish 2.1 without problem but recently the
hard drive failed and I tried to setup new VPS, Glassfish 3.1.1.3 (latest)
and the app start to hang. I thought I was facing some problems with
connections and Virtuozzo limits, so I've just upgrades to quad core i7
server with 16GB RAM and RAID1 ;-)
however, I have the same problem - glassfish hangs. I put ulimit -n to
256000.

Since the following app were working on previous setup with glassfish 2.1 I
hope some of glassfish developers could take a look.

I put a script which will log glassfish just before it hangs, I can post it
as soon as that script succeed with it, in the meantime, this is a current
working glassfish log if someone could take a look if seeing something
problematic here (so this is the server which didn't hang out yet, but
after a period in production):

Deadlock Detection:

No deadlocks found.

Thread 18134: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long)
@bci=20, line=196 (Interpreted frame)
 -
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.util.concurrent.SynchronousQueue$TransferStack$SNode,
boolean, long) @bci=174, line=424 (Interpreted frame)
 -
java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.lang.Object,
boolean, long) @bci=102, line=323 (Interpreted frame)
 - java.util.concurrent.SynchronousQueue.poll(long,
java.util.concurrent.TimeUnit) @bci=11, line=874 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=62, line=945
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17950: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17785: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17784: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17783: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17782: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17778: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17777: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17776: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 - wikilibrary.EmailQueueWorker.run() @bci=3, line=17 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17774: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17773: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17772: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17771: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17770: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.felix.scr.impl.ComponentActorThread.run() @bci=21, line=74
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17769: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17764: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=118 (Interpreted
frame)
 - sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run()
@bci=95, line=516 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17763: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long)
@bci=20, line=196 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
@bci=68, line=2025 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=57, line=164 (Interpreted
frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
@bci=4, line=609 (Interpreted frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
@bci=1, line=602 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17761: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.felix.cm.impl.UpdateThread.run() @bci=21, line=67
(Interpreted frame)


Thread 17760: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17759: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - sun.misc.GC$Daemon.run() @bci=51, line=100 (Interpreted frame)


Thread 17758: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=118 (Interpreted
frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=134 (Interpreted
frame)
 - sun.rmi.transport.ObjectTable$Reaper.run() @bci=3, line=333 (Interpreted
frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17757: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0
(Interpreted frame)
 - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=408
(Interpreted frame)
 - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=462
(Interpreted frame)
 - java.net.ServerSocket.accept() @bci=48, line=430 (Interpreted frame)
 - sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop()
@bci=55, line=369 (Interpreted frame)
 - sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() @bci=1, line=341
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17756: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - org.apache.felix.fileinstall.internal.DirectoryWatcher.run() @bci=302,
line=268 (Interpreted frame)


Thread 17678: (state = BLOCKED)


Thread 17755: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - com.sun.enterprise.v3.server.AppServerStartup$1.run() @bci=35, line=163
(Interpreted frame)


Thread 17753: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17752: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17750: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17749: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17747: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17741: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17740: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17739: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17738: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 -
com.sun.grizzly.util.OutputWriter.flushChannel(java.nio.channels.SelectableChannel,
java.nio.ByteBuffer, long) @bci=161, line=126 (Compiled frame)
 -
com.sun.grizzly.util.OutputWriter.flushChannel(java.nio.channels.SelectableChannel,
java.nio.ByteBuffer) @bci=6, line=76 (Compiled frame)
 -
com.sun.grizzly.http.SocketChannelOutputBuffer.flushChannel(java.nio.ByteBuffer)
@bci=117, line=426 (Compiled frame)
 - com.sun.grizzly.http.SocketChannelOutputBuffer.flushBuffer() @bci=30,
line=498 (Compiled frame)
 - com.sun.grizzly.http.SocketChannelOutputBuffer.endRequest() @bci=5,
line=489 (Compiled frame)
 -
com.sun.grizzly.http.ProcessorTask.action(com.sun.grizzly.tcp.ActionCode,
java.lang.Object) @bci=155, line=1127 (Compiled frame)
 - com.sun.grizzly.tcp.Response.action(com.sun.grizzly.tcp.ActionCode,
java.lang.Object) @bci=31, line=268 (Interpreted frame)
 - com.sun.grizzly.tcp.Response.finish() @bci=5, line=412 (Interpreted
frame)
 - org.apache.catalina.connector.OutputBuffer.close() @bci=75, line=341
(Interpreted frame)
 - org.apache.catalina.connector.Response.finishResponse() @bci=4, line=582
(Interpreted frame)
 -
org.apache.catalina.connector.CoyoteAdapter.afterService(com.sun.grizzly.tcp.Request,
com.sun.grizzly.tcp.Response) @bci=38, line=375 (Interpreted frame)
 -
com.sun.enterprise.v3.services.impl.ContainerMapper.afterService(com.sun.grizzly.tcp.Request,
com.sun.grizzly.tcp.Response) @bci=30, line=483 (Interpreted frame)
 - com.sun.grizzly.http.ProcessorTask.finishResponse() @bci=17, line=792
(Interpreted frame)
 - com.sun.grizzly.http.ProcessorTask.postResponse() @bci=16, line=782
(Interpreted frame)
 - com.sun.grizzly.http.ProcessorTask.doProcess() @bci=392, line=758
(Interpreted frame)
 - com.sun.grizzly.http.ProcessorTask.process(java.io.InputStream,
java.io.OutputStream) @bci=7, line=1056 (Interpreted frame)
 -
com.sun.grizzly.http.DefaultProtocolFilter.execute(com.sun.grizzly.Context)
@bci=483, line=229 (Interpreted frame)
 -
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(com.sun.grizzly.Context,
int) @bci=46, line=137 (Compiled frame)
 - com.sun.grizzly.DefaultProtocolChain.execute(com.sun.grizzly.Context,
int) @bci=21, line=104 (Compiled frame)
 - com.sun.grizzly.DefaultProtocolChain.execute(com.sun.grizzly.Context)
@bci=3, line=90 (Interpreted frame)
 - com.sun.grizzly.http.HttpProtocolChain.execute(com.sun.grizzly.Context)
@bci=63, line=79 (Interpreted frame)
 - com.sun.grizzly.ProtocolChainContextTask.doCall() @bci=13, line=54
(Interpreted frame)
 - com.sun.grizzly.SelectionKeyContextTask.call() @bci=7, line=59
(Interpreted frame)
 - com.sun.grizzly.ContextTask.run() @bci=1, line=71 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=46,
line=532 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17737: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17736: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17734: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17733: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17732: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17731: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17730: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17729: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17728: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17727: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
 -
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
@bci=22, line=1771 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17726: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17725: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17724: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17723: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17722: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17721: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17720: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17719: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17718: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17717: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=399
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17712: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
 - java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)


Thread 17711: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17710: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask() @bci=117,
line=374 (Compiled frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.doWork() @bci=10,
line=524 (Interpreted frame)
 - com.sun.grizzly.util.AbstractThreadPool$Worker.run() @bci=9, line=513
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17709: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 - com.sun.grizzly.TCPSelectorHandler.select(com.sun.grizzly.Context)
@bci=20, line=513 (Interpreted frame)
 -
com.sun.grizzly.SelectorHandlerRunner.doSelect(com.sun.grizzly.SelectorHandler,
com.sun.grizzly.NIOContext) @bci=24, line=190 (Interpreted frame)
 - com.sun.grizzly.SelectorHandlerRunner.run() @bci=218, line=132
(Interpreted frame)
 -
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
@bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17708: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.Controller.waitUntilSelectorHandlersStop() @bci=40,
line=1119 (Interpreted frame)
 - com.sun.grizzly.Controller.start() @bci=343, line=869 (Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startListener() @bci=11, line=1306
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.run() @bci=28, line=1144
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startEndpoint() @bci=1, line=1233
(Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint()
@bci=11, line=140 (Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run() @bci=43,
line=342 (Interpreted frame)


Thread 17707: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 - com.sun.grizzly.TCPSelectorHandler.select(com.sun.grizzly.Context)
@bci=20, line=513 (Interpreted frame)
 -
com.sun.grizzly.SelectorHandlerRunner.doSelect(com.sun.grizzly.SelectorHandler,
com.sun.grizzly.NIOContext) @bci=24, line=190 (Compiled frame)
 - com.sun.grizzly.SelectorHandlerRunner.run() @bci=218, line=132 (Compiled
frame)
 -
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
@bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17706: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.Controller.waitUntilSelectorHandlersStop() @bci=40,
line=1119 (Interpreted frame)
 - com.sun.grizzly.Controller.start() @bci=343, line=869 (Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startListener() @bci=11, line=1306
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.run() @bci=28, line=1144
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startEndpoint() @bci=1, line=1233
(Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint()
@bci=11, line=140 (Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run() @bci=43,
line=342 (Interpreted frame)


Thread 17705: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 - com.sun.grizzly.TCPSelectorHandler.select(com.sun.grizzly.Context)
@bci=20, line=513 (Interpreted frame)
 -
com.sun.grizzly.SelectorHandlerRunner.doSelect(com.sun.grizzly.SelectorHandler,
com.sun.grizzly.NIOContext) @bci=24, line=190 (Interpreted frame)
 - com.sun.grizzly.SelectorHandlerRunner.run() @bci=218, line=132
(Interpreted frame)
 -
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
@bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17704: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.Controller.waitUntilSelectorHandlersStop() @bci=40,
line=1119 (Interpreted frame)
 - com.sun.grizzly.Controller.start() @bci=343, line=869 (Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startListener() @bci=11, line=1306
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.run() @bci=28, line=1144
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startEndpoint() @bci=1, line=1233
(Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint()
@bci=21, line=142 (Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run() @bci=43,
line=342 (Interpreted frame)


Thread 17703: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 - com.sun.grizzly.TCPSelectorHandler.select(com.sun.grizzly.Context)
@bci=20, line=513 (Interpreted frame)
 -
com.sun.grizzly.SelectorHandlerRunner.doSelect(com.sun.grizzly.SelectorHandler,
com.sun.grizzly.NIOContext) @bci=24, line=190 (Compiled frame)
 - com.sun.grizzly.SelectorHandlerRunner.run() @bci=218, line=132 (Compiled
frame)
 -
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
@bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17702: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.Controller.waitUntilSelectorHandlersStop() @bci=40,
line=1119 (Interpreted frame)
 - com.sun.grizzly.Controller.start() @bci=343, line=869 (Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startListener() @bci=11, line=1306
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.run() @bci=28, line=1144
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startEndpoint() @bci=1, line=1233
(Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint()
@bci=21, line=142 (Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run() @bci=43,
line=342 (Interpreted frame)


Thread 17701: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=210 (Compiled
frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=65 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69 (Compiled
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80 (Compiled frame)
 - com.sun.grizzly.TCPSelectorHandler.select(com.sun.grizzly.Context)
@bci=20, line=513 (Interpreted frame)
 -
com.sun.grizzly.SelectorHandlerRunner.doSelect(com.sun.grizzly.SelectorHandler,
com.sun.grizzly.NIOContext) @bci=24, line=190 (Compiled frame)
 - com.sun.grizzly.SelectorHandlerRunner.run() @bci=218, line=132 (Compiled
frame)
 -
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)
@bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17700: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - com.sun.grizzly.Controller.waitUntilSelectorHandlersStop() @bci=40,
line=1119 (Interpreted frame)
 - com.sun.grizzly.Controller.start() @bci=343, line=869 (Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startListener() @bci=11, line=1306
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.run() @bci=28, line=1144
(Interpreted frame)
 - com.sun.grizzly.http.SelectorThread.startEndpoint() @bci=1, line=1233
(Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint()
@bci=21, line=142 (Interpreted frame)
 - com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run() @bci=43,
line=342 (Interpreted frame)


Thread 17699: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
line=156 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()
@bci=42, line=1987 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=315
(Interpreted frame)
 - com.sun.enterprise.server.logging.GFFileHandler.log() @bci=4, line=658
(Interpreted frame)
 - com.sun.enterprise.server.logging.GFFileHandler$1.run() @bci=17,
line=179 (Interpreted frame)


Thread 17698: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long)
@bci=20, line=196 (Interpreted frame)
 -
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
@bci=68, line=2025 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=57, line=164 (Interpreted
frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
@bci=4, line=609 (Interpreted frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
@bci=1, line=602 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=78, line=947
(Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=18, line=907
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17697: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.felix.framework.FrameworkStartLevelImpl.run() @bci=37,
line=272 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17696: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.felix.framework.util.EventDispatcher.run() @bci=28, line=1063
(Interpreted frame)
 - org.apache.felix.framework.util.EventDispatcher.access$000() @bci=0,
line=54 (Interpreted frame)
 - org.apache.felix.framework.util.EventDispatcher$1.run() @bci=0, line=101
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)


Thread 17690: (state = BLOCKED)


Thread 17689: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=118 (Interpreted
frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=134 (Interpreted
frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159
(Interpreted frame)


Thread 17688: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116
(Interpreted frame)