Message-ID: <4519FC40.4060208@sun.com> Date: Tue, 26 Sep 2006 21:21:20 -0700 From: vince kraemer User-Agent: Thunderbird 1.5.0.7 (X11/20060909) MIME-Version: 1.0 To: as-tools-iteam@sun.com, Lloyd L Chambers CC: Tim Quinn , Hong Zhang Subject: Interesting thread dump... Content-Type: multipart/mixed; boundary="------------080800050709060907030101" This is a multi-part message in MIME format. --------------080800050709060907030101 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I was doing some deployment testing from NetBeans 5.5 dev on Judy Tang's machine earlier today. The IDE was running in jdk 6 (build 98) and the server was in jdk 5 update 9. We had done a number of simple iteration of... deploy a project (which uses JSR 88 apis), wait for it to finish, repeat cycle. The server wasn't heavily loaded... We were doing a deploy every 3 to 5 seconds (I would guess). After several iterations, the IDE hung... The thread dump is interesting (to me at least). Does it look peculiar to you? I have a suspicion that the server had read the data that it expected and then stopped reading... So, both of the threads that are writers will never get their data "read".... Any other explanation of this state is welcome... Thanks, vbk lighter side note: according to Thunderbird's spell checker, the suggested correction for jdk is vodka.... --------------080800050709060907030101 Content-Type: text/plain; name="tdsep261507" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="tdsep261507" Full thread dump Java HotSpot(TM) Client VM (1.6.0-rc-b98 mixed mode, sharing): "exec_steless10 (run-deploy)_9" prio=4 tid=0x02ef9000 nid=0xd2c runnable [0x0e3ee000..0x0e3efa14] java.lang.Thread.State: RUNNABLE at java.io.ObjectOutputStream$HandleTable.lookup(ObjectOutputStream.java:2255) at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1198) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1387) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1338) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1146) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326) at com.sun.enterprise.admin.jmx.remote.comm.ServletConnection.sendReq(ServletConnection.java:188) at com.sun.enterprise.admin.jmx.remote.comm.ServletConnection.send(ServletConnection.java:151) at com.sun.enterprise.admin.jmx.remote.comm.MBeanServerMessageConductor.invoke(MBeanServerMessageConductor.java:60) at com.sun.enterprise.admin.jmx.remote.internal.RemoteMBeanServerConnection.invoke(RemoteMBeanServerConnection.java:408) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:287) at com.sun.appserv.management.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:647) at com.sun.appserv.management.client.handler.AMXProxyHandler._invoke(AMXProxyHandler.java:1091) at com.sun.appserv.management.client.handler.AMXProxyHandler.invoke(AMXProxyHandler.java:1002) at $Proxy12.waitAMXReady(Unknown Source) at com.sun.enterprise.deployapi.SunDeploymentManager.getRootProxy(SunDeploymentManager.java:1627) at com.sun.enterprise.deployapi.SunDeploymentManager.getModulesOnATarget(SunDeploymentManager.java:397) at com.sun.enterprise.deployapi.SunDeploymentManager.getModules(SunDeploymentManager.java:366) at com.sun.enterprise.deployapi.SunDeploymentManager.getAvailableModules(SunDeploymentManager.java:315) at org.netbeans.modules.j2ee.sun.ide.dm.SunDeploymentManager.getAvailableModules(SunDeploymentManager.java:554) - locked <0x1a1e6af0> (a org.netbeans.modules.j2ee.sun.ide.dm.SunDeploymentManager) at org.netbeans.modules.j2ee.deployment.impl.TargetServer.getAvailableTMIDsMap(TargetServer.java:298) at org.netbeans.modules.j2ee.deployment.impl.TargetServer.processLastTargetModules(TargetServer.java:334) at org.netbeans.modules.j2ee.deployment.impl.TargetServer.init(TargetServer.java:108) at org.netbeans.modules.j2ee.deployment.impl.TargetServer.deploy(TargetServer.java:445) at org.netbeans.modules.j2ee.deployment.devmodules.api.Deployment.deploy(Deployment.java:106) at org.netbeans.modules.j2ee.ant.Deploy.execute(Deploy.java:82) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275) at org.apache.tools.ant.Task.perform(Task.java:364) at org.apache.tools.ant.Target.execute(Target.java:341) at org.apache.tools.ant.Target.performTasks(Target.java:369) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216) at org.apache.tools.ant.Project.executeTarget(Project.java:1185) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40) at org.apache.tools.ant.Project.executeTargets(Project.java:1068) at org.apache.tools.ant.module.bridge.impl.BridgeImpl.run(BridgeImpl.java:240) at org.apache.tools.ant.module.run.TargetExecutor.run(TargetExecutor.java:293) - locked <0x1bcbdf38> (a org.apache.tools.ant.module.run.TargetExecutor) at org.netbeans.core.execution.RunClassThread.run(RunClassThread.java:131) "Thread-50" prio=4 tid=0x0c1ee800 nid=0x8ec waiting on condition [0x0e39f000..0x0e39fa94] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.netbeans.core.execution.DefaultSysProcess$1.run(DefaultSysProcess .java:105) at java.lang.Thread.run(Thread.java:619) "Thread-47" daemon prio=4 tid=0x0c573c00 nid=0x3e0 waiting on condition [0x0e65f 000..0x0e65fc14] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.netbeans.modules.j2ee.sun.ide.j2ee.LogViewerSupport.run(LogViewer Support.java:206) "Inactive RequestProcessor thread [Was:Default RequestProcessor/org.netbeans.mod ules.j2ee.sun.bridge.apis.AppserverMgmtContainerNode$ContainerChildren$1]" daemo n prio=2 tid=0x0c38bc00 nid=0xb98 in Object.wait() [0x0e2ff000..0x0e2ffb94] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x1b561ef8> (a java.lang.Object) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java :885) - locked <0x1b561ef8> (a java.lang.Object) "Default RequestProcessor" daemon prio=2 tid=0x038b4800 nid=0x4d8 in Object.wait () [0x0c02f000..0x0c02fd14] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x1bcbdfe8> (a org.netbeans.core.execution.ExecutorTaskImp l) at java.lang.Object.wait(Object.java:485) at org.openide.util.Task.waitFinished(Task.java:108) - locked <0x1bcbdfe8> (a org.netbeans.core.execution.ExecutorTaskImpl) at org.apache.tools.ant.module.run.TargetExecutor$WrapperRunnable.run(Ta rgetExecutor.java:199) at org.openide.util.Task.run(Task.java:222) at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:499) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java :932) "Thread-16" daemon prio=6 tid=0x0379e800 nid=0x544 runnable [0x0ce1f000..0x0ce1f c94] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:199) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x1a25ff78> (a java.io.BufferedInputStream) at java.io.FilterInputStream.read(FilterInputStream.java:90) at org.netbeans.modules.j2ee.sun.ide.j2ee.StartSunServer$StreamFlusher.r un(StartSunServer.java:1063) "Thread-15" daemon prio=6 tid=0x0390c400 nid=0x318 runnable [0x0cdcf000..0x0cdcf d14] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:177) at org.netbeans.modules.j2ee.sun.ide.j2ee.StartSunServer$StreamFlusher.r un(StartSunServer.java:1063) "Thread-10" daemon prio=6 tid=0x035b9400 nid=0xda4 runnable [0x0ccdf000..0x0ccdf b14] java.lang.Thread.State: RUNNABLE at java.lang.ProcessImpl.waitFor(Native Method) at org.netbeans.modules.derby.ExecSupport$1.run(ExecSupport.java:81) "Thread-9" daemon prio=6 tid=0x0371b800 nid=0x9f8 waiting on condition [0x0cc8f0 00..0x0cc8fb94] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.netbeans.modules.derby.ExecSupport$OutputCopier.read(ExecSupport. java:161) at org.netbeans.modules.derby.ExecSupport$OutputCopier.run(ExecSupport.j ava:136) "Thread-8" daemon prio=6 tid=0x035b9000 nid=0xb68 waiting on condition [0x0cc3f0 00..0x0cc3fc94] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.netbeans.modules.derby.ExecSupport$OutputCopier.read(ExecSupport. java:161) at org.netbeans.modules.derby.ExecSupport$OutputCopier.run(ExecSupport.j ava:136) "Thread-7" daemon prio=6 tid=0x02ebd800 nid=0xeb0 waiting on condition [0x0cbef0 00..0x0cbefc14] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.netbeans.modules.derby.ExecSupport$OutputCopier.read(ExecSupport. java:161) at org.netbeans.modules.derby.ExecSupport$OutputCopier.run(ExecSupport.j ava:136) "Inactive RequestProcessor thread [Was:TimedSoftReference/org.openide.util.Timed SoftReference]" daemon prio=2 tid=0x03869400 nid=0xb64 in Object.wait() [0x0347f 000..0x0347fa14] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x1a20a380> (a java.lang.Object) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java :885) - locked <0x1a20a380> (a java.lang.Object) "MDR event dispatcher" daemon prio=2 tid=0x02b23000 nid=0xc48 in Object.wait() [ 0x0342f000..0x0342fa94] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x19aaf3a8> (a java.util.LinkedList) at java.lang.Object.wait(Object.java:485) at org.netbeans.mdr.util.EventNotifier$EventsDelivery.run(EventNotifier. java:263) - locked <0x19aaf3a8> (a java.util.LinkedList) at java.lang.Thread.run(Thread.java:619) "DestroyJavaVM" prio=6 tid=0x00297800 nid=0xf48 waiting on condition [0x00000000 ..0x0090fd4c] java.lang.Thread.State: RUNNABLE "AWT-EventQueue-1" prio=6 tid=0x02f5d000 nid=0xf08 runnable [0x0bcff000..0x0bcffd14] java.lang.Thread.State: RUNNABLE at java.io.ObjectOutputStream$HandleTable.lookup(ObjectOutputStream.java:2255) at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1198) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1387) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1338) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1146) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326) at com.sun.enterprise.admin.jmx.remote.comm.ServletConnection.sendReq(ServletConnection.java:188) at com.sun.enterprise.admin.jmx.remote.comm.ServletConnection.send(ServletConnection.java:151) at com.sun.enterprise.admin.jmx.remote.comm.MBeanServerMessageConductor.invoke(MBeanServerMessageConductor.java:60) at com.sun.enterprise.admin.jmx.remote.internal.RemoteMBeanServerConnection.invoke(RemoteMBeanServerConnection.java:408) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:287) at com.sun.appserv.management.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:647) at com.sun.appserv.management.client.handler.AMXProxyHandler._invoke(AMXProxyHandler.java:1091) at com.sun.appserv.management.client.handler.AMXProxyHandler.invoke(AMXProxyHandler.java:1002) at $Proxy12.waitAMXReady(Unknown Source) at com.sun.appserv.management.client.ProxyFactory.getDomainRoot(ProxyFactory.java:325) at com.sun.appserv.management.client.AppserverConnectionSource.getDomainRoot(AppserverConnectionSource.java:503) at org.netbeans.modules.j2ee.sun.ide.controllers.ControllerUtil.createAppserverConnFromDeploymentMgr(ControllerUtil.java:160) at org.netbeans.modules.j2ee.sun.ide.controllers.ControllerUtil.getAppserverMgmtControllerFromDeployMgr(ControllerUtil.java:132) at org.netbeans.modules.j2ee.sun.ide.runtime.nodes.DomainRootNode.refresh(DomainRootNode.java:72) at org.netbeans.modules.j2ee.sun.ide.runtime.nodes.DomainRootNode$1.run(DomainRootNode.java:64) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209) at java.awt.EventQueue.dispatchEvent(EventQueue.java:597) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160) at java.awt.EventDispatchThread.run(EventDispatchThread.java:121) "TimerQueue" daemon prio=6 tid=0x03067000 nid=0xfc in Object.wait() [0x0359f000. .0x0359fb14] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x1943e598> (a javax.swing.TimerQueue) at javax.swing.TimerQueue.run(TimerQueue.java:236) - locked <0x1943e598> (a javax.swing.TimerQueue) at java.lang.Thread.run(Thread.java:619) "AWT-Windows" daemon prio=6 tid=0x02be8000 nid=0xf60 runnable [0x033af000..0x033 afc94] java.lang.Thread.State: RUNNABLE at sun.awt.windows.WToolkit.eventLoop(Native Method) at sun.awt.windows.WToolkit.run(WToolkit.java:290) at java.lang.Thread.run(Thread.java:619) "AWT-Shutdown" prio=6 tid=0x02ffec00 nid=0xc00 in Object.wait() [0x0335f000..0x0 335fd14] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x193d8780> (a java.lang.Object) at java.lang.Object.wait(Object.java:485) at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:259) - locked <0x193d8780> (a java.lang.Object) at java.lang.Thread.run(Thread.java:619) "Java2D Disposer" daemon prio=10 tid=0x03012800 nid=0xedc in Object.wait() [0x03 30f000..0x0330fd94] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x193d8818> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x193d8818> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at sun.java2d.Disposer.run(Disposer.java:125) at java.lang.Thread.run(Thread.java:619) "Timer-0" daemon prio=6 tid=0x02be9400 nid=0xf94 in Object.wait() [0x0325f000..0 x0325fa14] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x19372ca8> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x19372ca8> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462) "CLI Requests Server" daemon prio=6 tid=0x02bcfc00 nid=0xb34 runnable [0x031bf00 0..0x031bfb14] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) - locked <0x19372df0> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:450) at java.net.ServerSocket.accept(ServerSocket.java:421) at org.netbeans.CLIHandler$Server.run(CLIHandler.java:867) "Active Reference Queue Daemon" daemon prio=2 tid=0x02bcc400 nid=0x934 in Object .wait() [0x030cf000..0x030cfb94] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x19372f80> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x19372f80> (a java.lang.ref.ReferenceQueue$Lock) at org.openide.util.Utilities$ActiveQueue.run(Utilities.java:3002) at java.lang.Thread.run(Thread.java:619) "Low Memory Detector" daemon prio=6 tid=0x02b1f000 nid=0xdd4 runnable [0x0000000 0..0x00000000] java.lang.Thread.State: RUNNABLE "CompilerThread0" daemon prio=10 tid=0x02b1a400 nid=0xa64 waiting on condition [ 0x00000000..0x02dcf998] java.lang.Thread.State: RUNNABLE "Attach Listener" daemon prio=10 tid=0x02b19000 nid=0xbd8 runnable [0x00000000.. 0x00000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x02b18400 nid=0xffc waiting on condition [0x00000000..0x00000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=8 tid=0x02b13800 nid=0xf38 in Object.wait() [0x02cdf000. .0x02cdfa94] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x193731d8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x193731d8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) "Reference Handler" daemon prio=10 tid=0x02b0f400 nid=0x24c in Object.wait() [0x 02c8f000..0x02c8fb14] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x19372c18> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked <0x19372c18> (a java.lang.ref.Reference$Lock) "VM Thread" prio=10 tid=0x02b06000 nid=0x884 runnable "VM Periodic Task Thread" prio=10 tid=0x02b39c00 nid=0xb58 waiting on condition JNI global references: 2483 Heap def new generation total 3648K, used 2511K [0x18990000, 0x18d80000, 0x1936000 0) eden space 3264K, 67% used [0x18990000, 0x18bb5e58, 0x18cc0000) from space 384K, 81% used [0x18cc0000, 0x18d0de60, 0x18d20000) to space 384K, 0% used [0x18d20000, 0x18d20000, 0x18d80000) tenured generation total 47596K, used 42738K [0x19360000, 0x1c1db000, 0x20990 000) the space 47596K, 89% used [0x19360000, 0x1bd1ca48, 0x1bd1cc00, 0x1c1db000) compacting perm gen total 46080K, used 46075K [0x20990000, 0x23690000, 0x2a990 000) the space 46080K, 99% used [0x20990000, 0x2368ee18, 0x2368f000, 0x23690000) ro space 8192K, 66% used [0x2a990000, 0x2aed8dc8, 0x2aed8e00, 0x2b190000) rw space 12288K, 52% used [0x2b190000, 0x2b7d5d70, 0x2b7d5e00, 0x2bd90000) --------------080800050709060907030101--