dev@glassfish.java.net

Too many threads created in GlassFish DAS

From: glassfish.lu <glassfish.lu_at_hotmail.com>
Date: Tue, 8 Mar 2011 20:14:39 +0800

Hi, All:

I find the number of the active threads in GlassFish DAS will increase greatly as new server instance is created.
such as:

For each server instance, a {ClientNotifForwarder} thread will be created.

"ClientNotifForwarder-20" daemon prio=1 tid=0x0000000050b108e0 nid=0x6f27 runnable [0x00000000560af000..0x00000000560afb10]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
- locked <0x00002ab056932048> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:241)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:189)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:126)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl_Stub.fetchNotifications(Unknown Source)
at javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1285)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:559)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:441)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:422)
at com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:88)

For each server instance, a {JMX server connection timeout} thread will be created.

"JMX server connection timeout 463" daemon prio=1 tid=0x0000000050963310 nid=0x6f1e in Object.wait() [0x0000000055baa000..0x0000000055baac90]
at java.lang.Object.wait(Native Method)
- waiting on <0x00002ab056201dc0> (a [I)
at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
- locked <0x00002ab056201dc0> (a [I)
at java.lang.Thread.run(Thread.java:595)

For each server instance, a {RMI RenewClean } thread will be created.

"RMI RenewClean-[192.168.2.151:33057]" daemon prio=1 tid=0x0000000051370380 nid=0x6e94 in Object.wait() [0x0000000054796000..0x0000000054796c90]
at java.lang.Object.wait(Native Method)
- waiting on <0x00002ab03c1d8730> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
- locked <0x00002ab03c1d8730> (a java.lang.ref.ReferenceQueue$Lock)
at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:501)
at java.lang.Thread.run(Thread.java:595)

And some {RMI ConnectionExpiration} thread will also be created.

"RMI ConnectionExpiration-[192.168.2.151:41022]" daemon prio=1 tid=0x00000000514ecbb0 nid=0x73b6 waiting on condition [0x000000004d5cd000..0x000000004d5cdb10]
at java.lang.Thread.sleep(Native Method)
at sun.rmi.transport.tcp.TCPChannel$Reaper.run(TCPChannel.java:446)
at java.lang.Thread.run(Thread.java:595)

As the number of instances managed by DAS grow, the thread created by JVM will grow accordingly .
Sometimes the number will exceed the thread limit of JVM.

So my question now is how to avoid this situation, how to reduce these threads.
Is there any system property configuration or configuration in domain.xml to workaroud this.

Thanks guys in advance.

Best Regards.