I have a web application that responds to http requests and has a few threads running in the background. The background threads continue to run long after the application has stopped responding to http requests. It takes awhile for this to happen, hours or days. From the logs it appears that http requests are simply no longer reaching the application. Just before the application stops responding a bunch of errors appeared in my logs like the one below. The appearance of these errors corresponds closely to the problem appearing. What is the app server doing? Could this be related to my problem? Is there something in the app server configuration I could be trying to resolve this?
I've also including the output of a netstat command on the server.
SEVERE|24|2008-03-30 07:56:02:081|com.sun.enterprise.deployment.autodeploy.AutoDeployControllerImpl$AutoDeployTask.run(): enterprise.deployment.backend.autoDeploymentFailure
WARNING|25|2008-03-30 07:56:03:312|com.sun.enterprise.server.logging.SystemOutandErrHandler$LoggingByteArrayOutputStream.flush():
java.lang.NullPointerException
at com.sun.jbi.management.system.AutoAdminTask.pollAutoDirectory(AutoAdminTask.java:1044)
at com.sun.jbi.management.system.AutoAdminTask.performAutoInstall(AutoAdminTask.java:330)
at com.sun.jbi.management.system.AutoAdminTask.performAutoFunctions(AutoAdminTask.java:288)
at com.sun.jbi.management.system.AdminService.heartBeat(AdminService.java:994)
at com.sun.jbi.management.system.AdminService.handleNotification(AdminService.java:227)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor$ListenerWrapper.handleNotification(DefaultMBeanServerInterceptor.java:1732)
at javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:257)
at javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:322)
at javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:307)
at javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:229)
at javax.management.timer.Timer.sendNotification(Timer.java:1234)
at javax.management.timer.Timer.notifyAlarmClock(Timer.java:1203)
at javax.management.timer.TimerAlarmClock.run(Timer.java:1286)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
output from netstat
root_at_localhost:~# netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 ip-10-251-91-146.ec:www pool-141-155-18-1:50163 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www pool-72-79-189-18:50464 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www william.goodenoug:52560 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 70-91-85-201-washi:2181 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 75.141.45.150:2245 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 78.19.100.142:1223 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www cpe-65-24-208-235:34060 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www hlndoacache1.mt.g:47161 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www ip68-227-184-232.:42330 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-24-61-133-241.hs:2307 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-69-136-56-36.hs:52420 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www hlndoacache1.mt.g:33444 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-24-18-203-234.hs:2481 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www astound-66-234-198:3177 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www e179158028.adsl.al:1047 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 208.188.205.155:2224 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 4143_acl1.gw.smart:3953 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www gv-dialup-4-176.i:34233 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www rrcs-71-43-151-20:29198 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-69-250-197-188.:64098 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www dynamic.70-169-17:61328 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www d51538AB6.access.:55832 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www customer1.aao.org:15675 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www dD5763197.access.:33535 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www cpe-68-172-94-133.:2702 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-67-170-56-158.hs:2571 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-71-234-76-54.hsd:3387 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www pool-71-120-238-17:1625 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 203-214-86-111.dyn:1119 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www C-61-68-178-154.b:51021 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 89.1.51.183.dynami:1337 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-76-23-65-65.hsd1:1684 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www ip51ce538b.speed.:59095 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www c-67-180-217-82.hs:2124 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 207.61.123.57:52504 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 213.63.176.71:50039 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 317104228.telebar:57344 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www user-142g26k.cabl:33780 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www 79-117-201-26.rds:54345 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www seifert.plus.com:1852 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www hlndoacache1.mt.g:26773 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www seifert.plus.com:1852 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www hlndoacache1.mt.g:26773 SYN_RECV
tcp 0 0 ip-10-251-91-146.ec:www delta9.student.utw:3416 SYN_RECV
tcp 201 0 ip-10-251-91-146.ec:www 78.13.11.82:1125 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 66-240-64-131.atl:50441 CLOSE_WAIT
tcp 0 372 ip-10-251-91-146.ec:ssh 75-146-62-233-Was:60286 ESTABLISHED
tcp 308 0 ip-10-251-91-146.ec:www pool-71-127-255-18:1716 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 81.8.159.190:59711 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-69-138-22-65.hs:51911 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www mail.ghkco.com:32525 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www ool-18b8b4e5.dyn.:62961 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www william.goodenoug:47465 CLOSE_WAIT
tcp 0 0 ip-10-251-91-146.:57420 72.21.203.132:https TIME_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-68-38-134-69.hs:62187 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 64-194-89-2.wcg.n:50923 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www CPE001a92cb1dad-CM:1318 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-24-12-224-131.h:39429 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 79-113-232-223.rds:1839 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www adsl-75-11-145-20:56331 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 201.216-66-87.ads:56850 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-71-205-112-45.hs:3806 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 24.137.36.69:1703 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www cac94-10-88-170-23:1222 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 201-130-248-45-cab:4847 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 149-40-249.oke2-b:55549 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www adsl-ull-64-37.49-:4074 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www cpe-24-242-46-175:25297 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 54.Red-88-14-113.d:4072 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www adsl-69-153-101-18:2020 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www pool-162-84-202-1:50315 CLOSE_WAIT
tcp 0 0 ip-10-251-91-146.ec:ssh 75-146-62-233-Was:54741 ESTABLISHED
tcp 308 0 ip-10-251-91-146.ec:www 125-98-235-201.fib:1893 CLOSE_WAIT
tcp 307 0 ip-10-251-91-146.ec:www c-24-19-239-124.h:63767 ESTABLISHED
tcp 308 0 ip-10-251-91-146.ec:www 0x5551c0cd.adsl.c:11721 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www adsl-75-7-199-178:10230 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www cpe-71-65-82-223.w:4678 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-65-34-132-137.h:63399 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-98-203-126-171.h:1938 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www p54B1CF08.dip.t-d:62084 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www adsl-65-67-115-236:1066 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www xdsl-83-150-81-20:46581 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 75.141.45.150:3814 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 71-83-94-173.dhcp:63095 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 108-82-252-216-sta:2810 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 78.162.10.192:3158 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 137.80-203-49.nex:50509 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www adsl-dynamic-pool:50447 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 78-69-202-128-no1:63530 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-75-67-240-124.hs:4067 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www host-84-221-79-221:1634 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www host-84-221-79-221:1634 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www adsl-70-232-97-51.:4284 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-24-8-197-67.hsd:37884 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-24-99-22-144.hsd:2717 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 90.210.19.21:1206 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www bas1-toronto12-108:1202 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www ip565cca8c.direct-:1662 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 32.212-66-87.adsl:62419 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www a50192.upc-a.chell:1083 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 15.caba27.xdsl.nau:1592 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 209.33.252.4:60556 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www s5592bcc4.adsl.wa:56496 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www ip68-101-174-48.s:33721 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-24-19-203-45.hsd:4278 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www chmuglab1b.chem.ps:2884 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www 90-229-208-68-no1:52277 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 24-217-109-209.dhc:1071 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-69-138-201-21.hs:2585 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www sac91-1-89-85-92-1:1355 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www c-76-112-191-55.hs:3400 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www host103-209-dynami:4361 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-24-16-87-21.hsd1:2538 CLOSE_WAIT
tcp 340 0 ip-10-251-91-146.ec:www pool-96-232-215-1:50064 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 82-40-120-220.cabl:4649 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www cpe-71-72-57-127.n:3237 CLOSE_WAIT
tcp 200 0 ip-10-251-91-146.ec:www d51537131.access.:49379 ESTABLISHED
tcp 342 0 ip-10-251-91-146.ec:www ti0117a380-1363.b:56375 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www aolclient-24-33-80:1733 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www ip5659bc18.speed.:61483 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www pool-72-72-17-115:33279 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www pool-72-79-114-48.:3541 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www dsl-202-138-212-20:2896 CLOSE_WAIT
tcp 200 0 ip-10-251-91-146.ec:www cc191256-b.assen1.:3375 ESTABLISHED
tcp 201 0 ip-10-251-91-146.ec:www cc546786-e.gorre1.:1324 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www 72-55-206-206.mam:51455 CLOSE_WAIT
tcp 308 0 ip-10-251-91-146.ec:www host86-155-126-17:58011 CLOSE_WAIT
tcp 201 0 ip-10-251-91-146.ec:www c-69-254-200-132.:32892 CLOSE_WAIT
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ] DGRAM 648 @/com/ubuntu/upstart
unix 5 [ ] DGRAM 6476 /dev/log
unix 2 [ ] DGRAM 844 @/org/kernel/udev/udevd
unix 2 [ ] DGRAM 34002
unix 2 [ ] DGRAM 6632
unix 2 [ ] DGRAM 6586
unix 2 [ ] STREAM CONNECTED 6185
[Message sent by forum member 'kevinmacdonald' (kevinmacdonald)]
http://forums.java.net/jive/thread.jspa?messageID=266930