Hello everyone,
i have an issue, which is bugging me for some time now.
I am running a recent glassfish 2.1.1 (just updated it to see if the problem is gone, which it is not) on a 64bit Debian Linux.
The glassfish serves 6 instances of the same application using a different database and virtual host for each instance. The server is restarted each night for sanity reasons (i also experienced some slow downs in the past if i did not). The problem is, that sometimes for some reason not all applications start up properly, they just answer with a "503 - The requested service () is not currently available." message. I could not find anything usefull in the server logs. It is completely random if and which instance does not start up, sometimes even more than one. I noticed that a manual asadmin start/stop-domain always seemed to fix everything whereas doing it in a cron job needs several retries but this could also have been just luck.
I wrote a small script run by cron which checks the server after each restart and re-restarts it until all applications respond properly. Ill attach its output from this morning at the bottom of this post. Interestingly, some applications/hosts crash more often than others but this, again, could be just coincidence.
I'll also attach a logfile from one of the restarts to this post. There are some exceptions in it but they seem not problem-related to me.
A current jvm report is also attached.
This is the script's output (i replaced the domain names):
www.host1.de gave a bad 503 respsonse!
Restarting server...
Domain sams stopped.
Starting Domain sams, please wait.
Default Log location is /home/glassfish/glassfish/domains/sams/logs/server.log.
Redirecting output to /home/glassfish/glassfish/domains/sams/logs/server.log
Domain sams is ready to receive client requests. Additional services are being started in background.
Domain [sams] is running [Sun GlassFish Enterprise Server v2.1.1 ((v2.1 Patch06)(9.1_02 Patch12)) (build b31g-fcs)] with its configuration and logs at: [/home/glassfish/glassfish/domains].
Admin Console is available at [
http://localhost:8048].
Use the same port [8048] for "asadmin" commands.
User web applications are available at these URLs:
[
http://localhost:8080 https://localhost:8081 ].
Following web-contexts are available:
[/web1 /__wstx-services / / / / / ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://sams-master:8086/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8081 8048 8037 8038 8039 8086 ].
Domain does not support application server clusters and other standalone instances.
www.host2.de gave a bad 503 respsonse!
www.host1.de gave a bad 503 respsonse!
Restarting server...
Domain sams stopped.
Starting Domain sams, please wait.
Default Log location is /home/glassfish/glassfish/domains/sams/logs/server.log.
Redirecting output to /home/glassfish/glassfish/domains/sams/logs/server.log
Domain sams is ready to receive client requests. Additional services are being started in background.
Domain [sams] is running [Sun GlassFish Enterprise Server v2.1.1 ((v2.1 Patch06)(9.1_02 Patch12)) (build b31g-fcs)] with its configuration and logs at: [/home/glassfish/glassfish/domains].
Admin Console is available at [
http://localhost:8048].
Use the same port [8048] for "asadmin" commands.
User web applications are available at these URLs:
[
http://localhost:8080 https://localhost:8081 ].
Following web-contexts are available:
[/web1 /__wstx-services / / / / / ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://sams-master:8086/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8081 8048 8037 8038 8039 8086 ].
Domain does not support application server clusters and other standalone instances.
www.host2.de gave a bad 503 respsonse!
Restarting server...
Domain sams stopped.
Starting Domain sams, please wait.
Default Log location is /home/glassfish/glassfish/domains/sams/logs/server.log.
Redirecting output to /home/glassfish/glassfish/domains/sams/logs/server.log
Domain sams is ready to receive client requests. Additional services are being started in background.
Domain [sams] is running [Sun GlassFish Enterprise Server v2.1.1 ((v2.1 Patch06)(9.1_02 Patch12)) (build b31g-fcs)] with its configuration and logs at: [/home/glassfish/glassfish/domains].
Admin Console is available at [
http://localhost:8048].
Use the same port [8048] for "asadmin" commands.
User web applications are available at these URLs:
[
http://localhost:8080 https://localhost:8081 ].
Following web-contexts are available:
[/web1 /__wstx-services / / / / / ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://sams-master:8086/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8081 8048 8037 8038 8039 8086 ].
Domain does not support application server clusters and other standalone instances.
I'd appreciate any help with this!
Thanks in advance, Robert
[Message sent by forum member 'roben_rb']
http://forums.java.net/jive/thread.jspa?messageID=472121