dev@glassfish.java.net

Re: Instance instance xxx is not reachable]

From: Sreenivas Munnangi <Sreenivas.Munnangi_at_Sun.COM>
Date: Tue, 26 Sep 2006 13:08:09 -0700

Nandini Ektare wrote:

> Sreenivas Munnangi wrote:
>
>> Sreenivas Munnangi wrote:
>>
>>> Nandini Ektare wrote:
>>>
>>>> Jothir
>>>>
>>>> Not sure if my following suggestion reached you. (I remember there
>>>> was a thread of discussion and I assume Sreeni has already made you
>>>> aware of various suggestions in that thread.)
>>>> The only known environment change that happened between b13 and b15
>>>> was that of JDK minor version. So you might want to try with older
>>>> version.
>>>>
>>>> "One more important thing... JDK build version changed between the
>>>> two builds (switched to 1.5.0_08 on 8/29/06 so builds after that
>>>> are using this b15, b16..)... I have not found any specific fixes
>>>> related to java.io.ObjectOutputStream (see stack trace below)...
>>>> but it will be good to check against the old JDK version to rule
>>>> out this variable. JDK performance fixes had to be picked up... So
>>>> this change could be a regression due to that.
>>>>
>>>> [#|2006-09-12T12:36:04.046+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise.system.tools.admin|_ThreadID=52;_ThreadName=event-handler-2;_RequestID=3ea01d2b-1c34-429c-b32d-d4d8cc24dbb3;|Instance:
>>>> [instance3] not reachable
>>>> com.sun.enterprise.admin.servermgmt.InstanceException:
>>>> java.io.UTFDataFormatException
>>>> at
>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.mangleInvokeException(InstanceProxy.java:150)
>>>>
>>>> at
>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.invoke(InstanceProxy.java:176)
>>>>
>>>> at $Proxy23.forwardEvent(Unknown Source)
>>>> at
>>>> com.sun.enterprise.ee.admin.event.EndPointHandler.run(EndPointHandler.java:88)
>>>>
>>>> at java.lang.Thread.run(Thread.java:595)
>>>> Caused by: java.io.UTFDataFormatException
>>>> at
>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1969)
>>>>
>>>> at
>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1812)
>>>>
>>>> at java.io.ObjectOutputStream.writeUTF(ObjectOutputStream.java:816)
>>>> at
>>>> com.sun.enterprise.config.serverbeans.Domain.writeObject(Domain.java:606)
>>>>
>>>> at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>
>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>> at
>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>
>>>> at
>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>
>>>> at
>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>
>>>
>>>
>>>
>>> Nandini, I don't think the sole cause for "Instance instance1 is not
>>> reachable" is java.io.UTFDataFormatException. Because we have seen
>>> this error even without java.io.UTFDataFormatException. Seems we
>>> need some way of reproducing this consistently to better understand
>>> the issue.
>>
> So you are saying we have tried with earlier versions?

No, I am not ruling out that possibility.

It is possible that we have 2 issues here
 - one is java.io.UTFDataFormatException along with Instance instance1
is not reachable error and the other is
- Instance instance1 is not reachable error

The error related to Instance instance1 is not reachable error was
reported in the forum earlier
http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882

> Reason to double check on this is that two of us reported that we
> cannot reproduce the bug on b16 but both were using an older JDK
> version. There were no changes related to this from admin and
> deployment side(deployment team confirmed this) between these builds.
> So in all probability it is to do with the setup/environment.
>
> About consistently reproducing this, I though Jothir reported
> deployment always fails for a cluster.

No, if we make sure that the instances are running and deploy then it
works fine.

>
> Thanks
> Nandini
>
>>>
>> Also this seems an old issue
>> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>>
>>>>
>>>> Thanks
>>>> Nandini
>>>>
>>>>
>>>> jothir ganesan wrote:
>>>>
>>>>> Yes Kedar,
>>>>>
>>>>> I am stuck with this issue for 3 builds starting from build 16.
>>>>> It is reproducible on Linux too. Please see bug#6467623.
>>>>> Could you please explain what the EE QL is?
>>>>>
>>>>> Thanks
>>>>> Jothir
>>>>>
>>>>> Kedar Mhaswade wrote:
>>>>>
>>>>>> Also, can you try the same thing on an older build? (b16, b17 for
>>>>>> instance)?
>>>>>>
>>>>>> Thanks,
>>>>>> Kedar
>>>>>>
>>>>>> Kedar Mhaswade wrote:
>>>>>>
>>>>>>> Hi Jothir,
>>>>>>>
>>>>>>> I am seeing these errors while running the EE QL on Solaris.
>>>>>>> Let us investigate ...
>>>>>>> The tinderbox appears to be all green, however.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Kedar
>>>>>>>
>>>>>>> jothir ganesan wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Could anyone please let me know the possible reasons for
>>>>>>>> "EEADM0068: Instance instance xxx is not reachable" ? The
>>>>>>>> exception shows up when trying to deploy an app on the cluster
>>>>>>>> and the app fails to get deployed...
>>>>>>>>
>>>>>>>> I have the following setup:
>>>>>>>>
>>>>>>>> 3 nodeagents on 3 different machines with each nodeagent having
>>>>>>>> one instance.
>>>>>>>> Domain is started and the cluster,created on DAS, with the
>>>>>>>> three instances - running
>>>>>>>> Nodeagents- running
>>>>>>>>
>>>>>>>> Other configuration did:
>>>>>>>> - hadb nodes running and ha-cluster is configured.
>>>>>>>> - Access log buffer size ->
>>>>>>>> cluster1-config.http-service.virtual-server.server.property.accessLogBufferSize
>>>>>>>> set to 10 for cluster config
>>>>>>>> - cluster1-config.http-service.keep-alive.timeout-in-seconds
>>>>>>>> set to 0.
>>>>>>>> - Some jdbc resources and connection pools created.
>>>>>>>>
>>>>>>>>
>>>>>>>> I have my nodeagents and instances up and running.
>>>>>>>> Command start-cluster executed successfully.
>>>>>>>> bash-3.00# asadmin list-instances --passwordfile passwordfile
>>>>>>>> instance1 running
>>>>>>>> instance2 running
>>>>>>>> instance3 running
>>>>>>>> Command list-instances executed successfully.
>>>>>>>> bash-3.00# /space/91/appserver/bin/asadmin deploy --host
>>>>>>>> localhost --port 4848 --user admin --passwordfile passwordfile
>>>>>>>> --target cluster1
>>>>>>>> /space/91/sift/agent/file-repository/70/Apps/SampleApplications/Clusterjsp1/clu
>>>>>>>> sterjsp1.war
>>>>>>>> Command deploy executed successfully.
>>>>>>>> bash-3.00# asadmin list-node-agents --passwordfile passwordfile
>>>>>>>> sqe-agent1 running
>>>>>>>> sqe-agent2 running
>>>>>>>> sqe-agent3 running
>>>>>>>> Command list-node-agents executed successfully.
>>>>>>>>
>>>>>>>> But still when I try to deploy the app, I could see :
>>>>>>>> [#|2006-09-26T16:03:20.722+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>> .system.tools.admin|_ThreadID=198;_ThreadName=event-handler-0;instance1;_Request
>>>>>>>> ID=4c5551da-d066-451e-a7a9-ee36c5a8ab47;|EEADM0068: Instance
>>>>>>>> instance1 is not reachable.|#]
>>>>>>>>
>>>>>>>> [#|2006-09-26T16:03:20.739+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>> .system.tools.admin|_ThreadID=199;_ThreadName=event-handler-1;instance2;_Request
>>>>>>>> ID=3c7c1df3-a47f-44a9-a418-e506bab400bf;|EEADM0068: Instance
>>>>>>>> instance2 is not reachable.|#]
>>>>>>>>
>>>>>>>> [#|2006-09-26T16:03:20.759+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>> .system.tools.admin|_ThreadID=200;_ThreadName=event-handler-2;instance3;_Request
>>>>>>>> ID=da9f94fd-c35e-417d-ba3a-41c0cecc91c5;|EEADM0068: Instance
>>>>>>>> instance3 is not reachable.|#]
>>>>>>>>
>>>>>>>> What else do I need to check?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Jothir
>>>>>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>