Hi Sreeni,
Could you please follow the steps given in the attached doc and check if
the issue is reproducible? I tried it on a 1 machine setup and it is
reproducible. The problem showed up(was consistent) after configuring
hadb....
I was able to reproduce it consistently. If you want to cross check,
refer to machine eas-v240-18. (AS_INSTALL_DIR -
/space/jothir/91/appserver). I have left the setup for evaluation.
Please let me know if you have any comments. Also, let me know if I can
redispatch the bug with this info.
Thanks
Jothir
Sreenivas Munnangi wrote:
> Nandini Ektare wrote:
>
>> Sreenivas Munnangi wrote:
>>
>>> Sreenivas Munnangi wrote:
>>>
>>>> Nandini Ektare wrote:
>>>>
>>>>> Jothir
>>>>>
>>>>> Not sure if my following suggestion reached you. (I remember there
>>>>> was a thread of discussion and I assume Sreeni has already made
>>>>> you aware of various suggestions in that thread.)
>>>>> The only known environment change that happened between b13 and
>>>>> b15 was that of JDK minor version. So you might want to try with
>>>>> older version.
>>>>>
>>>>> "One more important thing... JDK build version changed between the
>>>>> two builds (switched to 1.5.0_08 on 8/29/06 so builds after that
>>>>> are using this b15, b16..)... I have not found any specific fixes
>>>>> related to java.io.ObjectOutputStream (see stack trace below)...
>>>>> but it will be good to check against the old JDK version to rule
>>>>> out this variable. JDK performance fixes had to be picked up...
>>>>> So this change could be a regression due to that.
>>>>>
>>>>> [#|2006-09-12T12:36:04.046+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise.system.tools.admin|_ThreadID=52;_ThreadName=event-handler-2;_RequestID=3ea01d2b-1c34-429c-b32d-d4d8cc24dbb3;|Instance:
>>>>> [instance3] not reachable
>>>>> com.sun.enterprise.admin.servermgmt.InstanceException:
>>>>> java.io.UTFDataFormatException
>>>>> at
>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.mangleInvokeException(InstanceProxy.java:150)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.invoke(InstanceProxy.java:176)
>>>>>
>>>>> at $Proxy23.forwardEvent(Unknown Source)
>>>>> at
>>>>> com.sun.enterprise.ee.admin.event.EndPointHandler.run(EndPointHandler.java:88)
>>>>>
>>>>> at java.lang.Thread.run(Thread.java:595)
>>>>> Caused by: java.io.UTFDataFormatException
>>>>> at
>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1969)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1812)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeUTF(ObjectOutputStream.java:816)
>>>>> at
>>>>> com.sun.enterprise.config.serverbeans.Domain.writeObject(Domain.java:606)
>>>>>
>>>>> at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>
>>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>>> at
>>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Nandini, I don't think the sole cause for "Instance instance1 is
>>>> not reachable" is java.io.UTFDataFormatException. Because we have
>>>> seen this error even without java.io.UTFDataFormatException. Seems
>>>> we need some way of reproducing this consistently to better
>>>> understand the issue.
>>>
>>>
>> So you are saying we have tried with earlier versions?
>
>
> No, I am not ruling out that possibility.
>
> It is possible that we have 2 issues here
> - one is java.io.UTFDataFormatException along with Instance instance1
> is not reachable error and the other is
> - Instance instance1 is not reachable error
>
> The error related to Instance instance1 is not reachable error was
> reported in the forum earlier
> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>
>> Reason to double check on this is that two of us reported that we
>> cannot reproduce the bug on b16 but both were using an older JDK
>> version. There were no changes related to this from admin and
>> deployment side(deployment team confirmed this) between these builds.
>> So in all probability it is to do with the setup/environment.
>>
>> About consistently reproducing this, I though Jothir reported
>> deployment always fails for a cluster.
>
>
> No, if we make sure that the instances are running and deploy then it
> works fine.
>
>>
>> Thanks
>> Nandini
>>
>>>>
>>> Also this seems an old issue
>>> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>>>
>>>>>
>>>>> Thanks
>>>>> Nandini
>>>>>
>>>>>
>>>>> jothir ganesan wrote:
>>>>>
>>>>>> Yes Kedar,
>>>>>>
>>>>>> I am stuck with this issue for 3 builds starting from build 16.
>>>>>> It is reproducible on Linux too. Please see bug#6467623.
>>>>>> Could you please explain what the EE QL is?
>>>>>>
>>>>>> Thanks
>>>>>> Jothir
>>>>>>
>>>>>> Kedar Mhaswade wrote:
>>>>>>
>>>>>>> Also, can you try the same thing on an older build? (b16, b17
>>>>>>> for instance)?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Kedar
>>>>>>>
>>>>>>> Kedar Mhaswade wrote:
>>>>>>>
>>>>>>>> Hi Jothir,
>>>>>>>>
>>>>>>>> I am seeing these errors while running the EE QL on Solaris.
>>>>>>>> Let us investigate ...
>>>>>>>> The tinderbox appears to be all green, however.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Kedar
>>>>>>>>
>>>>>>>> jothir ganesan wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Could anyone please let me know the possible reasons for
>>>>>>>>> "EEADM0068: Instance instance xxx is not reachable" ? The
>>>>>>>>> exception shows up when trying to deploy an app on the cluster
>>>>>>>>> and the app fails to get deployed...
>>>>>>>>>
>>>>>>>>> I have the following setup:
>>>>>>>>>
>>>>>>>>> 3 nodeagents on 3 different machines with each nodeagent
>>>>>>>>> having one instance.
>>>>>>>>> Domain is started and the cluster,created on DAS, with the
>>>>>>>>> three instances - running
>>>>>>>>> Nodeagents- running
>>>>>>>>>
>>>>>>>>> Other configuration did:
>>>>>>>>> - hadb nodes running and ha-cluster is configured.
>>>>>>>>> - Access log buffer size ->
>>>>>>>>> cluster1-config.http-service.virtual-server.server.property.accessLogBufferSize
>>>>>>>>> set to 10 for cluster config
>>>>>>>>> - cluster1-config.http-service.keep-alive.timeout-in-seconds
>>>>>>>>> set to 0.
>>>>>>>>> - Some jdbc resources and connection pools created.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I have my nodeagents and instances up and running.
>>>>>>>>> Command start-cluster executed successfully.
>>>>>>>>> bash-3.00# asadmin list-instances --passwordfile passwordfile
>>>>>>>>> instance1 running
>>>>>>>>> instance2 running
>>>>>>>>> instance3 running
>>>>>>>>> Command list-instances executed successfully.
>>>>>>>>> bash-3.00# /space/91/appserver/bin/asadmin deploy --host
>>>>>>>>> localhost --port 4848 --user admin --passwordfile passwordfile
>>>>>>>>> --target cluster1
>>>>>>>>> /space/91/sift/agent/file-repository/70/Apps/SampleApplications/Clusterjsp1/clu
>>>>>>>>> sterjsp1.war
>>>>>>>>> Command deploy executed successfully.
>>>>>>>>> bash-3.00# asadmin list-node-agents --passwordfile passwordfile
>>>>>>>>> sqe-agent1 running
>>>>>>>>> sqe-agent2 running
>>>>>>>>> sqe-agent3 running
>>>>>>>>> Command list-node-agents executed successfully.
>>>>>>>>>
>>>>>>>>> But still when I try to deploy the app, I could see :
>>>>>>>>> [#|2006-09-26T16:03:20.722+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=198;_ThreadName=event-handler-0;instance1;_Request
>>>>>>>>> ID=4c5551da-d066-451e-a7a9-ee36c5a8ab47;|EEADM0068: Instance
>>>>>>>>> instance1 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> [#|2006-09-26T16:03:20.739+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=199;_ThreadName=event-handler-1;instance2;_Request
>>>>>>>>> ID=3c7c1df3-a47f-44a9-a418-e506bab400bf;|EEADM0068: Instance
>>>>>>>>> instance2 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> [#|2006-09-26T16:03:20.759+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=200;_ThreadName=event-handler-2;instance3;_Request
>>>>>>>>> ID=da9f94fd-c35e-417d-ba3a-41c0cecc91c5;|EEADM0068: Instance
>>>>>>>>> instance3 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> What else do I need to check?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Jothir
>>>>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>
root_at_eas-v240-18 # export AS_ADMIN_PORT=4848
root_at_eas-v240-18 # export AS_ADMIN_USER=admin
root_at_eas-v240-18 # export PATH=/space/jothir/91/appserver/bin:$PATH
root_at_eas-v240-18 # export AS_ADMIN_HOST=eas-v240-18
root_at_eas-v240-18 # asadmin create-domain --adminport 4848 --adminuser admin sqe-domain
root_at_eas-v240-18 # asadmin start-domain --passwordfile passwordfile sqe-domain
root_at_eas-v240-18 # asadmin create-cluster --passwordfile passwordfile cluster1
root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent1
root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent2
root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent3
root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent1 --systemproperties IIOP_LISTEN ER_PORT=3335:HTTP_LISTENER_PORT=1111 instance1
root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent2 --systemproperties IIOP_LISTEN ER_PORT=3336:HTTP_LISTENER_PORT=1112 instance2
root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent3 --systemproperties IIOP_LISTEN ER_PORT=3337:HTTP_LISTENER_PORT=1113 instance3
root_at_eas-v240-18 # asadmin start-node-agent --passwordfile passwordfile sqe-agent1
root_at_eas-v240-18 # asadmin start-node-agent --passwordfile passwordfile sqe-agent2
root_at_eas-v240-18 # /space/jothir/91/appserver/hadb/4.5.0-9/bin/ma -j /space/jothir/91/appserver/jdk /space/jothir/91/appserver/hadb/4.5.0-9/bin/ma.cfg &
root_at_eas-v240-18 # asadmin configure-ha-cluster --passwordfile passwordfile --hosts eas-v240-18,eas-v240-18 cluster1
root_at_eas-v240-18 # asadmin list-instances --passwordfile passwordfile
root_at_eas-v240-18 # asadmin stop-cluster --passwordfile passwordfile cluster1
root_at_eas-v240-18 # asadmin start-cluster --passwordfile passwordfile cluster1
(The below given commands are the jdbc configuration commands that were run using a script)
asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource bookstorePool
asadmin create-jdbc-resource --user admin --passwordfile passwordfile --connectionpoolid bookstorePool --target cluster1 jdbc/BooksDB
asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:user=solsparc:password=solsparc:SID=siftsol --target cluster1 --restype javax.sql.XADataSource oracleconnectionpool
asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/oracle
asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/usertxn
asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/bmp-savingsaccount
asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/oracle --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/oracle
asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource productPool
asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid productPool --target cluster1 jdbc/product
asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/product --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/product
asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource rosterPool
asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid rosterPool --target cluster1 jdbc/roster
asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/roster --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/roster
root_at_eas-v240-18 # asadmin deploy --target cluster1 --passwordfile passwordfile clusterjsp.war