Sreenivas Munnangi wrote:
> jothir ganesan wrote:
>
>> Hi Sreeni,
>>
>> Could you please follow the steps given in the attached doc and check
>> if the issue is reproducible? I tried it on a 1 machine setup and it
>> is reproducible. The problem showed up(was consistent) after
>> configuring hadb....
>>
>> I was able to reproduce it consistently. If you want to cross check,
>> refer to machine eas-v240-18. (AS_INSTALL_DIR -
>> /space/jothir/91/appserver). I have left the setup for evaluation.
>
> So you think configuration of hadb is the root cause ?
I suspect it to be.
>
> I tried the reproducible steps provided by you, all the steps except 2
> related to hadb and the deployment was fine.
>
> Now when I was trying to configure-hadb, I get some exceptions. I'll
> send you a separate email with the details.
Can you try the passwordfile option instead of exporting the password in
configure-ha-cluster command?
asadmin> configure-ha-cluster --hosts bnr,bnr cluster1 : Could you
please let me know how you specified the password for this command or
were you waiting for password prompt?
I tried the command and I got the prompt for password.
asadmin configure-ha-cluster --hosts eas-v240-18,eas-v240-18 cluster1
Please enter the admin password>
I have replied with the steps that I did for configure-ha-cluster.
Please let me know if you have any issues.
Thanks
Jothir
>>
>> Please let me know if you have any comments. Also, let me know if I
>> can redispatch the bug with this info.
>>
>> Thanks
>> Jothir
>>
>>
>> Sreenivas Munnangi wrote:
>>
>>> Nandini Ektare wrote:
>>>
>>>> Sreenivas Munnangi wrote:
>>>>
>>>>> Sreenivas Munnangi wrote:
>>>>>
>>>>>> Nandini Ektare wrote:
>>>>>>
>>>>>>> Jothir
>>>>>>>
>>>>>>> Not sure if my following suggestion reached you. (I remember
>>>>>>> there was a thread of discussion and I assume Sreeni has already
>>>>>>> made you aware of various suggestions in that thread.)
>>>>>>> The only known environment change that happened between b13 and
>>>>>>> b15 was that of JDK minor version. So you might want to try with
>>>>>>> older version.
>>>>>>>
>>>>>>> "One more important thing... JDK build version changed between
>>>>>>> the two builds (switched to 1.5.0_08 on 8/29/06 so builds after
>>>>>>> that are using this b15, b16..)... I have not found any specific
>>>>>>> fixes related to java.io.ObjectOutputStream (see stack trace
>>>>>>> below)... but it will be good to check against the old JDK
>>>>>>> version to rule out this variable. JDK performance fixes had to
>>>>>>> be picked up... So this change could be a regression due to that.
>>>>>>>
>>>>>>> [#|2006-09-12T12:36:04.046+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise.system.tools.admin|_ThreadID=52;_ThreadName=event-handler-2;_RequestID=3ea01d2b-1c34-429c-b32d-d4d8cc24dbb3;|Instance:
>>>>>>> [instance3] not reachable
>>>>>>> com.sun.enterprise.admin.servermgmt.InstanceException:
>>>>>>> java.io.UTFDataFormatException
>>>>>>> at
>>>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.mangleInvokeException(InstanceProxy.java:150)
>>>>>>>
>>>>>>> at
>>>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.invoke(InstanceProxy.java:176)
>>>>>>>
>>>>>>> at $Proxy23.forwardEvent(Unknown Source)
>>>>>>> at
>>>>>>> com.sun.enterprise.ee.admin.event.EndPointHandler.run(EndPointHandler.java:88)
>>>>>>>
>>>>>>> at java.lang.Thread.run(Thread.java:595)
>>>>>>> Caused by: java.io.UTFDataFormatException
>>>>>>> at
>>>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1969)
>>>>>>>
>>>>>>> at
>>>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1812)
>>>>>>>
>>>>>>> at
>>>>>>> java.io.ObjectOutputStream.writeUTF(ObjectOutputStream.java:816)
>>>>>>> at
>>>>>>> com.sun.enterprise.config.serverbeans.Domain.writeObject(Domain.java:606)
>>>>>>>
>>>>>>> at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>>>>> at
>>>>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>>>>
>>>>>>> at
>>>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>>>>
>>>>>>> at
>>>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Nandini, I don't think the sole cause for "Instance instance1 is
>>>>>> not reachable" is java.io.UTFDataFormatException. Because we have
>>>>>> seen this error even without java.io.UTFDataFormatException.
>>>>>> Seems we need some way of reproducing this consistently to better
>>>>>> understand the issue.
>>>>>
>>>>>
>>>>>
>>>> So you are saying we have tried with earlier versions?
>>>
>>>
>>>
>>> No, I am not ruling out that possibility.
>>>
>>> It is possible that we have 2 issues here
>>> - one is java.io.UTFDataFormatException along with Instance
>>> instance1 is not reachable error and the other is
>>> - Instance instance1 is not reachable error
>>>
>>> The error related to Instance instance1 is not reachable error was
>>> reported in the forum earlier
>>> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>>>
>>>> Reason to double check on this is that two of us reported that we
>>>> cannot reproduce the bug on b16 but both were using an older JDK
>>>> version. There were no changes related to this from admin and
>>>> deployment side(deployment team confirmed this) between these
>>>> builds. So in all probability it is to do with the setup/environment.
>>>>
>>>> About consistently reproducing this, I though Jothir reported
>>>> deployment always fails for a cluster.
>>>
>>>
>>>
>>> No, if we make sure that the instances are running and deploy then
>>> it works fine.
>>>
>>>>
>>>> Thanks
>>>> Nandini
>>>>
>>>>>>
>>>>> Also this seems an old issue
>>>>> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> Nandini
>>>>>>>
>>>>>>>
>>>>>>> jothir ganesan wrote:
>>>>>>>
>>>>>>>> Yes Kedar,
>>>>>>>>
>>>>>>>> I am stuck with this issue for 3 builds starting from build
>>>>>>>> 16. It is reproducible on Linux too. Please see bug#6467623.
>>>>>>>> Could you please explain what the EE QL is?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Jothir
>>>>>>>>
>>>>>>>> Kedar Mhaswade wrote:
>>>>>>>>
>>>>>>>>> Also, can you try the same thing on an older build? (b16, b17
>>>>>>>>> for instance)?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Kedar
>>>>>>>>>
>>>>>>>>> Kedar Mhaswade wrote:
>>>>>>>>>
>>>>>>>>>> Hi Jothir,
>>>>>>>>>>
>>>>>>>>>> I am seeing these errors while running the EE QL on Solaris.
>>>>>>>>>> Let us investigate ...
>>>>>>>>>> The tinderbox appears to be all green, however.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Kedar
>>>>>>>>>>
>>>>>>>>>> jothir ganesan wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> Could anyone please let me know the possible reasons for
>>>>>>>>>>> "EEADM0068: Instance instance xxx is not reachable" ? The
>>>>>>>>>>> exception shows up when trying to deploy an app on the
>>>>>>>>>>> cluster and the app fails to get deployed...
>>>>>>>>>>>
>>>>>>>>>>> I have the following setup:
>>>>>>>>>>>
>>>>>>>>>>> 3 nodeagents on 3 different machines with each nodeagent
>>>>>>>>>>> having one instance.
>>>>>>>>>>> Domain is started and the cluster,created on DAS, with the
>>>>>>>>>>> three instances - running
>>>>>>>>>>> Nodeagents- running
>>>>>>>>>>>
>>>>>>>>>>> Other configuration did:
>>>>>>>>>>> - hadb nodes running and ha-cluster is configured.
>>>>>>>>>>> - Access log buffer size ->
>>>>>>>>>>> cluster1-config.http-service.virtual-server.server.property.accessLogBufferSize
>>>>>>>>>>> set to 10 for cluster config
>>>>>>>>>>> - cluster1-config.http-service.keep-alive.timeout-in-seconds
>>>>>>>>>>> set to 0.
>>>>>>>>>>> - Some jdbc resources and connection pools created.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I have my nodeagents and instances up and running.
>>>>>>>>>>> Command start-cluster executed successfully.
>>>>>>>>>>> bash-3.00# asadmin list-instances --passwordfile passwordfile
>>>>>>>>>>> instance1 running
>>>>>>>>>>> instance2 running
>>>>>>>>>>> instance3 running
>>>>>>>>>>> Command list-instances executed successfully.
>>>>>>>>>>> bash-3.00# /space/91/appserver/bin/asadmin deploy --host
>>>>>>>>>>> localhost --port 4848 --user admin --passwordfile
>>>>>>>>>>> passwordfile --target cluster1
>>>>>>>>>>> /space/91/sift/agent/file-repository/70/Apps/SampleApplications/Clusterjsp1/clu
>>>>>>>>>>> sterjsp1.war
>>>>>>>>>>> Command deploy executed successfully.
>>>>>>>>>>> bash-3.00# asadmin list-node-agents --passwordfile passwordfile
>>>>>>>>>>> sqe-agent1 running
>>>>>>>>>>> sqe-agent2 running
>>>>>>>>>>> sqe-agent3 running
>>>>>>>>>>> Command list-node-agents executed successfully.
>>>>>>>>>>>
>>>>>>>>>>> But still when I try to deploy the app, I could see :
>>>>>>>>>>> [#|2006-09-26T16:03:20.722+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>>>> .system.tools.admin|_ThreadID=198;_ThreadName=event-handler-0;instance1;_Request
>>>>>>>>>>> ID=4c5551da-d066-451e-a7a9-ee36c5a8ab47;|EEADM0068: Instance
>>>>>>>>>>> instance1 is not reachable.|#]
>>>>>>>>>>>
>>>>>>>>>>> [#|2006-09-26T16:03:20.739+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>>>> .system.tools.admin|_ThreadID=199;_ThreadName=event-handler-1;instance2;_Request
>>>>>>>>>>> ID=3c7c1df3-a47f-44a9-a418-e506bab400bf;|EEADM0068: Instance
>>>>>>>>>>> instance2 is not reachable.|#]
>>>>>>>>>>>
>>>>>>>>>>> [#|2006-09-26T16:03:20.759+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>>>> .system.tools.admin|_ThreadID=200;_ThreadName=event-handler-2;instance3;_Request
>>>>>>>>>>> ID=da9f94fd-c35e-417d-ba3a-41c0cecc91c5;|EEADM0068: Instance
>>>>>>>>>>> instance3 is not reachable.|#]
>>>>>>>>>>>
>>>>>>>>>>> What else do I need to check?
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> Jothir
>>>>>>>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>------------------------------------------------------------------------
>>
>>
>>root_at_eas-v240-18 # export AS_ADMIN_PORT=4848
>>root_at_eas-v240-18 # export AS_ADMIN_USER=admin
>>root_at_eas-v240-18 # export PATH=/space/jothir/91/appserver/bin:$PATH
>>root_at_eas-v240-18 # export AS_ADMIN_HOST=eas-v240-18
>>
>>root_at_eas-v240-18 # asadmin create-domain --adminport 4848 --adminuser admin sqe-domain
>>
>>root_at_eas-v240-18 # asadmin start-domain --passwordfile passwordfile sqe-domain
>>
>>root_at_eas-v240-18 # asadmin create-cluster --passwordfile passwordfile cluster1
>>
>>root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent1
>>
>>root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent2
>>
>>root_at_eas-v240-18 # asadmin create-node-agent --passwordfile passwordfile sqe-agent3
>>
>>
>>root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent1 --systemproperties IIOP_LISTEN ER_PORT=3335:HTTP_LISTENER_PORT=1111 instance1
>>
>>root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent2 --systemproperties IIOP_LISTEN ER_PORT=3336:HTTP_LISTENER_PORT=1112 instance2
>>
>>root_at_eas-v240-18 # asadmin create-instance --passwordfile passwordfile --cluster cluster1 --nodeagent sqe-agent3 --systemproperties IIOP_LISTEN ER_PORT=3337:HTTP_LISTENER_PORT=1113 instance3
>>
>>root_at_eas-v240-18 # asadmin start-node-agent --passwordfile passwordfile sqe-agent1
>>
>>root_at_eas-v240-18 # asadmin start-node-agent --passwordfile passwordfile sqe-agent2
>>
>>root_at_eas-v240-18 # /space/jothir/91/appserver/hadb/4.5.0-9/bin/ma -j /space/jothir/91/appserver/jdk /space/jothir/91/appserver/hadb/4.5.0-9/bin/ma.cfg &
>>
>>root_at_eas-v240-18 # asadmin configure-ha-cluster --passwordfile passwordfile --hosts eas-v240-18,eas-v240-18 cluster1
>>
>>root_at_eas-v240-18 # asadmin list-instances --passwordfile passwordfile
>>
>>root_at_eas-v240-18 # asadmin stop-cluster --passwordfile passwordfile cluster1
>>
>>root_at_eas-v240-18 # asadmin start-cluster --passwordfile passwordfile cluster1
>>
>>(The below given commands are the jdbc configuration commands that were run using a script)
>>asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource bookstorePool
>>
>>asadmin create-jdbc-resource --user admin --passwordfile passwordfile --connectionpoolid bookstorePool --target cluster1 jdbc/BooksDB
>>
>>asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:user=solsparc:password=solsparc:SID=siftsol --target cluster1 --restype javax.sql.XADataSource oracleconnectionpool
>>
>>asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/oracle
>>
>>asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/usertxn
>>
>>asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid oracleconnectionpool --target cluster1 jdbc/bmp-savingsaccount
>>
>>asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/oracle --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/oracle
>>
>>asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource productPool
>>
>>asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid productPool --target cluster1 jdbc/product
>>
>>asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/product --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/product
>>
>>asadmin create-jdbc-connection-pool --passwordfile passwordfile --datasourceclassname com.sun.sql.jdbcx.oracle.OracleDataSource --property User=solsparc:Password=solsparc:serverName=eas115.india.sun.com:portNumber=1521:SID=siftsol --target cluster1 --restype javax.sql.XADataSource rosterPool
>>
>>asadmin create-jdbc-resource --passwordfile passwordfile --connectionpoolid rosterPool --target cluster1 jdbc/roster
>>
>>asadmin create-persistence-resource --passwordfile passwordfile --jdbcjndiname jdbc/roster --factoryclass com.sun.jdo.spi.persistence.support.sqlstore.impl.PersistenceManagerFactoryImpl --target cluster1 jdo/roster
>>
>>root_at_eas-v240-18 # asadmin deploy --target cluster1 --passwordfile passwordfile clusterjsp.war
>>
>>
>>
>>------------------------------------------------------------------------
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>>
>