dev@glassfish.java.net

Re: Sending JMS messages between systems doesn't work any more

From: Kim Haase <Camilla.Haase_at_Sun.COM>
Date: Fri, 04 Dec 2009 15:31:37 -0500

After much experimentation, I've determined that the ability to send JMS
messages between systems disappeared between b68 and b71. It works fine
with promoted b68. Promoted b69 and promoted b70 both had the problem
that if you tried to run a JMS client, you would get a message in the
server log about jts starting up, and everything would hang -- so JMS
didn't really work at all. At promoted b71, the ConnectException
messages start to appear.

I'll file an issue.

Kim

On 12/03/09 13:04, Kim Haase wrote:
> Many thanks, Nigel. It looks as if the problem is with GlassFish, not MQ
> -- see below.
>
> On 12/03/09 12:05, Nigel Deakin wrote:
>> Kim,
>>
>> Kim Haase wrote:
>>> On 12/03/09 06:20, Nigel Deakin wrote:
>>>> Kim,
>>>>
>>>> Kim Haase wrote:
>>>>> JMS applications that send messages from one system to another
>>>>> stopped working for me in b73 and also fail in b74. They worked in
>>>>> earlier builds -- I think I last tested them around b69.
>>>>>
>>>>> Several examples in the Java EE Tutorial are based on sending JMS
>>>>> messages between systems, so this is an issue for docs.
>>>>>
>>>>> For a number of releases, the way to enable the sending of messages
>>>>> to a remote system has been to create a JMS connection factory
>>>>> resource whose AddressList property is set to the name of the
>>>>> remote system. (The connector connection pool, therefore, has its
>>>>> AddressList property set to this value.)
>>>>>
>>>>> Now, when I try to send messages to the remote system, I get the
>>>>> following error over and over:
>>>>>
>>>>> WARNING: [C4003]: Error occurred on connection creation
>>>>> [jdench:7676]. - cause: java.net.ConnectException: Connection
>>>>> refused: connect
>>>>> Dec 2, 2009 11:31:55 AM
>>>>> com.sun.messaging.jmq.jmsclient.ExceptionHandler
>>>>> throwConnectionException
>>>>
>>>> This means it is trying to connect to a MQ broker running on machine
>>>> jdench on port 7676, and it is getting no response, perhaps because
>>>> the broker isn't running or it is using a different port.
>>>>
>>>> Can you connect to it using other tools on your windows machine,
>>>> such as imqcmd (which is in the mq/bin directory of a Glassfish V3
>>>> installation)?
>>>>
>>>> For example this command lists the destinations on the broker:
>>>> imqcmd list dst -b jdench:7676
>>>> (it will prompt for user and password: typically admin/admin)
>>>
>>> Thanks, Nigel. The broker is running on jdench at port 7676 -- these
>>> are the server log messages on jdench, which look the way they always
>>> do:
>>>
>>> [#|2009-12-03T11:12:59.008-0500|INFO|glassfishv3.0|javax.enterprise.system.core.transaction.com.sun.jts.CosTransactions|_ThreadID=17;_ThreadName=Thread-1;|JTS5014:
>>> Recoverable JTS instance, serverId = [3700]|#]
>>>
>>> [#|2009-12-03T11:13:03.745-0500|INFO|glassfishv3.0|com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter|_ThreadID=32;_ThreadName=Thread-1;|JMS
>>> Service Connection URL is :mq://localhost:7676/|#]
>>>
>>> [#|2009-12-03T11:13:03.878-0500|INFO|glassfishv3.0|org.hibernate.validator.engine.resolver.DefaultTraversableResolver|_ThreadID=32;_ThreadName=Thread-1;|Instantiated
>>> an instance of
>>> org.hibernate.validator.engine.resolver.JPATraversableResolver.|#]
>>>
>>> [#|2009-12-03T11:13:04.170-0500|INFO|glassfishv3.0|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=32;_ThreadName=Thread-1;|MQJMSRA_RA1101:
>>> SJSMQ JMS Resource Adapter starting: DIRECT|#]
>>>
>>> [#|2009-12-03T11:13:14.151-0500|INFO|glassfishv3.0|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=32;_ThreadName=Thread-1;|MQJMSRA_RA1101:
>>> SJSMQ JMSRA Started:DIRECT|#]
>>>
>>> [#|2009-12-03T11:13:20.619-0500|INFO|glassfishv3.0|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=30;_ThreadName=Thread-1;|MQJMSRA_MF1101:
>>> setAddressList:NOT setting default value=localhost|#]
>>>
>>> [#|2009-12-03T11:13:20.633-0500|INFO|glassfishv3.0|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=30;_ThreadName=Thread-1;|MQJMSRA_MF1101:
>>> setPassword:NOT setting default value|#]
>>>
>>> [#|2009-12-03T11:13:20.634-0500|INFO|glassfishv3.0|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=30;_ThreadName=Thread-1;|MQJMSRA_MF1101:
>>> setUserName:NOT setting default value=guest|#]
>>
>> Yes, this suggests the embedded broker has started. You could also
>> look at the broker log in
>> domains\domain1\imq\instances\imqbroker\log\log.txt (or similar) to
>> confirm this.
>>
>>>
>>> I ran the imqcmd on jdench and after some experimentation I found
>>> that the correct username and password were admin/admin.
>>
>> I did mention the default username and password in my previous email :-)
>
> Sorry to have spaced out!
>
>>
>>>
>>> jdench 52 =>imqcmd list dst
>>> Username: admin
>>> Password:
>>> Listing all the destinations on the broker specified by:
>>>
>>> -------------------------
>>> Host Primary Port
>>> -------------------------
>>> localhost 7676
>>>
>>> ---------------------------------------------------------------------------
>>>
>>> Name Type State Producers Consumers Msgs
>>> Total Count UnAck
>>> Avg Size
>>> ---------------------------------------------------------------------------
>>>
>>> PhysicalQueue Queue RUNNING 0 1 0 0 0.0
>>> mq.sys.dmq Queue RUNNING 0 0 0 0 0.0
>>>
>>> Successfully listed destinations.
>>
>> Yes, so the broker is definitely working.
>>
>>>
>>> jdench 53 =>pwd
>>> /home/chaase/glassfish-v3-b74/mq/bin
>>>
>>> However, the imqcmd on the Windows system fails:
>>>
>>> C:\Program Files\glassfish-v3-b74\mq\bin>imqcmd list dst -b jdench:7676
>>> Username: admin
>>> Password: admin
>>> Listing all the destinations on the broker specified by:
>>>
>>> ----------------------
>>> Host Primary Port
>>> ----------------------
>>> jdench 7676
>>>
>>> Dec 3, 2009 11:19:40 AM
>>> com.sun.messaging.jmq.jmsclient.ExceptionHandler
>>> throwConnectionException
>>> WARNING: [C4003]: Error occurred on connection creation
>>> [jdench:7676]. - cause:
>>> java.net.ConnectException: Connection refused: connect
>>> Error while connecting to the broker on host 'jdench' and port '7676'.
>>> com.sun.messaging.jms.JMSException: [C4003]: Error occurred on
>>> connection creation [jdench:7676]. - cause:
>>> java.net.ConnectException: Connection refused: connect
>>> Please verify that there is a broker running on the specified host
>>> and port or use the '-b' option to specify the correct broker host
>>> and port.
>>>
>>> Listing destinations failed.
>>
>> This eliminates your client code as the cause of the problem. It has
>> found the host but received no reply on the port.
>>
>>>
>>> I notice that on Windows the password shows up in clear text, whereas
>>> on Solaris it doesn't appear. Hard to believe that makes a
>>> difference, though.
>>>
>>> And I can still ping jdench, save files to it, etc. from my Windows
>>> system.
>>
>> This is sounding like a network issue. Do you have a hardware or
>> software firewall?
>
> Not that I know of.
>
>>
>> You could try shutting down the Glassfish instance and instead
>> starting a standalone MQ broker by running the imqbrokerd command
>> directly, and seeing if you can "imqcmd list dst" it from the PC.
>
> Without GlassFish, it works! I started the broker on jdench, and it runs
> fine (had to specify -javahome to keep it from picking up some jdk 5
> somewhere):
>
> jdench 59 =>./imqbrokerd -javahome /home/chaase/jdk1.6.0_17
> [03/Dec/2009:12:59:11 EST]
> ================================================================================
>
> Open Message Queue 4.4
> Sun Microsystems, Inc.
> Version: 4.4 Update 1 (Build 7-b)
> Compile: Wed Nov 18 23:21:05 PST 2009
>
> Copyright (c) 2009 Sun Microsystems, Inc. All rights reserved. Use is
> subject to license terms.
> ================================================================================
>
> Java Runtime: 1.6.0_17 Sun Microsystems Inc. /home/chaase/jdk1.6.0_17/jre
> [03/Dec/2009:12:59:12 EST] IMQ_HOME=/home/chaase/glassfish-v3-b74/mq
> [03/Dec/2009:12:59:12 EST]
> IMQ_VARHOME=/home/chaase/glassfish-v3-b74/glassfish/domains/domain1/imq
> [03/Dec/2009:12:59:12 EST] SunOS 5.10 sparc jdench (1 cpu) chaase
> [03/Dec/2009:12:59:12 EST] Max file descriptors: 65536 (65536)
> [03/Dec/2009:12:59:12 EST] Java Heap Size: max=190080k, current=190080k
> [03/Dec/2009:12:59:12 EST] Arguments: -javahome /home/chaase/jdk1.6.0_17
> [03/Dec/2009:12:59:12 EST] [B1060]: Loading persistent data...
> [03/Dec/2009:12:59:12 EST] Using built-in file-based persistent store:
> /home/chaase/glassfish-v3-b74/glassfish/domains/domain1/imq/instances/imqbroker/
>
> [03/Dec/2009:12:59:13 EST] [B1039]: Broker "imqbroker_at_jdench:7676" ready.
>
> And I did an imqcmd list dst and got the system queue, as expected.
>
> Then on Windows, imqcmd is able to list the destinations for the jdench
> broker too:
>
> C:\Program Files\glassfish-v3-b74\mq\bin>imqcmd list dst -b jdench:7676
> Username: admin
> Password: admin
> Listing all the destinations on the broker specified by:
>
> ----------------------
> Host Primary Port
> ----------------------
> jdench 7676
>
> --------------------------------------------------------------------------------------------
>
> Name Type State Producers Consumers Msgs
>
> Total Wildcard Total Wildcard Count
> Remote UnAck Avg Size
> --------------------------------------------------------------------------------------------
>
> mq.sys.dmq Queue RUNNING 0 - 0 - 0 0
> 0 0.0
>
> Successfully listed destinations.
>
> So it's not MQ that's causing the problem but GlassFish apparently. What
> now, I wonder?
>
> Thanks again!!
>
> Kim
>
>>
>> Nigel
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>