Gajanan.Kulkarni_at_Sun.COM wrote:
> ccing glassfish team and corba team
> Sandra Dellafiora wrote On 04/25/07 23:35,:
>
>
>> Application Server Version (asadmin version):
>> Version = Sun Java System Application Server Platform Edition 9.0_01
>>
>> Solaris Version (uname -a):
>> SunOS sheep 5.10 Generic_118833-36 sun4v sparc SUNW,Sun-Fire-T200
>>
>> Customer is writing a web application against an existing set of EJBs.
>> These are currently in production used by a standalone CORBA/IIOP client.
>> They want to migrate the systems to SJSAS 9.0 (and possibly 9.1).
>>
>> They initially used SJSAS PE 9.0 without problems. They put the app
>> on Sun Fire T200 systems to do performance and load testing, and installed
>> SJSAS 9.0_01. Performance with this version of the application server
>> was terrible. After some investigation, they determined the problem
>> was not the hardware or application, but the size and fragmentation of
>> the IIOP messages used by the web app to communicate with the EJBs.
>>
>> The web application is deployed as a single war file and the EJBs are
>> packaged as a separate ear file, mostly for ease of development/deployment.
>> On the 9.0 version of the server they ran the standalone client with
>> option:
>>
>> -Dcom.sun.CORBA.ORBDebug=giop
>>
>> This outputs each IIOP message, The messages were of a reasonable length
>> compared to past versions of the application server. But with the same
>> option on 9.0_01, the messages seemed to include the servers entire
>> classpath in each message and were extremely fragmented. The standalone
>>
(the entire classpath part is really significant here)
>> client (which has been in production for many years) would fail to finish
>> receiving messages with an "OutOfMemoryException: java heap", even after
>> giving it 1GB+ heap.
>>
>> They have not been able to find any defects in the glassfish system that
>> describes this. I couldn't either. Is this "expected behavior", or a
>> known, or unknown, problem?
>>
>>
This sounds like a known problem that was fixed sometime around b22
in GlassFish v2/AS 9.1. Here is my description from an email I sent last
October:
--------------------------------------------------------------------
Part of the problem is due to the relative cost of the
RMIClassLoader.getClassAnnotation method
when called inside the app server as compared to our standalone ORB
tests. In a standalone test,
this method returns immediately with an empty string, because we test
without any special class loaders.
In the app server, this method computes a very large string (200
characters perhaps?) that is
determined by the classpath of the app server's class loader, and the
computation is not cached.
We are modifying the ORB code to cache the result getClassAnnotation.
This is compounded by an error in the CDROutputStream_1_0, where
codebaseCache is incorrectly
implemented with a CacheTable (which is object identity based) instead
of a simple HashMap
(which is equality based). This caused each instance of the large
codebase string to be
treated as a different string, so it was marshaled each time it was
repeated, and so that same
large codebase (much larger than the data in your case) was marshaled
for EACH data base
record returned in your query.
---------------------------------------------------------------------
I can't find an issue or bug for this either. We may have just fixed it
as part of a large
amount of ORB optimization work we've been doing. What may have happened
(this part
is just speculation) is that a change in the app server's ClassLoaders
at some point caused the
ORB problems to manifest themselves. One customer was returning around
300 Java objects
(all of the same type) in an ArrayList (a typical result of a database
query). This operation
was running probably 10X slower than it should, due to all of the extra
data in each message.
You should see the server's classpath only once per message
(unfortunately it does need
to appear once because of the getClassAnnotation method mentioned above,
but at
least this is all handled efficiently now).
Please note that this problem has NOT been patched in AS 9.0_xx. It
would be fairly simple
to patch this if there is sufficient customer demand. It may be helpful
to try GF v2 Beta 2
and see if this solves the problem.
Ken.