quality@glassfish.java.net

webapps, hessian services, gfv3 -> severe fuzzy problems?

From: Kristian Rink <rink_at_planconnect.de>
Date: Tue, 23 Mar 2010 10:07:06 +0100

Folks;


in one of our setups, we have a bunch of web applications (Spring based)
communicating with each other via Hessian remoting, using a dedicated
http-listener listening on port 9898 for the sole purpose of enabling
this communication, while client users access the applications using the
"regular" port 8080 listener. We use these two listeners because, using
just one, we used to have severe problems with threads locking each
other in the past.

It worked pretty well this way for years on gfv2, but after migrating to
gfv3, it seems not to work reliable anymore, as, under load, peculiar
errors start to show up, mainly involving access errors, processes
taking ages to come to an end, ... . Basically, it's not stable at all
so I will roll back to gfv2 today and examine the situation on v3...

Browsing the log file, here's what I found, ordered by frequency of
appearance:


- "Invalid chunk header", extremely often with Hessian being involved
and load on the system increasing a little (load test, in example):

[#|2010-03-23T09:55:27.725+0100|WARNING|glassfishv3.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=3
9;_ThreadName=http-backend-thread-pool-9898-(7);|StandardWrapperValve[remoting]:
PWC1406: Servlet.service() for servlet remoti
ng threw exception
java.io.IOException: Invalid chunk header
         at
com.sun.grizzly.tcp.http11.filters.ChunkedInputFilter.doRead(ChunkedInputFilter.java:169)

         at
com.sun.grizzly.tcp.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:731)

         at com.sun.grizzly.tcp.Request.doRead(Request.java:490)



- "Unexpected end of file from server", more or less often:

[#|2010-03-23T10:02:46.105+0100|WARNING|glassfishv3.0|com.caucho.hessian.server.HessianSkeleton|_ThreadID=149;_ThreadName=Thre
ad-1;|com.caucho.hessian.client.HessianConnectionException: 500:
java.net.SocketException: Unexpected end of file from server

com.caucho.hessian.client.HessianConnectionException: 500:
java.net.SocketException: Unexpected end of file from server
         at
com.caucho.hessian.client.HessianProxy.invoke(HessianProxy.java:197)
         at $Proxy180.getMetaFiles(Unknown Source)



- Exception "500" for a given URL, even though the service _seems_ up
and running and is accessible from a command line client:

[#|2010-03-23T09:55:27.734+0100|WARNING|glassfishv3.0|org.springframework.remoting.support.RemoteInvocationTraceInterceptor|_T
hreadID=108;_ThreadName=Thread-1;|Processing of HessianServiceExporter
remote call resulted in fatal exception: de.planconnect
.lib.core.api.IDocumentDAO.getDocumentById
com.caucho.hessian.client.HessianConnectionException: 500:
java.io.IOException: Server returned HTTP response code: 500 for UR
L:
http://localhost:9898/webapp-svcs-backend-fileservices/services/fileaccess
         at
com.caucho.hessian.client.HessianProxy.invoke(HessianProxy.java:197)
         at $Proxy180.getMetaFiles(Unknown Source)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)



Well... I know that tracing this down without knowing much about the
applications involved might be next to impossible, but, as this worked
flawlessly on v2, I am not sure whether it's an application issue or
"just"(?) a matter of HTTP configuration on v3. Do these errors make
something to anyone out here? Any inspirations, hints, ... on that?

TIA and all the best,
Kristian



-- 
Dipl.-Ing.(BA) Kristian Rink * Software- und Systemingenieur
planConnect GmbH * Könneritzstr. 33 * 01067 Dresden
fon: 0351 215 203 71 * cell: 0176 2447 2771 * mail: rink_at_planconnect.de
Amtsgericht Dresden HRB: 20 015 * St.-Nr. FA DD I 201 / 116 / 05360
Geschäftsführer: Stefan Voß, Karl Stierstorfer