users@glassfish.java.net

Re: Major Application client frustrations

From: <glassfish_at_javadesktop.org>
Date: Sat, 05 May 2007 01:55:21 PDT

Just a few more comments:

> Hello everybody,
>
>
> - Time for login: On slower machines (like a 1.4Ghz
> iBook), login takes >20 seconds, with most of the
> time 100% CPU. Profiling shows that the time is spent
> in ORB.init()... WTF is it doing?

This one is interesting. I know what ORB.init does both in the ORB
and in the app server, but 20 seconds is a long time. Could you share
your profiling info with us?


>
> - Request performance: As pointed out by others in
> this forum, the performance of the ORB greatly sucks.
> We're communicating with statless EJBs, transmitting
> value beans back and forth. Compared to other
> application servers, it takes up to 10 times as long.
>

It should suck less now, as some significant performance problems
have been fixed. Which build are you using? There was a big
problem with the huge classpath of the app server creating a huge
codebase interacting with a bug in the marshalling code that did not
properly handle repeated codebase strings. This fix is certainly in the recent
builds for the past few months.

We also greatly improved how the ORB reads messages from a socket.
It reads all available data as fast as possible, instead of reading 12
bytes then the message size. Internal benchmarks show a 50% or so
increase in the throughput.
 
>
> - Latency: This time, Jonas is 5 times as quick. Our
> average request latency for requests without payload
> (e.g. a simple ping()) is well above 10ms, sometimes
> as bad as 30ms. Without any network interference.
>

We'll need to look at this again. We are mainly focusing on
throughput oriented tests (like SpecJ) at this point.

> - fault tolerance: It may happen, that the server is
> not reachable (host down, glassfish stopped,
> whatever). When it happens and the user wants to
> login, we're getting tons of log output (written to
> stdout, as Java logging is obviously used in the ORB)
> with like this:
>

This should be better once the latest ORB is integrated (see my
previous post). This is exactly the problem I fixed.

>
> - configurability of ORB behaviour: The ORB as a ton
> of properties that could be set. Possibly, but
> whenever I thought I had found a setting it turned
> out to be non-settable. Lets talk about an retry
> count for communication (see above, could save a ton
> of poor electrons wasted for logging). Lets talk
> about a socket timeout - all not settable. This last
> one (network timeouts) is one of our major headaches
> so far.

We don't directly use a count, but the TcpTimeouts I described
give you more or less the same capability.

>
> - network timeout: There is no way to specify, how
> long I want to wait for a connect and how long I want
> a socket to be allowed idle.

Wait for connect is covered. The ORB also has settable
high water marks on the inbound and outbound connection caches,
which will cause connections to be closed. But so far we have
not seen a need to directly age out connections.

 I searched and tried a
> lot, even creating my own socket factory. But guess
> what - the current version does not allow it to be
> overridden. There is a property, which is even being
> described in some documents (e.g. Sun App Server 8
> docs), but Glassfish hardcoded sets it to its
> IIOPSSLSocketFactory.

You're right; I don't think there is a way to override this.
We do need to control this in order to support CSIv2.
Again, with the new timeouts, do you need anything else?

> We're having troubles with
> this, because from time to time, a client may
> experience network problems. When this happens, the
> client is frozen to a point where it is not usable.

You could use the request timeout to handle this,
if the granularity is OK (see previous post).

> If I cannot specify a timeout on socket level, I
> thought I maybe could just have watchdog thread and
> interrupt() some other threads (i.e. the locked
> thread). Try this - its a major show in the log
> console and a broken ORB. Shouldn't any blocking
> thread be prepared to receive an interrupt()?

In general, yes, but many thread could choose to ignore
interrupt, for example a while(!condition) wait loop would
have no choice but to spin and continue waiting if interrupted.
We could probably make the CorbaResponseWaitingRoom
code respond to an interrupt() call, but I'm not sure if that's a
good idea or not.

Ken.
[Message sent by forum member 'kcavanaugh' (kcavanaugh)]

http://forums.java.net/jive/thread.jspa?messageID=215664