dev@glassfish.java.net

Re: [Fwd: why is CPU usage so poor on Glassfish startup?]

From: Scott Oaks <Scott.Oaks_at_Sun.COM>
Date: Tue, 15 Aug 2006 14:26:00 -0400

There are two separate questions.

1) Is the system classloader multithreaded? It is not -- as Lloyd says,
it loads each class as it is requested. Could that be changed at a JDK
level? Possibly, but it's not trivial: you'll need a mechanism to tell
the background thread which of the thousands of JDK classes it should
preload, how many of the available CPUs it ought to use for that
background loading (so as not to interfere with what your program really
wants to be doing), and so on. Plus it will ultimately be impacted by
the second question, which is...

2) Is the system classloader scalable to multiple threads (i.e., when
the appserver creates mutiple application classloaders, does the system
classload become a bottleneck)? The loadClass() method of the system
classloader is synchronized, so particularly during startup, it can
hinder that scalablity. After startup, when the required JDK classes
have all been loaded, it's not such a big deal: the method just checks a
table and returns. But when it has actually to define the classes, the
lock is held for a long time. Given that the loader must resolve all the
class dependencies and the possibly circular nature of those
dependencies, I don't imagine that have a more complex locking scheme
that can deal with multiple threads loading different classes is a
trivial effort either. [Note that this applies as well to the shared
classloader of the appserver (the class loader that loads appserv-rt.jar
and all the rest), though the IEC engineers have some ideas on how to
improve that situation.]

Whether or not those two things might have the potential to be improved
in the JDK is a discussion for another alias, since none of the JDK
engineers will see this; perhaps it's not as complex as I envision.
However, such work wouldn't happen at earliest until Java SE 7 (and
really, the plan for that is probably complete, so it would take some
pushing, I think, to get it in before Java SE 8).

Given that state, the question is how we adapt glassfish to make the
best use of the existing JDK, but at some point, we'll always be limited
in startup by classloading.

-Scott

On Tue, 2006-08-15 at 12:43, Lloyd L Chambers wrote:
> Bill,
>
> I don't know, but I would hope you are right.
>
> But it might be efficient in an overall sense, but not very smart
> about pre-reading additional data and/or doing reads that are not
> efficient for fast disks.
>
> For example, it might be used a 16K or 32K buffer, when it ought to
> be doing 256K or 512K at a time.
>
> Or it might just load each class as it is requested, rather than pre-
> loading the class bytes of other classes in the background; then
> every time something is needed it's the bottleneck.
>
> Lloyd
>
>
> On Aug 15, 2006, at 12:03 AM, Bill Shannon wrote:
>
> > Scott Oaks wrote:
> >>> What I don't "get" is why the other modules don't perform their
> >>> initialization in a separate thread, with suitable threading
> >>> locks or barriers when there are dependencies. The server
> >>> *should* be able to start up in about 2 seconds on my MacBook Pro.
> >> One thing that impacts this is the Java classloading hierarchy; all
> >> class loaders used by the appserver delegate to the system
> >> classloader,
> >> so that becomes a big single-threaded bottleneck. We are
> >> investigating
> >> ways to improve this in 9.1.
> >
> > Is the system classloader really that big of a bottleneck?
> > *Everyone* uses the system classloader, I would expect that
> > it had been well tuned and multi-threaded by now. If not,
> > looks like we have feedback for the Java SE group!
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> > For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>