admin@glassfish.java.net

Re: Advice on wrapping MBeanServer

From: Lloyd L Chambers <Lloyd.Chambers_at_Sun.COM>
Date: Fri, 21 Mar 2008 11:10:03 -0700

Eamonn,

If we use our own MBeanServer, we can load it an appropriate time eg
when we can be sure all the classloaders we need are present.

- Using the standard one requires having everything loadable prior to
main().

- V3 could be loaded as a embedded subsystem in another environment
which has already loaded its own Platform MBeanServer and/or already
uses the 'builder' property. What do we do then?

Lloyd

On Mar 21, 2008, at 10:45 AM, Eamonn McManus wrote:

> Lloyd,
> It seems to me that the problem of "weird system properties" is
> independent of whether you use javax.management.initial.builder. In
> other words, even if you don't substitute the implementation of
> MBeanServer, you still have problems like logging properties
> requiring not-yet-loaded classes. I am aware of some of the problems
> that there have been with GF in this domain (having run into them),
> but I think they can be avoided with sufficient care, and such care
> is needed regardless of what you decide to do with AMX.
> Regards,
> Éamonn
>
> Lloyd L Chambers wrote:
>>
>> Eamonn,
>>
>> Yes, I agree we can do an implementation which doesn't know the
>> classes of MBeans to be loaded -- my code is already doing that; it
>> has no choice.
>>
>> But the crux of the problem is that this
>> javax.management.initial.builder property requires that all classes
>> be available before main() is called eg in the classpath when
>> 'java' is launched.
>>
>> In a module-based system, this is very hard to guarantee. It
>> caused and continues to cause bugs in V2. Even something as simple
>> as logging causes problems in V2, where an innocent use of logs
>> causes non-existent classes to be demanded. System properties are
>> the culprit--they effectively swap out subsystems with custom code
>> that might not be available prior to main(). I just don't know how
>> many other weird system properties might (indirectly) cause
>> headaches.
>>
>> I *can* wrap the MBeanServer with the
>> javax.management.initial.builder property, being ultra-careful not
>> to invoke anything but standard JDK code. I just don't know if
>> there aren't those *indirect* dependencies based on system
>> properties, which could cause problems.
>>
>> ...
>>
>> KOHSUKE (or Jerome):
>>
>> I would have to define this MBeanServer wrapper class in HK2
>> (currently org.glassfish.admin.mbeanserver.ApperverMBeanServer), in
>> a jar file which is specified in all distributions that potentially
>> use AMX MBeans. Where would I put such a file? This is what it
>> imports:
>>
>> package org.glassfish.admin.mbeanserver;
>> import java.util.Set;
>> import java.util.Collections;
>> import java.util.Map;
>> import java.util.HashMap;
>> import java.io.ObjectInputStream;
>> import javax.management.*;
>> import javax.management.loading.ClassLoaderRepository;
>>
>> public final class AppserverMBeanServer implements MBeanServer {...}
>>
>> Lloyd
>>
>> On Mar 21, 2008, at 9:58 AM, Eamonn McManus wrote:
>>> Lloyd,
>>> I already exchanged mail with Sreeni last month on this subject;
>>> see attached.
>>> Beyond what I say there, here are some things that occur to me.
>>> First, the javax.management.builder.initial property is consulted
>>> by MBeanServerFactory.createMBeanServer every time it is called.
>>> So if you want to have your custom MBeanServer wrapper, you could
>>> set that property just around the call to createMBeanServer, then
>>> unset it or set it back to the previous value. Doing so means that
>>> your MBeanServer will appear in the list returned by
>>> MBeanServerFactory.findMBeanServer(null), which I think is what
>>> you're asking for. Of course there's a small risk that you might
>>> interfere with somebody who's also creating an MBeanServer at the
>>> exact time as you, but you might be able to know that that's not
>>> the case.
>>> Secondly, I'm thinking you could have an MBeanServerBuilder that
>>> has some sort of SPI which means that it doesn't have to know the
>>> classes of the lazily-loaded MBeans at the time it is loaded. So
>>> each module could come along and register a hook for its MBeans.
>>> Then the MBeanServer wrapper will consult each of these hooks in
>>> turn. Perhaps it will do so only when it gets
>>> InstanceNotFoundException, and/or perhaps each hook is associated
>>> with an ObjectName pattern or patterns so you know which hook is
>>> appropriate. I'm thinking you might even use the jar SPI support
>>> so that the hooks would be found automatically for each module.
>>> I think it is important that you continue to use the Platform
>>> MBean Server rather than having a separate one. The reason is that
>>> it is very convenient to be able to attach JConsole from the local
>>> machine without having to authenticate, but that's only available
>>> for the Platform MBS.
>>> Regards,
>>> Éamonn
>>>
>>> Lloyd L Chambers wrote:
>>>>
>>>> Eamonn,
>>>>
>>>> PROBLEM: lazy-loading MBeans
>>>>
>>>> BACKGROUND
>>>>
>>>> In Glassfish V2 we "wrapped" the MBeanServer with our own
>>>> SunoneInterceptor. The internals were pretty nasty, causing a
>>>> variety of bugs, and I improved it as much as I could with a
>>>> "DynamicInterceptor" in later versions. The problems we wanted
>>>> to solve in V2 were not well-considered, but involved dynamically
>>>> loading certain MBeans, flushing config, etc. Not very good
>>>> decisions, but that's the way it was. Fortunately, Glassfish V3
>>>> offers us a new start.
>>>>
>>>> The main problems we encountered were of a classloading nature
>>>> (recursive problems also occurred):
>>>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=1409
>>>>
>>>> GLASSFISH V3
>>>>
>>>> My inclination in V3 is to use the standard MBeanServer
>>>> unmodified for the Platform MBeanServer. But this seems to
>>>> preclude doing certain things; there is a desire to lazy-load our
>>>> MBeans ("AMX"):
>>>> - if Glassfish V3 is running on a TIVO or iPhone, the MBeans
>>>> might never be loaded, or at least be loaded only upon demand.
>>>> - even in a full production system, there is a desire to defer
>>>> loading of MBeans until they are requested.
>>>>
>>>> It appears to me that there is no mechanism for such lazy loading
>>>> without "wrapping" the MBeanServer. Is that correct?
>>>>
>>>> Assuming we must "wrap" the MBeanServer, that means a class
>>>> specified to -Djavax.management.builder.initial must not rely on
>>>> any classes that aren't immediately available via the classpath
>>>> (eg whatever is available before main() is called).
>>>>
>>>> Glassfish V3 is highly modularized, which means that supporting
>>>> an optional feature thus requires loading code to wrap the
>>>> MBeanServer even if that feature were never used!
>>>>
>>>> ALTERNATIVE
>>>>
>>>> For now, I'm creating an alternative MBeanServer, wrapping it
>>>> with my own class, delegating from it to a standard one. This
>>>> leaves the Platform MBeanServer alone, which keeps it
>>>> compatible. This has several undesirable effects though:
>>>> - the JDK loads the java.lang: MBeans in the Platform MBeanServer
>>>> only
>>>> - I cannot find a way to put my MBeanServer in the named list of
>>>> MBeanServers
>>>>
>>>>
>>>> Am I missing something? Is there a better way to insert code
>>>> into the MBeanServer call chain?
>>>>
>>>> Lloyd
>>>>
>>>>
>>>> ---
>>>> Lloyd L Chambers
>>>> lloyd.chambers_at_sun.com
>>>> Sun Microsystems, Inc
>>>>
>>>>
>>>>
>>>
>>> From: Eamonn McManus <Eamonn.McManus_at_Sun.COM>
>>> Date: February 27, 2008 7:33:19 AM PST
>>> To: Sreenivas Munnangi <Sreenivas.Munnangi_at_Sun.COM>
>>> Subject: Re: Guidance on lazy loading and type of mBean
>>>
>>>
>>> Hi Sreeni,
>>> Interesting questions!
>>> Concerning lazy loading, the obvious approach would be to register
>>> some sort of stub object as the MBean, and have it instantiate the
>>> real object the first time any MBean operation is called on it.
>>> Unfortunately this simple approach won't work because when you
>>> register the MBean, the MBean Server will call getMBeanInfo on it.
>>> This is because registerMBean and createMBean return
>>> ObjectInstance and the MBean Server needs to put a class name in
>>> there. It gets this from MBeanInfo.getClassName(). You might be
>>> able to cook up some approach where your stub object knows what
>>> the MBeanInfo of the real MBean will be and can return it without
>>> instantiating the real MBean, but that sounds pretty hard to me.
>>> I believe you are already using the MBeanServerBuilder
>>> functionality to wrap the standard MBeanServer implementation, so
>>> another approach would be to implement the lazy loading in that
>>> wrapper. In other words, the first time the wrapper see a
>>> reference to a given MBean go by, it instantiates that MBean
>>> before forwarding the request to the real MBeanServer. The
>>> InterceptorMBeanServerBuilder will probably allow for that without
>>> too much difficulty.
>>> Finally you could always time how long it takes to create each
>>> MBean and, for the ones that are slow, rewrite them so that part
>>> of their initialization is postponed until the first access.
>>> I'm not sure you need to care about the performance of different
>>> types of MBean. The differences will be tiny and would only show
>>> up if local code is accessing MBeans thousands of times a second.
>>> In the usual case, MBeans are accessed remotely, so the time
>>> between receiving a remote request and getting to the targeted
>>> MBean is negligible compared with the time to do all the
>>> networking stuff. I would recommend sticking with whatever is
>>> simplest (usually Standard MBeans) until you have an identified
>>> need to improve access time for MBeans, which will probably be
>>> never.
>>> Regards,
>>> Éamonn McManus JMX Spec Lead http://weblogs.java.net/blog/emcmanus/
>>>
>>>
>>> Sreenivas Munnangi wrote:
>>>>
>>>> Eamonn,
>>>>
>>>> I was speaking to my manager about the possible optimizations
>>>> which we need for GlassFish V3 as compared to V2.
>>>>
>>>> One of them is lazy loading of mBeans. Is it possible to just
>>>> register the mBean names and defer the loading/creation/
>>>> registration of mBean to the first invocation ?
>>>> Any alternatives ?
>>>>
>>>> The second one is about type of mBean Standard, Dynamic and
>>>> Model, which is more performant, do we have any bench marks ?
>>>>
>>>> Appreciate your continued support and guidance.
>>>>
>>>> thanks
>>>> sreeni
>>>>
>>>
>>>
>>
>> ---
>> Lloyd L Chambers
>> lloyd.chambers_at_sun.com
>> Sun Microsystems, Inc
>>
>>
>>

---
Lloyd L Chambers
lloyd.chambers_at_sun.com
Sun Microsystems, Inc