[javaee-spec users] Re: bundle overriding

From: michael keith <>
Date: Wed, 28 Nov 2012 20:29:54 -0500

I appreciate your comments, David.
See inline.

On 28/11/2012 4:04 PM, David M. Lloyd wrote:
> Comments inline. Also please note that any time I use the word
> "module" I am referring to modules as defined in the Java EE
> specification, not any proposed SE modularity solution, unless I
> specify otherwise.
> On 11/28/2012 01:56 PM, michael keith wrote:
>> As you know, modularity was recently deferred from SE 8 (and
>> transitively from EE 8), but since there will now be a longer delay
>> before core modularity can be leveraged, a few people were wondering if
>> the spec could provide some specific solutions to some of the more
>> dominant problems that developers are facing now. The most common one
>> that modularity would have solved is the "conflicting library problem".
>> This is when there exists a version of a library that is bundled with
>> the application but conflicts with a different version of the library
>> used by the server and that is exposed on the server classpath.
>> One way to solve this would be to provide a way for an application to
>> indicate that a bundled library is to be placed on the classpath
>> *logically ahead* of all server libraries. Most app servers offer some
>> proprietary solution to this problem, but there is currently no portable
>> way of doing it. If folks are agreeable then we would like to
>> standardize a solution in EE 7. We would first like to get your input as
>> to whether it is worth doing this, though.
> Speaking from experience I think it's very safe to say that ordering
> is not a complete solution by any means, especially because in a
> simple class path system, classes can "leak through" very easily, and
> it only solves the most superficial part of the problem ("I can't see
> the classes I need") in the most superficial way possible ("just layer
> this one on top of that one and hope for the best"). Any solution
> that can regularly result in substantially different behavior for
> applications based upon import order of dependencies is suspect, and
> is certain to cause the user a great deal of trouble.

Yes, those issues can arise. To reiterate, we are not trying to come up
with a complete solution, just asking whether people thought it was
worth coming up with a specific solution to a fairly specific problem.
Is it your position, then, that a feature of this nature would not be
worth offering, or that the value it offered would be offset by the
problems that it would introduce?

>> In conjunction with the above question there is an additional question
>> relating to the limitation that we would need to impose in this initial
>> solution. Given that a) the majority of problem occurrences are due to
>> 3rd party libraries being exposed by the server, and b) overriding a
>> Java EE API library is going to introduce a larger problem space than
>> what we would likely be able to solve in the time remaining in Java EE,
>> do you think it is worth providing a soution in EE 7 if that solution
>> was limited to 3rd party libraries and did not include support for
>> overriding Java EE APIs? Or, would you prefer to leave it until the next
>> release and include support for both, or even wait until EE 9 when
>> modularity will allow a more encompassing solution?
> I think that under no circumstances should the packages in the Java SE
> or EE APIs be overridable by user applications or modules.

One can already override certain packages in Java SE in an SE
environment using the "endorsed" mechanism. There are some similar use
cases for upgrading certain Java EE APIs in a server as well, but given
your answer it sounds like you would not have a problem if the ability
to override these was not included immediately (or ever, from the sound
of it).

>> If enough people would like to solve the limited scope probem in EE 7
>> then we will follow up with a proposal and some additional questions.
> I think that it would not be hard to offer a more complete solution
> without introducing or conflicting with a JDK modularity concept.
> To me the range of the problem is as follows:
> 1) Applications, in some containers, may be able to "see" libraries
> they don't want to "see" for various reasons (perhaps it provides a
> SPI implementation that is undesirable, or it conflicts with a library
> in the application itself)
> 2) Application and module authors often desire the ability to link to
> other top-level applications or modules via Class-Path or similar
> mechanisms
> 3) Application authors often require the ability to establish
> non-transitive dependencies between applications, modules or JARs therein
> 4) The Java EE specification does not clearly or satisfactorily
> specify the semantics of Class-Path (it could imply that classes are
> copied between class loaders, or it could imply that class loaders
> link to one another), though it does clearly state that in most (maybe
> all) cases, Class-Path always imports the transitive closure of the
> Class-Paths of the referenced URI
> I think that the first steps to cleaning up this mess are as follows:
> 1) Update the Java EE platform specification to clearly define
> Class-Path as establishing a link between existing class loaders
> (possibly allowing for vendor-specific extensions which might perform
> old-style "class copying" for special cases)
> 2) Add a provision to the specification to allow for a non-transitive
> variation of Class-Path

I know where you are coming from, and I agree it would be nice and clean
if that were the case. However, the spec has purposely defined class
loading in terms of what should be available where, rather than being
prescriptive about class loaders, which are really an implementation
artifact. The spec does not even prescribe which class loaders actually
exist (apart from the context class loader), mostly to allow vendors the
flexibility to be able to implement it the way they so choose. I can't
see this ever changing (for a bunch of reasons, not the least of which
is that it would likely break backwards compatibility for at least some
vendor products), and most certainly not in this release.

> 3) Add a provision to the specification to allow the application or
> module to choose what non-mandatory APIs and server libraries are
> requested, providing for standard (Java SE and EE) and vendor-specific
> library namespaces (a Java SE modularity solution *could* be used to
> effect this, but does not *have* to be)

Yeah, this is right down modularity lane. The intent was not to try and
solve all of the problems that modularity is going to solve, just get
people past one or two of the pain points until modularity is available.
I agree solidly with what you you are describing, I just don't think we
are in a position to try to tackle it until we have a proper module
system to work with.

> 4) Add a provision to the specification to allow for a vendor-specific
> mechanism by which the user may add additional libraries to this set
> 5) Ensure that the specification requires that any imports of such
> libraries are generally non-transitive with respect to their
> dependencies (unless specific transitive dependencies are required;
> i.e. opt-in)

The modularity guy in you is really shining through, here :-)

> I know the EE specification has, until now, been pretty careful about
> avoiding specifying the behavior of class loaders. But, if this
> problem is to be truly solved (either in conjunction with SE
> modularity or simply in a way that would allow for future SE
> modularity), this problem has to be addressed head-on, and I think
> that these simple steps would make for a much more solid starting point.

At least the last three of these were being planned as part of an EE
modularity strategy... until modularity was dropped from SE 8 and it
became clear that we weren't going to be able to add modularity into EE
until EE 9 (at which point it became somewhat less urgent).