Thanks for replying; more goodies inline.
On 11/28/2012 07:29 PM, michael keith wrote:
> I appreciate your comments, David.
> See inline.
>
> On 28/11/2012 4:04 PM, David M. Lloyd wrote:
>> Comments inline. Also please note that any time I use the word
>> "module" I am referring to modules as defined in the Java EE
>> specification, not any proposed SE modularity solution, unless I
>> specify otherwise.
>>
>> On 11/28/2012 01:56 PM, michael keith wrote:
>>> As you know, modularity was recently deferred from SE 8 (and
>>> transitively from EE 8), but since there will now be a longer delay
>>> before core modularity can be leveraged, a few people were wondering if
>>> the spec could provide some specific solutions to some of the more
>>> dominant problems that developers are facing now. The most common one
>>> that modularity would have solved is the "conflicting library problem".
>>> This is when there exists a version of a library that is bundled with
>>> the application but conflicts with a different version of the library
>>> used by the server and that is exposed on the server classpath.
>>>
>>> One way to solve this would be to provide a way for an application to
>>> indicate that a bundled library is to be placed on the classpath
>>> *logically ahead* of all server libraries. Most app servers offer some
>>> proprietary solution to this problem, but there is currently no portable
>>> way of doing it. If folks are agreeable then we would like to
>>> standardize a solution in EE 7. We would first like to get your input as
>>> to whether it is worth doing this, though.
>>
>> Speaking from experience I think it's very safe to say that ordering
>> is not a complete solution by any means, especially because in a
>> simple class path system, classes can "leak through" very easily, and
>> it only solves the most superficial part of the problem ("I can't see
>> the classes I need") in the most superficial way possible ("just layer
>> this one on top of that one and hope for the best"). Any solution
>> that can regularly result in substantially different behavior for
>> applications based upon import order of dependencies is suspect, and
>> is certain to cause the user a great deal of trouble.
>
> Yes, those issues can arise. To reiterate, we are not trying to come up
> with a complete solution, just asking whether people thought it was
> worth coming up with a specific solution to a fairly specific problem.
> Is it your position, then, that a feature of this nature would not be
> worth offering, or that the value it offered would be offset by the
> problems that it would introduce?
I think that the value is not worth the problems. Especially when
vendors (such as, well, us) have other solutions to the same problem
already. And because I basically agree with Markus Eisele in this
point: in general, applications *shouldn't* see third-party libraries in
the first place unless they specifically want to, in which case it's not
a problem anyway.
>>> In conjunction with the above question there is an additional question
>>> relating to the limitation that we would need to impose in this initial
>>> solution. Given that a) the majority of problem occurrences are due to
>>> 3rd party libraries being exposed by the server, and b) overriding a
>>> Java EE API library is going to introduce a larger problem space than
>>> what we would likely be able to solve in the time remaining in Java EE,
>>> do you think it is worth providing a soution in EE 7 if that solution
>>> was limited to 3rd party libraries and did not include support for
>>> overriding Java EE APIs? Or, would you prefer to leave it until the next
>>> release and include support for both, or even wait until EE 9 when
>>> modularity will allow a more encompassing solution?
>>
>> I think that under no circumstances should the packages in the Java SE
>> or EE APIs be overridable by user applications or modules.
>
> One can already override certain packages in Java SE in an SE
> environment using the "endorsed" mechanism. There are some similar use
> cases for upgrading certain Java EE APIs in a server as well, but given
> your answer it sounds like you would not have a problem if the ability
> to override these was not included immediately (or ever, from the sound
> of it).
Yeah the problem is that unfortunately not all SE or EE APIs are purely
implementation-free, so simply upgrading them is often not meaningful
(though superficially attractive in some cases).
>>> If enough people would like to solve the limited scope probem in EE 7
>>> then we will follow up with a proposal and some additional questions.
>>
>> I think that it would not be hard to offer a more complete solution
>> without introducing or conflicting with a JDK modularity concept.
>>
>> To me the range of the problem is as follows:
>>
>> 1) Applications, in some containers, may be able to "see" libraries
>> they don't want to "see" for various reasons (perhaps it provides a
>> SPI implementation that is undesirable, or it conflicts with a library
>> in the application itself)
>>
>> 2) Application and module authors often desire the ability to link to
>> other top-level applications or modules via Class-Path or similar
>> mechanisms
>>
>> 3) Application authors often require the ability to establish
>> non-transitive dependencies between applications, modules or JARs therein
>>
>> 4) The Java EE specification does not clearly or satisfactorily
>> specify the semantics of Class-Path (it could imply that classes are
>> copied between class loaders, or it could imply that class loaders
>> link to one another), though it does clearly state that in most (maybe
>> all) cases, Class-Path always imports the transitive closure of the
>> Class-Paths of the referenced URI
>>
>> I think that the first steps to cleaning up this mess are as follows:
>>
>> 1) Update the Java EE platform specification to clearly define
>> Class-Path as establishing a link between existing class loaders
>> (possibly allowing for vendor-specific extensions which might perform
>> old-style "class copying" for special cases)
>>
>> 2) Add a provision to the specification to allow for a non-transitive
>> variation of Class-Path
>
> I know where you are coming from, and I agree it would be nice and clean
> if that were the case. However, the spec has purposely defined class
> loading in terms of what should be available where, rather than being
> prescriptive about class loaders, which are really an implementation
> artifact. The spec does not even prescribe which class loaders actually
> exist (apart from the context class loader), mostly to allow vendors the
> flexibility to be able to implement it the way they so choose. I can't
> see this ever changing (for a bunch of reasons, not the least of which
> is that it would likely break backwards compatibility for at least some
> vendor products), and most certainly not in this release.
I understand the desire to keep the spec flexible. But I think in this
case, more flexibility for us vendors results in less flexibility for
the end user. I don't really *want* flexibility, really; I want my
specifications to be specific. :)
But, that conceptual stuff aside, I see the compatibility argument and I
acknowledge its validity, though I think that the same problems will
still exist when modularity rolls around. If we can find a way to
provide more clear and strict definitions for the way linkage works in
terms of class loaders now, with some provision for vendor-specific
behavior, then it will only make modularity simpler when the time comes.
>> 3) Add a provision to the specification to allow the application or
>> module to choose what non-mandatory APIs and server libraries are
>> requested, providing for standard (Java SE and EE) and vendor-specific
>> library namespaces (a Java SE modularity solution *could* be used to
>> effect this, but does not *have* to be)
>
> Yeah, this is right down modularity lane. The intent was not to try and
> solve all of the problems that modularity is going to solve, just get
> people past one or two of the pain points until modularity is available.
> I agree solidly with what you you are describing, I just don't think we
> are in a position to try to tackle it until we have a proper module
> system to work with.
I don't know; we managed to do it pretty easily and fairly completely
(there are perhaps one or two lessons to be carried forward from that
experience), and I don't think the solution we have chosen (which is
more or less what I described above) is going to be substantially
incompatible with SE modularity (even if it ends up being strongly
Jigsaw-flavored). But even if only #1/#2 are accomplished then I think
that will represent a significant improvement in the current situation -
though at that point we have drifted pretty far away from the original
problem statement I think!
>> 4) Add a provision to the specification to allow for a vendor-specific
>> mechanism by which the user may add additional libraries to this set
>>
>> 5) Ensure that the specification requires that any imports of such
>> libraries are generally non-transitive with respect to their
>> dependencies (unless specific transitive dependencies are required;
>> i.e. opt-in)
>
> The modularity guy in you is really shining through, here :-)
:-)
>> I know the EE specification has, until now, been pretty careful about
>> avoiding specifying the behavior of class loaders. But, if this
>> problem is to be truly solved (either in conjunction with SE
>> modularity or simply in a way that would allow for future SE
>> modularity), this problem has to be addressed head-on, and I think
>> that these simple steps would make for a much more solid starting point.
>
> At least the last three of these were being planned as part of an EE
> modularity strategy... until modularity was dropped from SE 8 and it
> became clear that we weren't going to be able to add modularity into EE
> until EE 9 (at which point it became somewhat less urgent).
--
- DML