On 29/11/2012 1:26 PM, David M. Lloyd wrote:
> Thanks for replying; more goodies inline.
>
> On 11/28/2012 07:29 PM, michael keith wrote:
>> I appreciate your comments, David.
>> See inline.
>>
>> On 28/11/2012 4:04 PM, David M. Lloyd wrote:
>>> Comments inline. Also please note that any time I use the word
>>> "module" I am referring to modules as defined in the Java EE
>>> specification, not any proposed SE modularity solution, unless I
>>> specify otherwise.
>>>
>>> On 11/28/2012 01:56 PM, michael keith wrote:
>>>> As you know, modularity was recently deferred from SE 8 (and
>>>> transitively from EE 8), but since there will now be a longer delay
>>>> before core modularity can be leveraged, a few people were
>>>> wondering if
>>>> the spec could provide some specific solutions to some of the more
>>>> dominant problems that developers are facing now. The most common one
>>>> that modularity would have solved is the "conflicting library
>>>> problem".
>>>> This is when there exists a version of a library that is bundled with
>>>> the application but conflicts with a different version of the library
>>>> used by the server and that is exposed on the server classpath.
>>>>
>>>> One way to solve this would be to provide a way for an application to
>>>> indicate that a bundled library is to be placed on the classpath
>>>> *logically ahead* of all server libraries. Most app servers offer some
>>>> proprietary solution to this problem, but there is currently no
>>>> portable
>>>> way of doing it. If folks are agreeable then we would like to
>>>> standardize a solution in EE 7. We would first like to get your
>>>> input as
>>>> to whether it is worth doing this, though.
>>>
>>> Speaking from experience I think it's very safe to say that ordering
>>> is not a complete solution by any means, especially because in a
>>> simple class path system, classes can "leak through" very easily, and
>>> it only solves the most superficial part of the problem ("I can't see
>>> the classes I need") in the most superficial way possible ("just layer
>>> this one on top of that one and hope for the best"). Any solution
>>> that can regularly result in substantially different behavior for
>>> applications based upon import order of dependencies is suspect, and
>>> is certain to cause the user a great deal of trouble.
>>
>> Yes, those issues can arise. To reiterate, we are not trying to come up
>> with a complete solution, just asking whether people thought it was
>> worth coming up with a specific solution to a fairly specific problem.
>> Is it your position, then, that a feature of this nature would not be
>> worth offering, or that the value it offered would be offset by the
>> problems that it would introduce?
>
> I think that the value is not worth the problems. Especially when
> vendors (such as, well, us) have other solutions to the same problem
> already. And because I basically agree with Markus Eisele in this
> point: in general, applications *shouldn't* see third-party libraries
> in the first place unless they specifically want to, in which case
> it's not a problem anyway.
We all agree on that as the endgame, but having proper modularity will
make that so much easier that I don't think it is worthwhile trying to
get there beforehand. (Of course, there are still problems with it, such
as having to expose implementation APIs invoked by classes that the
container code-generates into the application, or APIs that get added as
part of container-based class enhancement, etc).
>>>> In conjunction with the above question there is an additional question
>>>> relating to the limitation that we would need to impose in this
>>>> initial
>>>> solution. Given that a) the majority of problem occurrences are due to
>>>> 3rd party libraries being exposed by the server, and b) overriding a
>>>> Java EE API library is going to introduce a larger problem space than
>>>> what we would likely be able to solve in the time remaining in Java
>>>> EE,
>>>> do you think it is worth providing a soution in EE 7 if that solution
>>>> was limited to 3rd party libraries and did not include support for
>>>> overriding Java EE APIs? Or, would you prefer to leave it until the
>>>> next
>>>> release and include support for both, or even wait until EE 9 when
>>>> modularity will allow a more encompassing solution?
>>>
>>> I think that under no circumstances should the packages in the Java SE
>>> or EE APIs be overridable by user applications or modules.
>>
>> One can already override certain packages in Java SE in an SE
>> environment using the "endorsed" mechanism. There are some similar use
>> cases for upgrading certain Java EE APIs in a server as well, but given
>> your answer it sounds like you would not have a problem if the ability
>> to override these was not included immediately (or ever, from the sound
>> of it).
>
> Yeah the problem is that unfortunately not all SE or EE APIs are
> purely implementation-free, so simply upgrading them is often not
> meaningful (though superficially attractive in some cases).
There would be a limited well-defined set of APIs that could be
upgraded, and those specs would need to be coded/referenced in such a
way as to be replacable. Obviously, fair notice must be given ahead of
time in those specs and the ones that depend upon them.
>>>> If enough people would like to solve the limited scope probem in EE 7
>>>> then we will follow up with a proposal and some additional questions.
>>>
>>> I think that it would not be hard to offer a more complete solution
>>> without introducing or conflicting with a JDK modularity concept.
>>>
>>> To me the range of the problem is as follows:
>>>
>>> 1) Applications, in some containers, may be able to "see" libraries
>>> they don't want to "see" for various reasons (perhaps it provides a
>>> SPI implementation that is undesirable, or it conflicts with a library
>>> in the application itself)
>>>
>>> 2) Application and module authors often desire the ability to link to
>>> other top-level applications or modules via Class-Path or similar
>>> mechanisms
>>>
>>> 3) Application authors often require the ability to establish
>>> non-transitive dependencies between applications, modules or JARs
>>> therein
>>>
>>> 4) The Java EE specification does not clearly or satisfactorily
>>> specify the semantics of Class-Path (it could imply that classes are
>>> copied between class loaders, or it could imply that class loaders
>>> link to one another), though it does clearly state that in most (maybe
>>> all) cases, Class-Path always imports the transitive closure of the
>>> Class-Paths of the referenced URI
>>>
>>> I think that the first steps to cleaning up this mess are as follows:
>>>
>>> 1) Update the Java EE platform specification to clearly define
>>> Class-Path as establishing a link between existing class loaders
>>> (possibly allowing for vendor-specific extensions which might perform
>>> old-style "class copying" for special cases)
>>>
>>> 2) Add a provision to the specification to allow for a non-transitive
>>> variation of Class-Path
>>
>> I know where you are coming from, and I agree it would be nice and clean
>> if that were the case. However, the spec has purposely defined class
>> loading in terms of what should be available where, rather than being
>> prescriptive about class loaders, which are really an implementation
>> artifact. The spec does not even prescribe which class loaders actually
>> exist (apart from the context class loader), mostly to allow vendors the
>> flexibility to be able to implement it the way they so choose. I can't
>> see this ever changing (for a bunch of reasons, not the least of which
>> is that it would likely break backwards compatibility for at least some
>> vendor products), and most certainly not in this release.
>
> I understand the desire to keep the spec flexible. But I think in
> this case, more flexibility for us vendors results in less flexibility
> for the end user. I don't really *want* flexibility, really; I want
> my specifications to be specific. :)
>
> But, that conceptual stuff aside, I see the compatibility argument and
> I acknowledge its validity, though I think that the same problems will
> still exist when modularity rolls around. If we can find a way to
> provide more clear and strict definitions for the way linkage works in
> terms of class loaders now, with some provision for vendor-specific
> behavior, then it will only make modularity simpler when the time comes.
>
>>> 3) Add a provision to the specification to allow the application or
>>> module to choose what non-mandatory APIs and server libraries are
>>> requested, providing for standard (Java SE and EE) and vendor-specific
>>> library namespaces (a Java SE modularity solution *could* be used to
>>> effect this, but does not *have* to be)
>>
>> Yeah, this is right down modularity lane. The intent was not to try and
>> solve all of the problems that modularity is going to solve, just get
>> people past one or two of the pain points until modularity is available.
>> I agree solidly with what you you are describing, I just don't think we
>> are in a position to try to tackle it until we have a proper module
>> system to work with.
>
> I don't know; we managed to do it pretty easily and fairly completely
> (there are perhaps one or two lessons to be carried forward from that
> experience), and I don't think the solution we have chosen (which is
> more or less what I described above) is going to be substantially
> incompatible with SE modularity (even if it ends up being strongly
> Jigsaw-flavored). But even if only #1/#2 are accomplished then I
> think that will represent a significant improvement in the current
> situation - though at that point we have drifted pretty far away from
> the original problem statement I think!
>
>>> 4) Add a provision to the specification to allow for a vendor-specific
>>> mechanism by which the user may add additional libraries to this set
>>>
>>> 5) Ensure that the specification requires that any imports of such
>>> libraries are generally non-transitive with respect to their
>>> dependencies (unless specific transitive dependencies are required;
>>> i.e. opt-in)
>>
>> The modularity guy in you is really shining through, here :-)
>
> :-)
>
>>> I know the EE specification has, until now, been pretty careful about
>>> avoiding specifying the behavior of class loaders. But, if this
>>> problem is to be truly solved (either in conjunction with SE
>>> modularity or simply in a way that would allow for future SE
>>> modularity), this problem has to be addressed head-on, and I think
>>> that these simple steps would make for a much more solid starting
>>> point.
>>
>> At least the last three of these were being planned as part of an EE
>> modularity strategy... until modularity was dropped from SE 8 and it
>> became clear that we weren't going to be able to add modularity into EE
>> until EE 9 (at which point it became somewhat less urgent).
>