[javaee-spec users] Re: descriptor overrides

From: Bill Shannon <>
Date: Thu, 26 Jul 2012 16:06:37 -0700

Let's talk about deployment descriptor overrides in this message...

We already have the ability to override deployment descriptors using the
"alt-dd" mechanism in the ear deployment descriptor. We're going to expand that
in EE 7 to include the ability to override the ear deployment descriptor itself.

However, that's not a general mechanism, and doesn't cover all possible
descriptors such as beans.xml.

It's easy to imagine a general override mechanism where any file in an
"overriding" ear/war/jar file replaces the corresponding file in the
application. One concern with that approach is that it provides too much
"rope", and people will tend to trip over it or wrap it around their necks.

A more limited override mechanism might be sufficient, but how do you define the
limits? Does it work for descriptors used by third party libraries that are
part of the application? It's hard to do this in a way that will be consistent
and easily understood.

A related problem that we haven't yet spent enough time on is the need to allow
tenants in a multi-tenant environment to provide customizations for an
application they're all sharing. If we come up with a good way to do this, it
might be usable in a non-multi-tenant environment as well.

Also, as you point out, there's the issue of whether such customizations or
overrides should replace the corresponding information in the app, or should be
merged with it. I'd rather not define merging at the syntactic level (unless
it's just concatenation) and instead allow multiple sources of information that
are merged at the semantics level. That's what we currently do for things like
web fragment descriptors.

My feeling right now is that we might be able to fix a few of the individual
problems, such as the alt-dd problem, but that a more complete solution should
come out of our work to support multi-tenant applications. Unfortunately, we
haven't made much progress on that part of the problem yet.

I would love some more feedback from you and others as to how we should solve
any part of this problem. It's tempting to go with the quick fix of defining a
general file-based override/overlay mechanism, but I worry about the
consequences of that approach.

Craig Ringer wrote on 07/16/2012 09:16 PM:
> Hi all
> I'd like to raise two related areas of difficulty I've encountered with Java
> EE 6 projects:
> * The lack of a simple, user-friendly and reliable mechanism for application
> configuration; and
> * The difficulty of overriding or overlaying deployment descriptors like
> beans.xml on a per-deployment basis.
> I've written a detailed post about the topic here:
> which I reproduce below. I'd appreciate your thoughts, especially on the
> suggestions for improvements at the end of the piece. This seems like a
> HTML-friendly mailing list so I've retained the original formatting rather
> than reformatting in plain text. Hope that's OK.
> Most applications need configuration options or preferences, a way for the
> user to edit them, and a way for the application to read and edit them. Java
> EE applications are no exception. Right now Java EE applications have numerous
> options for such settings - but no single choice that's simple, admin- and
> app-author friendly, and easy for both admin and app to modify.
> Java EE is supposed to provide much of the groundwork for apps as part of the
> container, so the app author can get on with solving their problem. Right now
> it doesn't help much with application settings, and this needs improvement so
> that apps:
> * Don't need to each provide a custom UI in order to edit even the simplest
> settings
> * Can easily modify their own settings
> * Can guarantee the persistence of their settings across redeploys and
> across cluster nodes
> * Don't require a full database and data access layer / JPA / etc just to
> store simple configuration options.
> I'd like to highlight this as an issue for Java EE 7 or 8.
> Approaches
> A Java SE app can:
> * Use a .properties file stored in a well-known location like the user's
> homedir. The user can edit this file and/or the app can load and rewrite
> it easily.
> * Use the Preferences API. This doesn't provide easy direct editing by the
> user, but gives the app a very simple way to store its settings.
> * Have the user set system properties
> All these options are available to Java EE apps, but with caveats that render
> them unappealing as noted below. Java EE apps also have some additional options:
> * Servlet context parameters
> * JNDI environment resources
> * Storing settings in a container-defined SQL datasource
> * For JBoss AS 7: Deploy a companion archive
> Again, none of them are particularly nice, though keeping the configuration in
> the database is usually the most reasonable choice.
> Direct file system access
> Direct file system access for storing configuration isn't trivial, because
> there's no clear and obvious place for configuration files to live. Should
> they be stored in the homedir of the user who's running the app server? In the
> app server's directory? In some arbitrary platform-specific location?
> More importantly, will the app actually have rights to read or write the file
> wherever it lands up, given that a SecurityManager may be in effect?
> Preferences API
> The Preferences API would seem like the ideal solution despite the lack of
> direct .properties editing by users. Unfortunately, at least some application
> servers seem to like to overwrite preferences when an application is
> re-deployed. Replication of preferences in a clustered app server is undefined
> and app-server specific. It's a mess.
> The app must also provide an editing UI for settings in the preferences API;
> there's no way for the user to edit it directly.
> System properties
> System properties work well, but they're set via different methods in each
> application server. They're globally visible across all applications, which
> could be a security concern for some properties. Most importantly, they're
> difficult or impossible for the application to update - they're effectively
> read-only.
> Servlet context parameters
> Servlet context parameters can be difficult to access globally across the
> application without hacks like adding a servlet filter to capture them.
> Methods for setting them without source changes are application-server
> specific and not at all friendly.
> JNDI environment resources
> JNDI environment resources are even less fun to define for the admin, and can
> be clumsy to use within the application. They have a legacy Java EE 5-or-older
> feel and appear little-used.
> JBoss AS 7 deployment companion JARs
> JBoss AS 7's ability to deploy companion JARs with a deployment archive is
> extremely useful. These JARs are private to the deployment and on the
> classpath. They provide an easy mechanism for customising some application
> descriptors and configuration. Unfortunately you can't deploy a companion jar
> as a jar overlay to override descriptors in the original jar, but
> you /can/ use it to provide app-server-specific injectable beans and services,
> and to provide deployment-specific configuration. However, this feature (a)
> isn't available on Glassfish and (b) requires the user to build a .jar and
> deploy it with your app using |jboss-cli|. Again, the app cannot easily modify
> the configuration bundled in the archive.
> Best choice: Configuration in the database
> A very common solution for Java EE applications is to store their
> configuration in a database. Many (most) EE apps are database driven anyway,
> so keeping the configuration in the DB makes sense. The DB is generally
> accessible to all instances of a distributed/clustered app.
> This is all fine so long as you're using a container-provided JDBC / JTA
> datasource. However it isn't suitable for apps that want to
> use |_at_DataSourceDefinition| or a bundled datasource like a |-ds.xml| file to
> make deployment easier for users.
> It's also useless for the purpose of /specifying the name of the data source
> you want the application to connect to/ if you need to avoid hard-coding that.
> You can use a system property for that purpose and get the rest of the
> configuration from the database, though.
> Even for apps that require a container data source anyway, one annoyance with
> this approach is that it's hard for users to go in and edit it. You are likely
> to need to write configuration pages for everything; you can't just direct
> them to a config file. That's OK for highly dynamic configuration, but it's a
> real pain for largely static stuff that gets set at the start of the
> deployment and forgotten.
> Alternative choice: System properties pointing to properties files
> If your app isn't database driven you probably don't want to make the user set
> up a datasource in the container for your app to store its settings. You just
> want them to be able to deploy the app. This isn't really practical at the moment.
> A system property pointing to a properties file location can be a somewhat
> ugly but reasonable compromise in this case. It avoids the user needing to
> deal with per-app-server quirks re how to configure a new persistent H2 /
> Derby datasource. Setting system properties is at least usually easier.
> So what do I think should be available?
> Preferences API constrained for Java EE applications
> My ideal would be for the Java EE 7 spec to constrain the Preferences API
> support on Java EE 7 compliant app servers with the following additional
> requirements:
> * Implementations /must/ preserve the preferences API settings across
> re-deploys of an application. They /may/ offer a deployment option to
> clear and reset preferences.
> * Implementations /must/ provide an interface to edit, back up, clear, and
> restore/load stored preferences for a deployment. Command line, web,
> whatever; just provide an interface for it. This interface /must/ accept
> and produce compliant preferences XML
> <>;
> it /may/ accept and produce other formats.
> * On EE7 implementations that provide clustering
> features, |java.util.prefs.Preferences
> <>| /must/ implement
> the new java.util.prefs.SharedPreferences interface, which provides
> methods (yet to be defined) to allow applications to control
> synchronization of preferences across the
> cluster. |SharedPreferences| /must/ be injectable via CDI.
> * On non-clusterable EE7
> implementations, |java.util.prefs.SharedPreferences| must be
> CDI-injectable as a stub that throws UnsupportedOperationException on
> implemented methods.
> This would make j.u.p.Preferences useful in EE apps. I'm sure the details
> would need work - for example, maybe java.util.prefs.SharedPreferences
> shouldn't be injectable unless supported, so you'd use an Instance to inject it.
> Standardize a deployment overlay mechanism
> Right now it's areal pain to change things like bean alternatives
> in |beans.xml| or the settings in|persistence.xml| on a per-deployment or
> per-app-server basis.
> How can we make it easy to override or merge/overlay deployment arbitrary
> deployment descriptors on a per-deployment basis? Things like:
> * persistence.xml
> * beans.xml
> * JBoss's -ds.xml files
> * glassfish-resource.xml
> * ... and you name it, lots more
> Right now, changing most of these involves modifying unpacking the jar,
> editing the contents, and repacking it, or requires altering the source tree
> and rebuilding. That isn't good for users or app developers/deployers:
> * It makes it hard for "end users" to just deploy an app in a fuss-free manner;
> * It dramatically reduces the utility of CDI's |beans.xml| configuration file
> * It forces the creation of per-app-server flavours of many app builds
> * It's slow, fiddly, and annoying
> What's needed is a standard way to deploy additional descriptors alongside a
> deployment archive, with control over whether these descriptors /override/ the
> bundled descriptor, or are /merged with/ the bundled descriptor. The latter
> case would probably only be supported for selected descriptors
> like |beans.xml| where merging could be made relatively well-defined.
> --
> Craig Ringer