jsr338-experts@jpa-spec.java.net

[jsr338-experts] Re: schema generation open issues

From: Linda DeMichiel <linda.demichiel_at_oracle.com>
Date: Tue, 05 Feb 2013 11:20:21 -0800

thanks, Steve

On 2/5/2013 9:13 AM, Steve Ebersole wrote:
> 1) I think the split into database-action and script-action plays much nicer. Nice thought.
>
> 2) You wrote '* javax.persistence.schema-generation.database-source' in the updated spec section, as a replacement for
> javax.persistence.schema-generation-source. IMO this should be named 'javax.persistence.schema-generation.create-source'
> instead to pair with the new 'javax.persistence.schema-generation.drop-source'
>
> 3) I wonder if a better consistent naming scheme would be to use javax.persistence.schema-generation.database.<setting>
> and javax.persistence.schema-generation.scripts.<setting>? To me that plays much nicer:
>

I am fine with this. See the below for minor tweaks/questions.
Otherwise, I plan to commit these changes to the spec.

> javax.persistence.database-product-name
> javax.persistence.database-major-version
> javax.persistence.database-minor-version
>
> javax.persistence.schema-generation.create-source
> javax.persistence.schema-generation.drop-source
> javax.persistence.schema-generation.create-script-source
> javax.persistence.schema-generation.drop-script-source

This would be javax.persistence.schema-generation.scripts.create-source and drop-source, right?

Should the preceding be javax.persistence.schema-generation.database.create-source and drop-source ?
Or is that obvious?

> javax.persistence.schema-generation.connection
> javax.persistence.schema-generation.create-database-schemas*
>
> javax.persistence.schema-generation.database.action
>
> javax.persistence.schema-generation.scripts.action
> javax.persistence.schema-generation.scripts.create-target
> javax.persistence.schema-generation.scripts.drop-target
>
> * - I was not clear whether create-database-schemas is meant to describe just database target or scripts target or both..
>

Should cover both

> 4) WRT "data loading", the question of delimiter is still open. I think this ought to be spelled out in the spec,
> otherwise provider portability will be a major problem.
>
>
>
> On 02/04/2013 03:49 PM, Linda DeMichiel wrote:
>> Steve, all,
>>
>> You have raised good questions. To clarify all this up, I am proposing that we
>> refactor the metadata a bit.
>>
>> To summarize, I am proposing that we do the following:
>>
>> 1) modify javax.persistence.schema-generation-action to be
>> javax.persistence.schema-generation.database-action. The property
>> values and semantics are the same, with the exception that if this
>> property is not specified, no schema generation actions must be
>> taken *on the database*.
>>
>> (2) remove javax.persistence.schema-generation-target and replace with
>> javax.persistence.schema-generation.script-action. The values of
>> this property and semantics would be the same as
>> javax.persistence.schema-generation.database-action, except that this
>> pertains to scripts only. If this property is not specified, no
>> scripts will be generated.
>>
>> (3) To accommodate the feedback about drop execution order,
>> I think we should add another property,
>> javax.persistence.schema-generation.drop-source that
>> is just like javax.persistence.schema-generation-source, but allows
>> control over dropping the database artifacts and the order in which
>> this is to occur.
>>
>> (4) make some minor naming changes to make things more consistent/clearer.
>>
>> More formally, the resulting spec section would look like this:
>> Section 8.2.1.9 would be updated similarly, as appropriate.
>>
>> [Section 9.4]
>>
>> * javax.persistence.schema-generation.database-action
>>
>> The javax.persistence.schema-generation.database-action property
>> specifies the action to be taken by the persistence provider with
>> regard to the database artifacts. The values for this property are
>> "none", "create", "drop-and-create", "drop". If the
>> javax.persistence.schema-generation.database-action property is not
>> specified, no schema generation actions must be taken on the database.
>>
>>
>> * javax.persistence.schema-generation.script-action
>>
>> The javax.persistence.schema-generation.script-action property
>> specifies which scripts are to be generated by the persistence
>> provider. The values for this property are "none", "create",
>> "drop-and-create", "drop". Scripts will only be generated if script
>> targets are specified. If this property is not specified, no scripts
>> will be generated.
>>
>>
>> * javax.persistence.schema-generation.database-source
>>
>> The javax.persistence.schema-generation.database-source property
>> specifies whether the creation of database artifacts is to occur on
>> the basis of the object/relational mapping metadata, DDL script, or a
>> combination of the two. The values for this property are "metadata",
>> "script", "metadata-then-script", "script-then-metadata". If this
>> property is not specified, and a script is specified by the
>> javax.persistence.schema-generation.create-script-source property,
>> the script (only) will be used for schema generation; otherwise if
>> this property is not specified, schema generation will occur on the
>> basis of the object/relational mapping metadata (only). The
>> "metadata-then-script" and "script-then-metadata" values specify that a
>> combination of metadata and script is to be used and the order in
>> which this use is to occur. If either of these values is specified and
>> the resulting database actions are not disjoint, the results are
>> undefined and schema generation may fail.
>>
>>
>> * javax.persistence.schema-generation.drop-source
>>
>> The javax.persistence.schema-generation.drop-source property specifies
>> whether the dropping of database artifacts is to occur on the basis of
>> the object/relational mapping metadata, DDL script, or a combination
>> of the two. The values for this property are "metadata", "script",
>> "metadata-then-script", "script-then-metadata". If this property is
>> not specified, and a script is specified by the
>> javax.persistence.schema-generation.drop-script-source property,
>> the script (only) will be used for the dropping of database artifacts;
>> otherwise if this property is not specified, the dropping of database
>> artifacts will occur on the basis of the object/relational mapping
>> metadata (only). The "metadata-then-script" and "script-then-metadata"
>> values specify that a combination of metadata and script is to be used
>> and the order in which this use is to occur. If either of these values
>> is specified and the resulting database actions are not disjoint, the
>> results are undefined and the dropping of database artifacts may fail.
>>
>>
>> * javax.persistence.schema-generation.create-database-schemas
>>
>> In Java EE environments, it is anticipated that the Java EE platform
>> provider may wish to control the creation of database schemas rather
>> than delegate this task to the persistence provider. The
>> javax.persistence.schema-generation.create-database-schemas property
>> specifies whether the persistence provider is to create the database
>> schema(s) in addition to creating database objects such as tables,
>> sequences, constraints, etc. The value of this boolean property should
>> be set to true if the persistence provider is to create schemas in the
>> database or to generate DDL that contains "CREATE SCHEMA" commands. If
>> this property is not supplied, the provider should not attempt to
>> create database schemas. This property may also be specified in Java
>> SE environments.
>>
>>
>> * javax.persistence.schema-generation.create-script-target,
>> javax.persistence.schema-generation.drop-script-target
>>
>> If scripts are to be generated, the target locations for the writing
>> of these scripts must be specified.
>>
>> The javax.persistence.schema-generation.create-script-target property
>> specifies a java.IO.Writer configured for use by the persistence
>> provider for output of the DDL script or a string specifying the file
>> URL for the DDL script. This property should only be specified if
>> scripts are to be generated.
>>
>> The javax.persistence.schema-generation.drop-script-target property
>> specifies a java.IO.Writer configured for use by the persistence
>> provider for output of the DDL script or a string specifying the file
>> URL for the DDL script. This property should only be specified if
>> scripts are to be generated.
>>
>>
>> * javax.persistence.database-product-name,
>> javax.persistence.database-major-version,
>> javax.persistence.database-minor-version
>>
>> If scripts are to be generated by the persistence provider and a
>> connection to the target database is not supplied, the
>> javax.persistence.database-product-name property must be
>> specified. The value of this property should be the value returned for
>> the target database by the JDBC DatabaseMetaData method
>> getDatabaseProductName. If sufficient database version information is
>> not included in the result of this method, the javax.persistence.
>> database-major-version and javax.persistence.database-minor-version
>> properties should be specified as needed. These should contain the
>> values returned by the JDBC getDatabaseMajorVersion and
>> getDatabaseMinorVersion methods respectively.
>>
>>
>> * javax.persistence.schema-generation.create-script-source,
>> * javax.persistence.schema-generation.drop-script-source
>>
>> The javax.persistence.schema-generation.create-script-source and
>> javax.persistence.schema-generation.drop-script-source properties are
>> used for script execution. In Java EE container environments, it is
>> generally expected that the container will be responsible for
>> executing DDL scripts, although the container is permitted to delegate
>> this task to the persistence provider. If DDL scripts are to be used
>> in Java SE environments or if the Java EE container delegates the
>> execution of scripts to the persistence provider, these properties
>> must be specified. The javax.persistence.schema-generation.create-script-source
>> property specifies a java.IO.Reader configured for reading of the DDL script or
>> a string designating a file URL for the DDL script.
>>
>> The javax.persistence.schema-generation.drop-script-source property
>> specifies a java.IO.Reader configured for reading of the DDL script or
>> a string designating a file URL for the DDL script.
>>
>>
>> * javax.persistence.schema-generation.connection
>>
>> The javax.persistence.schema-generation.connection property specifies
>> the JDBC connection to be used for schema generation. This is intended
>> for use in Java EE environments, where the platform provider may want
>> to control the database privileges that are available to the
>> persistence provider. This connection is provided by the container,
>> and should be closed by the container when the schema generation
>> request or entity manager factory creation completes. The connection
>> provided must have credentials sufficient for the persistence provider
>> to carry out the requested actions. If this property is not specified,
>> the persistence provider should use the DataSource that has otherwise
>> been provided.
>>
>>
>> [subsection 9.4.1] Data Loading
>>
>> Data loading, by means of the use of SQL scripts, may occur as part of
>> the schema generation process after the creation of the database
>> artifacts or independently of schema generation. The specification of
>> the javax.persistence.sql-load-script-source controls whether data
>> loading will occur.
>>
>> * javax.persistence.sql-load-script-source
>>
>> In Java EE container environments, it is generally expected that the
>> container will be responsible for executing data load scripts,
>> although the container is permitted to delegate this task to the
>> persistence provider. If a load script is to be used in Java SE
>> environments or if the Java EE container delegates the execution of
>> the load script to the persistence provider, this property must be
>> specified. The javax.persistence.sql-load-script-source property
>> specifies a java.IO.Reader configured for reading of the SQL load
>> script for database initialization or a string designating a file URL
>> for the script.
>>
>>
>>
>>
>>
>> On 2/1/2013 7:23 AM, Steve Ebersole wrote:
>>> I guess what I was getting at yesterday is that it *seems* as if the intent is that
>>> 'javax.persistence.ddl-create-script-target' and 'javax.persistence.ddl-drop-script-target':
>>> 1) need to be specified either both together or neither
>>> 2) Control:
>>> 2.a) whether scripts are generated
>>> 2.b) what types of scripts are generated.
>>>
>>> So if I have:
>>>
>>> javax.persistence.schema-generation-action=create
>>> javax.persistence.schema-generation-target=database-and-scripts
>>> javax.persistence.ddl-create-script-target=<something>
>>> javax.persistence.ddl-drop-script-target=<something>
>>>
>>> the "action=create" bit really only describes the database target side of things. On the script target side, both create
>>> and drop are created.
>>>
>>> To me at least that is not the most intuitive set up. If I say "action=create", why are the drop scripts being
>>> generated?
>>>
>>>
>>>
>>> On Thu 31 Jan 2013 03:03:43 PM CST, Steve Ebersole wrote:
>>>> Well the confusion IMO as a user is that I specifically asked for only
>>>> creation scripts to be generated, but got both creation and drop
>>>> scripts. Thats not going to change by making schema-generation-target
>>>> required. Even this is still confusing:
>>>>
>>>> javax.persistence.schema-generation-action=create
>>>> javax.persistence.schema-generation-target=scripts
>>>> javax.persistence.ddl-create-script-target=<something>
>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>
>>>>
>>>> because i would not expect to get a drop script. But if I read the
>>>> other parts of the spec correctly, the following actually results in
>>>> an exception:
>>>>
>>>> javax.persistence.schema-generation-action=create
>>>> javax.persistence.schema-generation-target=scripts
>>>> javax.persistence.ddl-create-script-target=<something>
>>>>
>>>> The exception is because 'javax.persistence.ddl-drop-script-target' is
>>>> not specified, again even though i just said I only want create
>>>> scripts generated.
>>>>
>>>> On Thu 31 Jan 2013 02:13:20 PM CST, Linda DeMichiel wrote:
>>>>>
>>>>>
>>>>> On 1/31/2013 11:52 AM, Steve Ebersole wrote:
>>>>>> I also am not sure about this passage in discussing
>>>>>> 'javax.persistence.ddl-create-script-target' and
>>>>>> 'javax.persistence.ddl-drop-script-target':
>>>>>>
>>>>>> <quote>
>>>>>> The persistence provider must produce both create and drop scripts if
>>>>>> the corresponding
>>>>>> DDL targets are specified. This is independent of whether a drop
>>>>>> action is included in the
>>>>>> value passed for the javax.persistence.schema-generation-action
>>>>>> property.
>>>>>> </quote>
>>>>>>
>>>>>> To me, this sounds like you are saying that the following config will
>>>>>> generate both create and drop scripts:
>>>>>>
>>>>>> javax.persistence.schema-generation-action=create
>>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>>>
>>>>>> even though the config explicitly specified just create scripts. Is
>>>>>> that your intent?
>>>>>>
>>>>>
>>>>> yes
>>>>>
>>>>>> Same for:
>>>>>>
>>>>>> javax.persistence.schema-generation-action=drop
>>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>>>
>>>>>> that is supposed to generate both create and drop scripts?
>>>>>>
>>>>>
>>>>> yes
>>>>>
>>>>>>
>>>>>> Perhaps the reasoning is that
>>>>>> javax.persistence.schema-generation-action is really just geared
>>>>>> towards
>>>>>> javax.persistence.schema-generation-action=database.
>>>>>
>>>>> partially. also it was intended to control whether any action would
>>>>> be taken.
>>>>>
>>>>> I'm thinking that all this might be clearer if we required
>>>>> schema-generation-target to be specified.
>>>>>
>>>>> But if that is the case, maybe we ought to rename the setting to be
>>>>>> more explicit of that fact or at least document it in the discussion
>>>>>> of 'javax.persistence.schema-generation-action'
>>>>>>
>>>>>>
>>>>>> On 01/31/2013 01:13 PM, Steve Ebersole wrote:
>>>>>>> One quick question.
>>>>>>>
>>>>>>> For 'javax.persistence.schema-generation-source' values of
>>>>>>> metadata-then-scripts and scripts-then-metadata is the
>>>>>>> order the same for both create and drop?
>>>>>>>
>>>>>>> I can envision a situation, for example, with metadata-then-scripts
>>>>>>> where the metadata creates a table and then the
>>>>>>> script creates a procedure. I am not sure that all databases will
>>>>>>> allow you to drop the table if other database
>>>>>>> objects refer to it. Would it be better to say that the defined
>>>>>>> order (x-then-y) applies to create, but the reverse
>>>>>>> order (y-then-x) applies to drop?
>>>>>>>
>>>>>>>
>>>>>>> On 01/31/2013 12:49 PM, Steve Ebersole wrote:
>>>>>>>> These look great. Thanks Linda.
>>>>>>>>
>>>>>>>> On 01/28/2013 05:14 PM, Linda DeMichiel wrote:
>>>>>>>>> There are currently a few open issues in schema generation that need
>>>>>>>>> clarification. These pertain to sections 9.4 ("Schema generation")
>>>>>>>>> and 8.2.1.9 ("properties") in the spec.
>>>>>>>>>
>>>>>>>>> I've made some clarifications and proposed some additions to the
>>>>>>>>> metadata for schema generation as described below. The draft I've
>>>>>>>>> just uploaded reflects these changes
>>>>>>>>> (http://java.net/projects/jpa-spec/downloads/download/JavaPersistence01282013.pdf).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please review these sections and post any feedback. Changes are
>>>>>>>>> summarized below.
>>>>>>>>>
>>>>>>>>> ----
>>>>>>>>>
>>>>>>>>> Section 9.4
>>>>>>>>>
>>>>>>>>> This section isn't sufficiently clear about requirements for the
>>>>>>>>> specification of schema-generation-action and
>>>>>>>>> schema-generation-target.
>>>>>>>>>
>>>>>>>>> I've clarified that schema-generation-action must be specified or no
>>>>>>>>> actions will be taken. This was an assumed default in the current
>>>>>>>>> public draft, but IMO not sufficiently explicit.
>>>>>>>>>
>>>>>>>>> Given the clarification to schema-generation-action, I think it now
>>>>>>>>> makes sense for schema-generation-target to have default values.
>>>>>>>>> That is, if script targets are specified by the properties for
>>>>>>>>> scripts,
>>>>>>>>> then generation actions should default into scripts. If not, actions
>>>>>>>>> should be directly into the database.
>>>>>>>>>
>>>>>>>>> The current public draft assumes that if script sources are
>>>>>>>>> specified,
>>>>>>>>> generation should occur from scripts exclusively; and conversely, if
>>>>>>>>> script sources are not specified, generation should occur
>>>>>>>>> exclusively
>>>>>>>>> from the ORM metadata. I've gotten feedback that this wasn't
>>>>>>>>> sufficiently
>>>>>>>>> flexible, and that we should support a combination of ORM and script
>>>>>>>>> sources. (For example, consider the case where generation should
>>>>>>>>> occur
>>>>>>>>> from scripts, but it is desirable to add stored procedures to the
>>>>>>>>> database.)
>>>>>>>>> I therefore propose that we add a schema-generation-source property
>>>>>>>>> that will support such combination. Again, if script sources are
>>>>>>>>> specified
>>>>>>>>> by the properties for scripts, then generation will default to
>>>>>>>>> use of
>>>>>>>>> only script sources; if script sources aren't specified, then
>>>>>>>>> generation
>>>>>>>>> will default from the ORM metadata.
>>>>>>>>>
>>>>>>>>> I've also received feedback (from our JPA/EE integration engineer)
>>>>>>>>> that
>>>>>>>>> we need to specify that in Java EE environments that strings
>>>>>>>>> corresponding
>>>>>>>>> to file URLs must use absolute (and not relative) paths. I've
>>>>>>>>> added that
>>>>>>>>> clarification as well.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Section 8.2.1.9
>>>>>>>>>
>>>>>>>>> In the current public draft, this section lacks information as to
>>>>>>>>> how
>>>>>>>>> script locations are specified. I've added a clarification that
>>>>>>>>> scripts packaged as part of the application must be specified
>>>>>>>>> relative
>>>>>>>>> to the root of the persistence unit, as we do for other packaged
>>>>>>>>> artifacts. I've also added a clarification about the expectations
>>>>>>>>> for
>>>>>>>>> use of file URLs in Java EE environments as above.
>>>>>>>>>
>>>>>>>>> Finally, when I wrote this up initially, I was reluctant to add
>>>>>>>>> metadata for schema generation actions (including targets) because I
>>>>>>>>> believed that such metadata should not be specific to the
>>>>>>>>> persistence
>>>>>>>>> unit definition itself. It seems however that people expect that the
>>>>>>>>> use of these properties (as specified in section 9.4) will be
>>>>>>>>> available anyway, whether they are specified in the spec document or
>>>>>>>>> not, and that some vendors will support these anyway as well.
>>>>>>>>> Further, this additional metadata may facilitate ease of use in
>>>>>>>>> prototyping/development environments. I've therefore added these
>>>>>>>>> properties into 8.2.1.9. If you disagree, please speak up.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>>
>>>>>>>>> -Linda
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>