users@jpa-spec.java.net

[jpa-spec users] [jsr338-experts] Re: schema generation open issues

From: Gordon Yorke <gordon.yorke_at_oracle.com>
Date: Tue, 05 Feb 2013 16:30:03 -0400

>> javax.persistence.database-product-name
>> javax.persistence.database-major-version
>> javax.persistence.database-minor-version
>>
>> javax.persistence.schema-generation.create-source
>> javax.persistence.schema-generation.drop-source
>> javax.persistence.schema-generation.create-script-source
>> javax.persistence.schema-generation.drop-script-source
>
> This would be
> javax.persistence.schema-generation.scripts.create-source and
> drop-source, right?
I think it would be confusing to call these
javax.persistence.schema-generation.scripts.create-source when they
reference the actual scripts and are not actions. We could call them
javax.persistence.schema-generation.database.create-script-source if you
need to qualify them but I believe they are fine as they are.

>
> Should the preceding be
> javax.persistence.schema-generation.database.create-source and
> drop-source ?
> Or is that obvious?


On 05/02/2013 3:20 PM, Linda DeMichiel wrote:
> thanks, Steve
>
> On 2/5/2013 9:13 AM, Steve Ebersole wrote:
>> 1) I think the split into database-action and script-action plays
>> much nicer. Nice thought.
>>
>> 2) You wrote '* javax.persistence.schema-generation.database-source'
>> in the updated spec section, as a replacement for
>> javax.persistence.schema-generation-source. IMO this should be named
>> 'javax.persistence.schema-generation.create-source'
>> instead to pair with the new
>> 'javax.persistence.schema-generation.drop-source'
>>
>> 3) I wonder if a better consistent naming scheme would be to use
>> javax.persistence.schema-generation.database.<setting>
>> and javax.persistence.schema-generation.scripts.<setting>? To me that
>> plays much nicer:
>>
>
> I am fine with this. See the below for minor tweaks/questions.
> Otherwise, I plan to commit these changes to the spec.
>
>> javax.persistence.database-product-name
>> javax.persistence.database-major-version
>> javax.persistence.database-minor-version
>>
>> javax.persistence.schema-generation.create-source
>> javax.persistence.schema-generation.drop-source
>> javax.persistence.schema-generation.create-script-source
>> javax.persistence.schema-generation.drop-script-source
>
> This would be
> javax.persistence.schema-generation.scripts.create-source and
> drop-source, right?
>
> Should the preceding be
> javax.persistence.schema-generation.database.create-source and
> drop-source ?
> Or is that obvious?
>
>> javax.persistence.schema-generation.connection
>> javax.persistence.schema-generation.create-database-schemas*
>>
>> javax.persistence.schema-generation.database.action
>>
>> javax.persistence.schema-generation.scripts.action
>> javax.persistence.schema-generation.scripts.create-target
>> javax.persistence.schema-generation.scripts.drop-target
>>
>> * - I was not clear whether create-database-schemas is meant to
>> describe just database target or scripts target or both..
>>
>
> Should cover both
>
>> 4) WRT "data loading", the question of delimiter is still open. I
>> think this ought to be spelled out in the spec,
>> otherwise provider portability will be a major problem.
>>
>>
>>
>> On 02/04/2013 03:49 PM, Linda DeMichiel wrote:
>>> Steve, all,
>>>
>>> You have raised good questions. To clarify all this up, I am
>>> proposing that we
>>> refactor the metadata a bit.
>>>
>>> To summarize, I am proposing that we do the following:
>>>
>>> 1) modify javax.persistence.schema-generation-action to be
>>> javax.persistence.schema-generation.database-action. The property
>>> values and semantics are the same, with the exception that if this
>>> property is not specified, no schema generation actions must be
>>> taken *on the database*.
>>>
>>> (2) remove javax.persistence.schema-generation-target and replace with
>>> javax.persistence.schema-generation.script-action. The values of
>>> this property and semantics would be the same as
>>> javax.persistence.schema-generation.database-action, except that this
>>> pertains to scripts only. If this property is not specified, no
>>> scripts will be generated.
>>>
>>> (3) To accommodate the feedback about drop execution order,
>>> I think we should add another property,
>>> javax.persistence.schema-generation.drop-source that
>>> is just like javax.persistence.schema-generation-source, but allows
>>> control over dropping the database artifacts and the order in which
>>> this is to occur.
>>>
>>> (4) make some minor naming changes to make things more
>>> consistent/clearer.
>>>
>>> More formally, the resulting spec section would look like this:
>>> Section 8.2.1.9 would be updated similarly, as appropriate.
>>>
>>> [Section 9.4]
>>>
>>> * javax.persistence.schema-generation.database-action
>>>
>>> The javax.persistence.schema-generation.database-action property
>>> specifies the action to be taken by the persistence provider with
>>> regard to the database artifacts. The values for this property are
>>> "none", "create", "drop-and-create", "drop". If the
>>> javax.persistence.schema-generation.database-action property is not
>>> specified, no schema generation actions must be taken on the database.
>>>
>>>
>>> * javax.persistence.schema-generation.script-action
>>>
>>> The javax.persistence.schema-generation.script-action property
>>> specifies which scripts are to be generated by the persistence
>>> provider. The values for this property are "none", "create",
>>> "drop-and-create", "drop". Scripts will only be generated if script
>>> targets are specified. If this property is not specified, no scripts
>>> will be generated.
>>>
>>>
>>> * javax.persistence.schema-generation.database-source
>>>
>>> The javax.persistence.schema-generation.database-source property
>>> specifies whether the creation of database artifacts is to occur on
>>> the basis of the object/relational mapping metadata, DDL script, or a
>>> combination of the two. The values for this property are "metadata",
>>> "script", "metadata-then-script", "script-then-metadata". If this
>>> property is not specified, and a script is specified by the
>>> javax.persistence.schema-generation.create-script-source property,
>>> the script (only) will be used for schema generation; otherwise if
>>> this property is not specified, schema generation will occur on the
>>> basis of the object/relational mapping metadata (only). The
>>> "metadata-then-script" and "script-then-metadata" values specify that a
>>> combination of metadata and script is to be used and the order in
>>> which this use is to occur. If either of these values is specified and
>>> the resulting database actions are not disjoint, the results are
>>> undefined and schema generation may fail.
>>>
>>>
>>> * javax.persistence.schema-generation.drop-source
>>>
>>> The javax.persistence.schema-generation.drop-source property specifies
>>> whether the dropping of database artifacts is to occur on the basis of
>>> the object/relational mapping metadata, DDL script, or a combination
>>> of the two. The values for this property are "metadata", "script",
>>> "metadata-then-script", "script-then-metadata". If this property is
>>> not specified, and a script is specified by the
>>> javax.persistence.schema-generation.drop-script-source property,
>>> the script (only) will be used for the dropping of database artifacts;
>>> otherwise if this property is not specified, the dropping of database
>>> artifacts will occur on the basis of the object/relational mapping
>>> metadata (only). The "metadata-then-script" and "script-then-metadata"
>>> values specify that a combination of metadata and script is to be used
>>> and the order in which this use is to occur. If either of these values
>>> is specified and the resulting database actions are not disjoint, the
>>> results are undefined and the dropping of database artifacts may fail.
>>>
>>>
>>> * javax.persistence.schema-generation.create-database-schemas
>>>
>>> In Java EE environments, it is anticipated that the Java EE platform
>>> provider may wish to control the creation of database schemas rather
>>> than delegate this task to the persistence provider. The
>>> javax.persistence.schema-generation.create-database-schemas property
>>> specifies whether the persistence provider is to create the database
>>> schema(s) in addition to creating database objects such as tables,
>>> sequences, constraints, etc. The value of this boolean property should
>>> be set to true if the persistence provider is to create schemas in the
>>> database or to generate DDL that contains "CREATE SCHEMA" commands. If
>>> this property is not supplied, the provider should not attempt to
>>> create database schemas. This property may also be specified in Java
>>> SE environments.
>>>
>>>
>>> * javax.persistence.schema-generation.create-script-target,
>>> javax.persistence.schema-generation.drop-script-target
>>>
>>> If scripts are to be generated, the target locations for the writing
>>> of these scripts must be specified.
>>>
>>> The javax.persistence.schema-generation.create-script-target property
>>> specifies a java.IO.Writer configured for use by the persistence
>>> provider for output of the DDL script or a string specifying the file
>>> URL for the DDL script. This property should only be specified if
>>> scripts are to be generated.
>>>
>>> The javax.persistence.schema-generation.drop-script-target property
>>> specifies a java.IO.Writer configured for use by the persistence
>>> provider for output of the DDL script or a string specifying the file
>>> URL for the DDL script. This property should only be specified if
>>> scripts are to be generated.
>>>
>>>
>>> * javax.persistence.database-product-name,
>>> javax.persistence.database-major-version,
>>> javax.persistence.database-minor-version
>>>
>>> If scripts are to be generated by the persistence provider and a
>>> connection to the target database is not supplied, the
>>> javax.persistence.database-product-name property must be
>>> specified. The value of this property should be the value returned for
>>> the target database by the JDBC DatabaseMetaData method
>>> getDatabaseProductName. If sufficient database version information is
>>> not included in the result of this method, the javax.persistence.
>>> database-major-version and javax.persistence.database-minor-version
>>> properties should be specified as needed. These should contain the
>>> values returned by the JDBC getDatabaseMajorVersion and
>>> getDatabaseMinorVersion methods respectively.
>>>
>>>
>>> * javax.persistence.schema-generation.create-script-source,
>>> * javax.persistence.schema-generation.drop-script-source
>>>
>>> The javax.persistence.schema-generation.create-script-source and
>>> javax.persistence.schema-generation.drop-script-source properties are
>>> used for script execution. In Java EE container environments, it is
>>> generally expected that the container will be responsible for
>>> executing DDL scripts, although the container is permitted to delegate
>>> this task to the persistence provider. If DDL scripts are to be used
>>> in Java SE environments or if the Java EE container delegates the
>>> execution of scripts to the persistence provider, these properties
>>> must be specified. The
>>> javax.persistence.schema-generation.create-script-source
>>> property specifies a java.IO.Reader configured for reading of the
>>> DDL script or
>>> a string designating a file URL for the DDL script.
>>>
>>> The javax.persistence.schema-generation.drop-script-source property
>>> specifies a java.IO.Reader configured for reading of the DDL script or
>>> a string designating a file URL for the DDL script.
>>>
>>>
>>> * javax.persistence.schema-generation.connection
>>>
>>> The javax.persistence.schema-generation.connection property specifies
>>> the JDBC connection to be used for schema generation. This is intended
>>> for use in Java EE environments, where the platform provider may want
>>> to control the database privileges that are available to the
>>> persistence provider. This connection is provided by the container,
>>> and should be closed by the container when the schema generation
>>> request or entity manager factory creation completes. The connection
>>> provided must have credentials sufficient for the persistence provider
>>> to carry out the requested actions. If this property is not specified,
>>> the persistence provider should use the DataSource that has otherwise
>>> been provided.
>>>
>>>
>>> [subsection 9.4.1] Data Loading
>>>
>>> Data loading, by means of the use of SQL scripts, may occur as part of
>>> the schema generation process after the creation of the database
>>> artifacts or independently of schema generation. The specification of
>>> the javax.persistence.sql-load-script-source controls whether data
>>> loading will occur.
>>>
>>> * javax.persistence.sql-load-script-source
>>>
>>> In Java EE container environments, it is generally expected that the
>>> container will be responsible for executing data load scripts,
>>> although the container is permitted to delegate this task to the
>>> persistence provider. If a load script is to be used in Java SE
>>> environments or if the Java EE container delegates the execution of
>>> the load script to the persistence provider, this property must be
>>> specified. The javax.persistence.sql-load-script-source property
>>> specifies a java.IO.Reader configured for reading of the SQL load
>>> script for database initialization or a string designating a file URL
>>> for the script.
>>>
>>>
>>>
>>>
>>>
>>> On 2/1/2013 7:23 AM, Steve Ebersole wrote:
>>>> I guess what I was getting at yesterday is that it *seems* as if
>>>> the intent is that
>>>> 'javax.persistence.ddl-create-script-target' and
>>>> 'javax.persistence.ddl-drop-script-target':
>>>> 1) need to be specified either both together or neither
>>>> 2) Control:
>>>> 2.a) whether scripts are generated
>>>> 2.b) what types of scripts are generated.
>>>>
>>>> So if I have:
>>>>
>>>> javax.persistence.schema-generation-action=create
>>>> javax.persistence.schema-generation-target=database-and-scripts
>>>> javax.persistence.ddl-create-script-target=<something>
>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>
>>>> the "action=create" bit really only describes the database target
>>>> side of things. On the script target side, both create
>>>> and drop are created.
>>>>
>>>> To me at least that is not the most intuitive set up. If I say
>>>> "action=create", why are the drop scripts being
>>>> generated?
>>>>
>>>>
>>>>
>>>> On Thu 31 Jan 2013 03:03:43 PM CST, Steve Ebersole wrote:
>>>>> Well the confusion IMO as a user is that I specifically asked for
>>>>> only
>>>>> creation scripts to be generated, but got both creation and drop
>>>>> scripts. Thats not going to change by making schema-generation-target
>>>>> required. Even this is still confusing:
>>>>>
>>>>> javax.persistence.schema-generation-action=create
>>>>> javax.persistence.schema-generation-target=scripts
>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>>
>>>>>
>>>>> because i would not expect to get a drop script. But if I read the
>>>>> other parts of the spec correctly, the following actually results in
>>>>> an exception:
>>>>>
>>>>> javax.persistence.schema-generation-action=create
>>>>> javax.persistence.schema-generation-target=scripts
>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>>
>>>>> The exception is because
>>>>> 'javax.persistence.ddl-drop-script-target' is
>>>>> not specified, again even though i just said I only want create
>>>>> scripts generated.
>>>>>
>>>>> On Thu 31 Jan 2013 02:13:20 PM CST, Linda DeMichiel wrote:
>>>>>>
>>>>>>
>>>>>> On 1/31/2013 11:52 AM, Steve Ebersole wrote:
>>>>>>> I also am not sure about this passage in discussing
>>>>>>> 'javax.persistence.ddl-create-script-target' and
>>>>>>> 'javax.persistence.ddl-drop-script-target':
>>>>>>>
>>>>>>> <quote>
>>>>>>> The persistence provider must produce both create and drop
>>>>>>> scripts if
>>>>>>> the corresponding
>>>>>>> DDL targets are specified. This is independent of whether a drop
>>>>>>> action is included in the
>>>>>>> value passed for the javax.persistence.schema-generation-action
>>>>>>> property.
>>>>>>> </quote>
>>>>>>>
>>>>>>> To me, this sounds like you are saying that the following config
>>>>>>> will
>>>>>>> generate both create and drop scripts:
>>>>>>>
>>>>>>> javax.persistence.schema-generation-action=create
>>>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>>>>
>>>>>>> even though the config explicitly specified just create scripts. Is
>>>>>>> that your intent?
>>>>>>>
>>>>>>
>>>>>> yes
>>>>>>
>>>>>>> Same for:
>>>>>>>
>>>>>>> javax.persistence.schema-generation-action=drop
>>>>>>> javax.persistence.ddl-create-script-target=<something>
>>>>>>> javax.persistence.ddl-drop-script-target=<something>
>>>>>>>
>>>>>>> that is supposed to generate both create and drop scripts?
>>>>>>>
>>>>>>
>>>>>> yes
>>>>>>
>>>>>>>
>>>>>>> Perhaps the reasoning is that
>>>>>>> javax.persistence.schema-generation-action is really just geared
>>>>>>> towards
>>>>>>> javax.persistence.schema-generation-action=database.
>>>>>>
>>>>>> partially. also it was intended to control whether any action would
>>>>>> be taken.
>>>>>>
>>>>>> I'm thinking that all this might be clearer if we required
>>>>>> schema-generation-target to be specified.
>>>>>>
>>>>>> But if that is the case, maybe we ought to rename the setting to be
>>>>>>> more explicit of that fact or at least document it in the
>>>>>>> discussion
>>>>>>> of 'javax.persistence.schema-generation-action'
>>>>>>>
>>>>>>>
>>>>>>> On 01/31/2013 01:13 PM, Steve Ebersole wrote:
>>>>>>>> One quick question.
>>>>>>>>
>>>>>>>> For 'javax.persistence.schema-generation-source' values of
>>>>>>>> metadata-then-scripts and scripts-then-metadata is the
>>>>>>>> order the same for both create and drop?
>>>>>>>>
>>>>>>>> I can envision a situation, for example, with
>>>>>>>> metadata-then-scripts
>>>>>>>> where the metadata creates a table and then the
>>>>>>>> script creates a procedure. I am not sure that all databases will
>>>>>>>> allow you to drop the table if other database
>>>>>>>> objects refer to it. Would it be better to say that the defined
>>>>>>>> order (x-then-y) applies to create, but the reverse
>>>>>>>> order (y-then-x) applies to drop?
>>>>>>>>
>>>>>>>>
>>>>>>>> On 01/31/2013 12:49 PM, Steve Ebersole wrote:
>>>>>>>>> These look great. Thanks Linda.
>>>>>>>>>
>>>>>>>>> On 01/28/2013 05:14 PM, Linda DeMichiel wrote:
>>>>>>>>>> There are currently a few open issues in schema generation
>>>>>>>>>> that need
>>>>>>>>>> clarification. These pertain to sections 9.4 ("Schema
>>>>>>>>>> generation")
>>>>>>>>>> and 8.2.1.9 ("properties") in the spec.
>>>>>>>>>>
>>>>>>>>>> I've made some clarifications and proposed some additions to the
>>>>>>>>>> metadata for schema generation as described below. The draft
>>>>>>>>>> I've
>>>>>>>>>> just uploaded reflects these changes
>>>>>>>>>> (http://java.net/projects/jpa-spec/downloads/download/JavaPersistence01282013.pdf).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please review these sections and post any feedback. Changes are
>>>>>>>>>> summarized below.
>>>>>>>>>>
>>>>>>>>>> ----
>>>>>>>>>>
>>>>>>>>>> Section 9.4
>>>>>>>>>>
>>>>>>>>>> This section isn't sufficiently clear about requirements for the
>>>>>>>>>> specification of schema-generation-action and
>>>>>>>>>> schema-generation-target.
>>>>>>>>>>
>>>>>>>>>> I've clarified that schema-generation-action must be
>>>>>>>>>> specified or no
>>>>>>>>>> actions will be taken. This was an assumed default in the
>>>>>>>>>> current
>>>>>>>>>> public draft, but IMO not sufficiently explicit.
>>>>>>>>>>
>>>>>>>>>> Given the clarification to schema-generation-action, I think
>>>>>>>>>> it now
>>>>>>>>>> makes sense for schema-generation-target to have default values.
>>>>>>>>>> That is, if script targets are specified by the properties for
>>>>>>>>>> scripts,
>>>>>>>>>> then generation actions should default into scripts. If not,
>>>>>>>>>> actions
>>>>>>>>>> should be directly into the database.
>>>>>>>>>>
>>>>>>>>>> The current public draft assumes that if script sources are
>>>>>>>>>> specified,
>>>>>>>>>> generation should occur from scripts exclusively; and
>>>>>>>>>> conversely, if
>>>>>>>>>> script sources are not specified, generation should occur
>>>>>>>>>> exclusively
>>>>>>>>>> from the ORM metadata. I've gotten feedback that this wasn't
>>>>>>>>>> sufficiently
>>>>>>>>>> flexible, and that we should support a combination of ORM and
>>>>>>>>>> script
>>>>>>>>>> sources. (For example, consider the case where generation should
>>>>>>>>>> occur
>>>>>>>>>> from scripts, but it is desirable to add stored procedures to
>>>>>>>>>> the
>>>>>>>>>> database.)
>>>>>>>>>> I therefore propose that we add a schema-generation-source
>>>>>>>>>> property
>>>>>>>>>> that will support such combination. Again, if script sources are
>>>>>>>>>> specified
>>>>>>>>>> by the properties for scripts, then generation will default to
>>>>>>>>>> use of
>>>>>>>>>> only script sources; if script sources aren't specified, then
>>>>>>>>>> generation
>>>>>>>>>> will default from the ORM metadata.
>>>>>>>>>>
>>>>>>>>>> I've also received feedback (from our JPA/EE integration
>>>>>>>>>> engineer)
>>>>>>>>>> that
>>>>>>>>>> we need to specify that in Java EE environments that strings
>>>>>>>>>> corresponding
>>>>>>>>>> to file URLs must use absolute (and not relative) paths. I've
>>>>>>>>>> added that
>>>>>>>>>> clarification as well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Section 8.2.1.9
>>>>>>>>>>
>>>>>>>>>> In the current public draft, this section lacks information
>>>>>>>>>> as to
>>>>>>>>>> how
>>>>>>>>>> script locations are specified. I've added a clarification that
>>>>>>>>>> scripts packaged as part of the application must be specified
>>>>>>>>>> relative
>>>>>>>>>> to the root of the persistence unit, as we do for other packaged
>>>>>>>>>> artifacts. I've also added a clarification about the
>>>>>>>>>> expectations
>>>>>>>>>> for
>>>>>>>>>> use of file URLs in Java EE environments as above.
>>>>>>>>>>
>>>>>>>>>> Finally, when I wrote this up initially, I was reluctant to add
>>>>>>>>>> metadata for schema generation actions (including targets)
>>>>>>>>>> because I
>>>>>>>>>> believed that such metadata should not be specific to the
>>>>>>>>>> persistence
>>>>>>>>>> unit definition itself. It seems however that people expect
>>>>>>>>>> that the
>>>>>>>>>> use of these properties (as specified in section 9.4) will be
>>>>>>>>>> available anyway, whether they are specified in the spec
>>>>>>>>>> document or
>>>>>>>>>> not, and that some vendors will support these anyway as well.
>>>>>>>>>> Further, this additional metadata may facilitate ease of use in
>>>>>>>>>> prototyping/development environments. I've therefore added these
>>>>>>>>>> properties into 8.2.1.9. If you disagree, please speak up.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>>
>>>>>>>>>> -Linda
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>