persistence@glassfish.java.net

Re: Direct DB updates are not visible in in JPA (even after clearing)

From: Adam Bien <abien_at_adam-bien.com>
Date: Wed, 16 Jul 2008 18:11:33 +0200

Hi Gordon,

Gordon Yorke schrieb:
> I took a look at the code in fisheye and this property is not
> available in v2ur2 but it is available in v2.1 and v3. You will have
> to ask the Sun folks about getting a patch into this stream.
> EclipseLink released 1.0 on July 9th and works well in Glassfish. You
> could run EclipseLink 1.0 as the persistence provider in Glassfish v2ur2.
Thank you very much for your support!

adam
> --Gordon
>
> Adam Bien wrote:
>> Hi, Gordon,
>>
>> thank you very much for the detailed explanation. I provided the
>> "toplink.transaction.join-existing", "true" setting, but the changes
>> are still not visible.
>> I disabled the cache already but it seems, like it is not enough. How
>> to find out whether this feature / fix is already implemented in GF
>> v2ur2?
>> Would it be possible to patch GF? From my perspective it is a serious
>> issue for projects using JPA persistence together with DAOs.
>>
>> regards,
>>
>> adam
>> Gordon Yorke schrieb:
>>> Hello Adam,
>>> You will need the property "toplink.transaction.join-existing",
>>> "true". You can set this on the persistence unit for the default or
>>> only on the EntityManager that requires the special behaviour. This
>>> setting will effectively disable the cache so you will also want to
>>> set "toplink.cache.shared.<EntityName>" "false" for the entities
>>> involved (or use shared.default for all entities.
>>>
>>> This property was added for this issue (
>>> http://glassfish.dev.java.net/issues/show_bug.cgi?id=3334 ) almost a
>>> year ago but may not be in 2ur2.
>>> EclipseLink and therefore Glassfish V3 has this fix as well.
>>> --Gordon
>>>
>>> Adam Bien wrote:
>>>> Hi Gordon,
>>>>
>>>> thank you for the detailed explanation. I experimented with the
>>>> following attributes:
>>>>
>>>> <property name="toplink.cache.shared.default"
>>>> value="true|false"/>
>>>> <property name="toplink.cache.type.default" value="WEAK|NONE"/>
>>>>
>>>> Hint: I'm using Glassfish v2ur2 with Toplink, so the eclipselink
>>>> prefix do not apply :-)
>>>>
>>>> but with no effect. Direct changes to the database, in the same TX
>>>> are still not visible for the EntityManager, even after clear. The
>>>> pseudo code looks like this:
>>>>
>>>> @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
>>>> @Stateless
>>>> public class VehicleManagerBean implements VehicleManager {
>>>>
>>>> @PersistenceContext
>>>> private EntityManager entityManager;
>>>> @Resource(mappedName = "jdbc/db1")
>>>> private DataSource dataSource;
>>>>
>>>>
>>>> public Vehicle findChangeDBClearFind(int id, String newColor) {
>>>> Vehicle vehicle = this.find(id);
>>>> try {
>>>> //changes the color of the vehicle directly in the DB
>>>> this.changeVehiclesColor(id, newColor);
>>>> } catch (Exception e) {
>>>> throw new EJBException("Problem executing statement" +
>>>> e, e);
>>>> }
>>>> this.entityManager.clear();
>>>> //changes are not visible here
>>>> return this.find(id);
>>>> }
>>>>
>>>> public Vehicle find(int id) {
>>>> return this.entityManager.find(Vehicle.class, id);
>>>> }
>>>>
>>>> private void changeVehiclesColor(int id, String color) throws
>>>> Exception {
>>>> Connection connection = this.dataSource.getConnection();
>>>> Statement statement = connection.createStatement();
>>>> String sqlStatement = "update vehicle set color=\'" + color
>>>> + "\' where id=" + id;
>>>> System.out.println(sqlStatement);
>>>> int updatedRecords = statement.executeUpdate(sqlStatement);
>>>> if (updatedRecords != 1) {
>>>> throw new IllegalStateException("Vehicle with id: " + id
>>>> + " cannot be updated!");
>>>> }
>>>> connection.close();
>>>> statement.close();
>>>> }
>>>>
>>>> }
>>>>
>>>> In the persistence.xml the same DataSource is configured:
>>>>
>>>> <persistence-unit name="jpa" transaction-type="JTA">
>>>> <jta-data-source>jdbc/db1</jta-data-source>
>>>> <!-- properties etc. -->
>>>> </persistence-unit>
>>>> </persistence>
>>>>
>>>> I suspect either:
>>>>
>>>> - the DataSource does not participate in the local TX
>>>> or
>>>> - TopLink caches to aggressively.
>>>>
>>>>
>>>> Should I file a bug? The same code runs on other appservers without
>>>> any problems. It is really important, in case you are using e.g.
>>>> DAOs and JPA in the same TX. This is actually a very common use
>>>> case. I still hope I find a configuration to overcome this problem.
>>>>
>>>> Optimistic Locking does not really solve the problem, because the
>>>> read / update operations are performed in the same TX...
>>>>
>>>> regards,
>>>>
>>>> adam
>>>>
>>>>
>>>> Gordon Yorke schrieb:
>>>>> This case would generally be managed in a few different ways. If
>>>>> these updates occurred concurrently because of a third party
>>>>> application then they would be managed through locking
>>>>> (optimistic) and refreshing. If they occurred irregularly
>>>>> (monthly batch process) then one of EclipseLink's invalidation
>>>>> policies can be used to anticipate the bulk update. If the
>>>>> updates are part of the application then the updates could be
>>>>> executed through JPQL and EclipseLink would be aware of the
>>>>> updates and alter behaviour accordingly. If it had to be SQL in
>>>>> the application then this entity manager must be created with the
>>>>> config property "eclipselink.transaction.join-existing", "true".
>>>>> If this update is a small percentage of the interaction (reads
>>>>> included) with this Entity Class then the shared cache for this
>>>>> class can be invalidated by getting the ServerSession from the
>>>>> EntityManager delegate. If it is the majority of interactions
>>>>> then class type that is being updated should have it's cache set
>>>>> no shared cache. "eclipselink.cache.shared.<entityName>", "false"
>>>>> The vast majority of applications with appropriate locking and
>>>>> refreshing policies have no conflict with the EclipseLink cache
>>>>> but there are special cases for which we have developed
>>>>> configuration options.
>>>>> I hope this resolved your issue,
>>>>> --Gordon
>>>>>
>>>>> Markus KARG wrote:
>>>>>>
>>>>>>> Forgot to mention essential information:
>>>>>>>
>>>>>>> The EntityManager uses the same DataSource as the connection
>>>>>>> which updates the table...
>>>>>> Sure, but DataSource is only a factory for JDBC connection
>>>>>> instances -- not more. It can be implemented in a lot of ways (e.
>>>>>> g. it can pool, or create new instances all the time, etc.) --
>>>>>> you have neither control about nor any guarantee. If it is
>>>>>> implemented in a way that you get a new JDBC Connection instance,
>>>>>> you'll get a new TX. So how to guarantee that EntityManager is
>>>>>> using the same JDBC Connection instance? Since you cannot
>>>>>> register a non-XA JDBC connection to use a preexisting TX (AFAIK
>>>>>> only XADataSource has the needed methods to do that), you will
>>>>>> end up with using the same DataSource (factory) but different
>>>>>> TXs. If you need the same TX then I do not see another way than
>>>>>> using XA (which, in turn, means using 2PC). Or is there a way to
>>>>>> tell an EntityManager to use a preexisting JDBC Connection
>>>>>> instance (have not checked the API, actually)?
>>>>>>
>>>>>> Besides that I think it should work to clear the JPA cache -- the
>>>>>> O/R mapper is forced to reload from the DB then (what certainly
>>>>>> is a major performance drawback and might not work depending on
>>>>>> the actual TX isolation level).
>>>>>>
>>>>>> Have Fun
>>>>>> Markus
>>>>>>>
>>>>>>> regards,
>>>>>>>
>>>>>>> adam
>>>>>>> Adam Bien schrieb:
>>>>>>>> Markus,
>>>>>>>>
>>>>>>>> I use both in the same TX. So using the same connection is the
>>>>>>>> only way to keep the data consistent. Otherwise you would
>>>>>>>> really have to use XA.
>>>>>>>>
>>>>>>>> On WLS 10 it worked without XA - the configuration was very
>>>>>>>> similar... However the connection is only my suspicion.
>>>>>>>> Aggresive caching could be the cause as well...
>>>>>>>>
>>>>>>>> regards,
>>>>>>>>
>>>>>>>> adam
>>>>>>>> Markus KARG schrieb:
>>>>>>>>> Adam,
>>>>>>>>>
>>>>>>>>> what makes you so certain that the EntityManager and the
>>>>>>>>> DataSource are using technically the same JDBC Connection
>>>>>>>>> instance (unless they will do, they will end up with two
>>>>>>>>> different connections provided over the same physical link)? I
>>>>>>>>> mean, unless you take explicit care for that, the
>>>>>>>>> EntityManager will use a different JDBC Connection provided by
>>>>>>>>> the same DataSource.
>>>>>>>>>
>>>>>>>>> Have Fun
>>>>>>>>> Markus
>>>>>>>>>
>>>>>>>>> Adam Bien schrieb:
>>>>>>>>>> Markus,
>>>>>>>>>>
>>>>>>>>>> you are right, but actually I do not want to have 2PC. I
>>>>>>>>>> would expect, that the EntityManager as well as the
>>>>>>>>>> DataSource would use exactly the *same* connection.
>>>>>>>>>> Both are accessing the same resource - so 2PC / XA are
>>>>>>>>>> actually not needed. The application server should not
>>>>>>>>>> initiate a two-phase-commit protocol in this case.
>>>>>>>>>>
>>>>>>>>>> thank you for the link and help!,
>>>>>>>>>>
>>>>>>>>>> adam
>>>>>>>>>>
>>>>>>>>>> Markus KARG schrieb:
>>>>>>>>>>> Adam,
>>>>>>>>>>>
>>>>>>>>>>> I guess that the problem is that you declared the connection
>>>>>>>>>>> factory to use DataSource instead of XADataSource. AFAIK
>>>>>>>>>>> only XADataSource can participate in distributed TX, while
>>>>>>>>>>> DataSource always will use a local (different) exception. At
>>>>>>>>>>> least this is what I have understood from the JDBC spec what
>>>>>>>>>>> the main difference between DataSource and XADataSource is.
>>>>>>>>>>> See
>>>>>>>>>>> http://download.oracle.com/docs/cd/B19306_01/java.102/b14355/xadistra.htm#i1066469.
>>>>>>>>>>> Can you try the same use case with "|OracleXADataSource| "
>>>>>>>>>>> instead of "OracleDataSource" and "XADataSource" instead of
>>>>>>>>>>> "DataSource"? Maybe that will help. :-)
>>>>>>>>>>>
>>>>>>>>>>> Have Fun
>>>>>>>>>>> Markus
>>>>>>>>>>>
>>>>>>>>>>> Adam Bien schrieb:
>>>>>>>>>>>> Forgot screenshot / configuration,
>>>>>>>>>>>>
>>>>>>>>>>>> adam
>>>>>>>>>>>>
>>>>>>>>>>>> Adam Bien schrieb:
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I ran into following problem: Direct changes in the
>>>>>>>>>>>>> database (SQL updates) performed in the same Bean-method,
>>>>>>>>>>>>> and so transaction, are not visible for the EntityManager
>>>>>>>>>>>>> (subsequent find).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Even invoking EntityManager#clear does not solve the
>>>>>>>>>>>>> problem. The problem is strange, I suspect that the
>>>>>>>>>>>>> DB-connection from the datasource does not participate in
>>>>>>>>>>>>> the TX. This would explain such behaviour.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Server: Glassfishv2u2
>>>>>>>>>>>>>
>>>>>>>>>>>>> Database: OracleXE
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The Bean looks like:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Stateless
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Remote(VehicleManager.class)
>>>>>>>>>>>>>
>>>>>>>>>>>>> @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
>>>>>>>>>>>>>
>>>>>>>>>>>>> public class VehicleManagerBean implements VehicleManager {
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> @PersistenceContext
>>>>>>>>>>>>>
>>>>>>>>>>>>> private EntityManager entityManager;
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Resource(mappedName = "jdbc/db1")
>>>>>>>>>>>>>
>>>>>>>>>>>>> * private DataSource dataSource;*
>>>>>>>>>>>>>
>>>>>>>>>>>>> @EJB
>>>>>>>>>>>>>
>>>>>>>>>>>>> private DataService dataService;
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> public Vehicle findChangeDBClearFind(int id, String
>>>>>>>>>>>>> newColor) {
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vehicle vehicle = this.find(id);
>>>>>>>>>>>>>
>>>>>>>>>>>>> try {
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> //changes the color of the vehicle directly in
>>>>>>>>>>>>> the DB
>>>>>>>>>>>>>
>>>>>>>>>>>>> * this.changeVehiclesColor(id, newColor);*
>>>>>>>>>>>>>
>>>>>>>>>>>>> * * } catch (Exception e) {
>>>>>>>>>>>>>
>>>>>>>>>>>>> throw new EJBException("Problem executing
>>>>>>>>>>>>> statement" + e, e);
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> this.entityManager.clear();
>>>>>>>>>>>>>
>>>>>>>>>>>>> return this.find(id);
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> private void changeVehiclesColor(int id, String color)
>>>>>>>>>>>>> throws Exception {
>>>>>>>>>>>>>
>>>>>>>>>>>>> Connection connection =
>>>>>>>>>>>>> this.dataSource.getConnection();
>>>>>>>>>>>>>
>>>>>>>>>>>>> Statement statement = connection.createStatement();
>>>>>>>>>>>>>
>>>>>>>>>>>>> String sqlStatement = "update vehicle set color=\'"
>>>>>>>>>>>>> + color + "\' where id=" + id;
>>>>>>>>>>>>>
>>>>>>>>>>>>> System.out.println(sqlStatement);
>>>>>>>>>>>>>
>>>>>>>>>>>>> int updatedRecords =
>>>>>>>>>>>>> statement.executeUpdate(sqlStatement);
>>>>>>>>>>>>>
>>>>>>>>>>>>> if (updatedRecords != 1) {
>>>>>>>>>>>>>
>>>>>>>>>>>>> throw new IllegalStateException("Vehicle with
>>>>>>>>>>>>> id: " + id + " cannot be updated!");
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> connection.close();
>>>>>>>>>>>>>
>>>>>>>>>>>>> statement.close();
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I deployed the same code to WLS 10 and it worked - the
>>>>>>>>>>>>> changes were visible. I suppose it is probably a
>>>>>>>>>>>>> configuration problem. I tried with different DataSource
>>>>>>>>>>>>> configurations - with no effect. I attached my current
>>>>>>>>>>>>> configuration as well (screenshot).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hopefully I overlooked something. If not - I will file it
>>>>>>>>>>>>> as a bug.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you in advance,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> adam bien
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>