persistence@glassfish.java.net

Re: Direct DB updates are not visible in in JPA (even after clearing)

From: Adam Bien <abien_at_adam-bien.com>
Date: Thu, 17 Jul 2008 12:33:34 +0200

Hi Mitesh,

exactly what I did. I just replaced the TopLink Jars in the Glassfish
installation with the V2.1 branch - it worked perfectly.
The remaining question is: would be such combination officially
supported :). But this is another story :-),

thanks,

adam
Mitesh Meswani schrieb:
> Hi Adam,
>
> You should be able to get the fix using Toplink Essentials from V2.1
> branch. You can download it from here
> https://glassfish.dev.java.net/downloads/persistence/JavaPersistence.html
>
> -Mitesh
>
> Gordon Yorke wrote:
>> Hello Adam,
>> You will need the property "toplink.transaction.join-existing",
>> "true". You can set this on the persistence unit for the default or
>> only on the EntityManager that requires the special behaviour. This
>> setting will effectively disable the cache so you will also want to
>> set "toplink.cache.shared.<EntityName>" "false" for the entities
>> involved (or use shared.default for all entities.
>>
>> This property was added for this issue (
>> http://glassfish.dev.java.net/issues/show_bug.cgi?id=3334 ) almost a
>> year ago but may not be in 2ur2.
>> EclipseLink and therefore Glassfish V3 has this fix as well.
>> --Gordon
>>
>> Adam Bien wrote:
>>> Hi Gordon,
>>>
>>> thank you for the detailed explanation. I experimented with the
>>> following attributes:
>>>
>>> <property name="toplink.cache.shared.default"
>>> value="true|false"/>
>>> <property name="toplink.cache.type.default" value="WEAK|NONE"/>
>>>
>>> Hint: I'm using Glassfish v2ur2 with Toplink, so the eclipselink
>>> prefix do not apply :-)
>>>
>>> but with no effect. Direct changes to the database, in the same TX
>>> are still not visible for the EntityManager, even after clear. The
>>> pseudo code looks like this:
>>>
>>> @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
>>> @Stateless
>>> public class VehicleManagerBean implements VehicleManager {
>>>
>>> @PersistenceContext
>>> private EntityManager entityManager;
>>> @Resource(mappedName = "jdbc/db1")
>>> private DataSource dataSource;
>>>
>>>
>>> public Vehicle findChangeDBClearFind(int id, String newColor) {
>>> Vehicle vehicle = this.find(id);
>>> try {
>>> //changes the color of the vehicle directly in the DB
>>> this.changeVehiclesColor(id, newColor);
>>> } catch (Exception e) {
>>> throw new EJBException("Problem executing statement" + e,
>>> e);
>>> }
>>> this.entityManager.clear();
>>> //changes are not visible here
>>> return this.find(id);
>>> }
>>>
>>> public Vehicle find(int id) {
>>> return this.entityManager.find(Vehicle.class, id);
>>> }
>>>
>>> private void changeVehiclesColor(int id, String color) throws
>>> Exception {
>>> Connection connection = this.dataSource.getConnection();
>>> Statement statement = connection.createStatement();
>>> String sqlStatement = "update vehicle set color=\'" + color +
>>> "\' where id=" + id;
>>> System.out.println(sqlStatement);
>>> int updatedRecords = statement.executeUpdate(sqlStatement);
>>> if (updatedRecords != 1) {
>>> throw new IllegalStateException("Vehicle with id: " + id
>>> + " cannot be updated!");
>>> }
>>> connection.close();
>>> statement.close();
>>> }
>>>
>>> }
>>>
>>> In the persistence.xml the same DataSource is configured:
>>>
>>> <persistence-unit name="jpa" transaction-type="JTA">
>>> <jta-data-source>jdbc/db1</jta-data-source>
>>> <!-- properties etc. -->
>>> </persistence-unit>
>>> </persistence>
>>>
>>> I suspect either:
>>>
>>> - the DataSource does not participate in the local TX
>>> or
>>> - TopLink caches to aggressively.
>>>
>>>
>>> Should I file a bug? The same code runs on other appservers without
>>> any problems. It is really important, in case you are using e.g.
>>> DAOs and JPA in the same TX. This is actually a very common use
>>> case. I still hope I find a configuration to overcome this problem.
>>>
>>> Optimistic Locking does not really solve the problem, because the
>>> read / update operations are performed in the same TX...
>>>
>>> regards,
>>>
>>> adam
>>>
>>>
>>> Gordon Yorke schrieb:
>>>> This case would generally be managed in a few different ways. If
>>>> these updates occurred concurrently because of a third party
>>>> application then they would be managed through locking (optimistic)
>>>> and refreshing. If they occurred irregularly (monthly batch
>>>> process) then one of EclipseLink's invalidation policies can be
>>>> used to anticipate the bulk update. If the updates are part of the
>>>> application then the updates could be executed through JPQL and
>>>> EclipseLink would be aware of the updates and alter behaviour
>>>> accordingly. If it had to be SQL in the application then this
>>>> entity manager must be created with the config property
>>>> "eclipselink.transaction.join-existing", "true". If this update
>>>> is a small percentage of the interaction (reads included) with this
>>>> Entity Class then the shared cache for this class can be
>>>> invalidated by getting the ServerSession from the EntityManager
>>>> delegate. If it is the majority of interactions then class type
>>>> that is being updated should have it's cache set no shared cache.
>>>> "eclipselink.cache.shared.<entityName>", "false"
>>>> The vast majority of applications with appropriate locking and
>>>> refreshing policies have no conflict with the EclipseLink cache but
>>>> there are special cases for which we have developed configuration
>>>> options.
>>>> I hope this resolved your issue,
>>>> --Gordon
>>>>
>>>> Markus KARG wrote:
>>>>>
>>>>>> Forgot to mention essential information:
>>>>>>
>>>>>> The EntityManager uses the same DataSource as the connection
>>>>>> which updates the table...
>>>>> Sure, but DataSource is only a factory for JDBC connection
>>>>> instances -- not more. It can be implemented in a lot of ways (e.
>>>>> g. it can pool, or create new instances all the time, etc.) -- you
>>>>> have neither control about nor any guarantee. If it is implemented
>>>>> in a way that you get a new JDBC Connection instance, you'll get a
>>>>> new TX. So how to guarantee that EntityManager is using the same
>>>>> JDBC Connection instance? Since you cannot register a non-XA JDBC
>>>>> connection to use a preexisting TX (AFAIK only XADataSource has
>>>>> the needed methods to do that), you will end up with using the
>>>>> same DataSource (factory) but different TXs. If you need the same
>>>>> TX then I do not see another way than using XA (which, in turn,
>>>>> means using 2PC). Or is there a way to tell an EntityManager to
>>>>> use a preexisting JDBC Connection instance (have not checked the
>>>>> API, actually)?
>>>>>
>>>>> Besides that I think it should work to clear the JPA cache -- the
>>>>> O/R mapper is forced to reload from the DB then (what certainly is
>>>>> a major performance drawback and might not work depending on the
>>>>> actual TX isolation level).
>>>>>
>>>>> Have Fun
>>>>> Markus
>>>>>>
>>>>>> regards,
>>>>>>
>>>>>> adam
>>>>>> Adam Bien schrieb:
>>>>>>> Markus,
>>>>>>>
>>>>>>> I use both in the same TX. So using the same connection is the
>>>>>>> only way to keep the data consistent. Otherwise you would really
>>>>>>> have to use XA.
>>>>>>>
>>>>>>> On WLS 10 it worked without XA - the configuration was very
>>>>>>> similar... However the connection is only my suspicion.
>>>>>>> Aggresive caching could be the cause as well...
>>>>>>>
>>>>>>> regards,
>>>>>>>
>>>>>>> adam
>>>>>>> Markus KARG schrieb:
>>>>>>>> Adam,
>>>>>>>>
>>>>>>>> what makes you so certain that the EntityManager and the
>>>>>>>> DataSource are using technically the same JDBC Connection
>>>>>>>> instance (unless they will do, they will end up with two
>>>>>>>> different connections provided over the same physical link)? I
>>>>>>>> mean, unless you take explicit care for that, the EntityManager
>>>>>>>> will use a different JDBC Connection provided by the same
>>>>>>>> DataSource.
>>>>>>>>
>>>>>>>> Have Fun
>>>>>>>> Markus
>>>>>>>>
>>>>>>>> Adam Bien schrieb:
>>>>>>>>> Markus,
>>>>>>>>>
>>>>>>>>> you are right, but actually I do not want to have 2PC. I would
>>>>>>>>> expect, that the EntityManager as well as the DataSource would
>>>>>>>>> use exactly the *same* connection.
>>>>>>>>> Both are accessing the same resource - so 2PC / XA are
>>>>>>>>> actually not needed. The application server should not
>>>>>>>>> initiate a two-phase-commit protocol in this case.
>>>>>>>>>
>>>>>>>>> thank you for the link and help!,
>>>>>>>>>
>>>>>>>>> adam
>>>>>>>>>
>>>>>>>>> Markus KARG schrieb:
>>>>>>>>>> Adam,
>>>>>>>>>>
>>>>>>>>>> I guess that the problem is that you declared the connection
>>>>>>>>>> factory to use DataSource instead of XADataSource. AFAIK only
>>>>>>>>>> XADataSource can participate in distributed TX, while
>>>>>>>>>> DataSource always will use a local (different) exception. At
>>>>>>>>>> least this is what I have understood from the JDBC spec what
>>>>>>>>>> the main difference between DataSource and XADataSource is.
>>>>>>>>>> See
>>>>>>>>>> http://download.oracle.com/docs/cd/B19306_01/java.102/b14355/xadistra.htm#i1066469.
>>>>>>>>>> Can you try the same use case with "|OracleXADataSource| "
>>>>>>>>>> instead of "OracleDataSource" and "XADataSource" instead of
>>>>>>>>>> "DataSource"? Maybe that will help. :-)
>>>>>>>>>>
>>>>>>>>>> Have Fun
>>>>>>>>>> Markus
>>>>>>>>>>
>>>>>>>>>> Adam Bien schrieb:
>>>>>>>>>>> Forgot screenshot / configuration,
>>>>>>>>>>>
>>>>>>>>>>> adam
>>>>>>>>>>>
>>>>>>>>>>> Adam Bien schrieb:
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I ran into following problem: Direct changes in the
>>>>>>>>>>>> database (SQL updates) performed in the same Bean-method,
>>>>>>>>>>>> and so transaction, are not visible for the EntityManager
>>>>>>>>>>>> (subsequent find).
>>>>>>>>>>>>
>>>>>>>>>>>> Even invoking EntityManager#clear does not solve the
>>>>>>>>>>>> problem. The problem is strange, I suspect that the
>>>>>>>>>>>> DB-connection from the datasource does not participate in
>>>>>>>>>>>> the TX. This would explain such behaviour.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Server: Glassfishv2u2
>>>>>>>>>>>>
>>>>>>>>>>>> Database: OracleXE
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The Bean looks like:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> @Stateless
>>>>>>>>>>>>
>>>>>>>>>>>> @Remote(VehicleManager.class)
>>>>>>>>>>>>
>>>>>>>>>>>> @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
>>>>>>>>>>>>
>>>>>>>>>>>> public class VehicleManagerBean implements VehicleManager {
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> @PersistenceContext
>>>>>>>>>>>>
>>>>>>>>>>>> private EntityManager entityManager;
>>>>>>>>>>>>
>>>>>>>>>>>> @Resource(mappedName = "jdbc/db1")
>>>>>>>>>>>>
>>>>>>>>>>>> * private DataSource dataSource;*
>>>>>>>>>>>>
>>>>>>>>>>>> @EJB
>>>>>>>>>>>>
>>>>>>>>>>>> private DataService dataService;
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> public Vehicle findChangeDBClearFind(int id, String
>>>>>>>>>>>> newColor) {
>>>>>>>>>>>>
>>>>>>>>>>>> Vehicle vehicle = this.find(id);
>>>>>>>>>>>>
>>>>>>>>>>>> try {
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> //changes the color of the vehicle directly in
>>>>>>>>>>>> the DB
>>>>>>>>>>>>
>>>>>>>>>>>> * this.changeVehiclesColor(id, newColor);*
>>>>>>>>>>>>
>>>>>>>>>>>> * * } catch (Exception e) {
>>>>>>>>>>>>
>>>>>>>>>>>> throw new EJBException("Problem executing
>>>>>>>>>>>> statement" + e, e);
>>>>>>>>>>>>
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> this.entityManager.clear();
>>>>>>>>>>>>
>>>>>>>>>>>> return this.find(id);
>>>>>>>>>>>>
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> private void changeVehiclesColor(int id, String color)
>>>>>>>>>>>> throws Exception {
>>>>>>>>>>>>
>>>>>>>>>>>> Connection connection =
>>>>>>>>>>>> this.dataSource.getConnection();
>>>>>>>>>>>>
>>>>>>>>>>>> Statement statement = connection.createStatement();
>>>>>>>>>>>>
>>>>>>>>>>>> String sqlStatement = "update vehicle set color=\'"
>>>>>>>>>>>> + color + "\' where id=" + id;
>>>>>>>>>>>>
>>>>>>>>>>>> System.out.println(sqlStatement);
>>>>>>>>>>>>
>>>>>>>>>>>> int updatedRecords =
>>>>>>>>>>>> statement.executeUpdate(sqlStatement);
>>>>>>>>>>>>
>>>>>>>>>>>> if (updatedRecords != 1) {
>>>>>>>>>>>>
>>>>>>>>>>>> throw new IllegalStateException("Vehicle with
>>>>>>>>>>>> id: " + id + " cannot be updated!");
>>>>>>>>>>>>
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> connection.close();
>>>>>>>>>>>>
>>>>>>>>>>>> statement.close();
>>>>>>>>>>>>
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I deployed the same code to WLS 10 and it worked - the
>>>>>>>>>>>> changes were visible. I suppose it is probably a
>>>>>>>>>>>> configuration problem. I tried with different DataSource
>>>>>>>>>>>> configurations - with no effect. I attached my current
>>>>>>>>>>>> configuration as well (screenshot).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hopefully I overlooked something. If not - I will file it
>>>>>>>>>>>> as a bug.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you in advance,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> regards,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> adam bien
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>


-- 
 Consultant, Author, Java Champion
 
 Homepage: www.adam-bien.com
 Weblog: blog.adam-bien.com
 eMail:  abien_at_adam-bien.com
 Mobile: 0049(0)170 280 3144
 Books: Enterprise Architekturen (ISBN: 393504299X),
        Java EE 5 Architekturen  (ISBN: 3939084247),
        J2EE Patterns, J2EE Hotspots, Enterprise Frameworks and Struts