## Berkeley DB Java Edition 4.0.103 Change Log

### Changes in 4.0.103

### Features

1. Added the LocalCBVLSN field for replication group members to the information displayed by `com.sleepycat.je.rep.util.DbGroupAdmin -dumpGroup` or through the JMX MBean dumpReplicationState() operation. The LocalCBVLSN field is an indicator of the progress a replicated node has made in executing replicated operations.

   Users can monitor the LocalCBVLSN values for group members as a way of gauging if the group is executing at an even pace. The values need not be identical, because they are only update periodically, but a large or growing discrepancy indicates that some group members are lagging.[#18430]

2. Several new `com.sleepycat.je.CacheMode` enumeration values have been added which can help to improve cache memory usage and reduce Java GC costs for some applications:

   - `MAKE_COLD`
   - `EVICT_LN`
   - `EVICT_BIN`

   The `CacheMode` is configurable using new database and environment properties:

   - `DatabaseConfig.setCacheMode, getCacheMode`
   - `EnvironmentConfig.setCacheMode, getCacheMode`

   Note that the `MAKE_COLD` mode has been enhanced since its introduction in JE 3.3. In JE 4.0, this mode will perform explicit eviction when the JE cache is full, and blocking between threads may be reduced.

   See the `CacheMode` Javadoc for more information. [#18410]

3. Several new statistics have been added `com.sleepycat.je.EnvironmentStats` to support the new `CacheMode` functionality and to make it easier to understand caching behavior. [#18512]

   - `nCachedBINs and nCachedUpperINs` show the proportion types of btree nodes in cache.
   - `nBINsFetch, nBINsFetchMiss, nUpperINsFetch, nUpperINsFetchMiss` provide a more detailed look at cache hit/miss ratio
   - `nBINsEvictedCACHEMODE, nBINsEvictedCRITICAL, nBINsEvictedMANUAL, nBINsEvictedEVICTORTHREAD, nUpperINsEvictedCACHEMODE, nUpperINsEvictedCRITICAL, nUpperINsEvictedMANUAL, nUpperINsEvictedEVICTORTHREAD` provide some guidance about the effectiveness of different CacheModes.
4. Two new properties: `com.sleepycat.je.rep.ReplicationConfig.REPLICA_TIMEOUT` and `com.sleepycat.je.rep.ReplicationConfig.FEEDER_TIMEOUT` have been defined to permit application level control of connection timeouts. (4.0.100)

### General Changes

1. Fixed a bug which can cause `Database.count()` to return incorrect values when an environment is opened read-only. [#18180]
2. JE applications run on the Resin application server could experience a NullPointerException at startup. The problem occurs because Resin's implementation of java.util.logging.Loggers.getHandlers() can return null instead of an empty array. A check for a null return was added. The problem was originally reported on the OTN forum. [#18277]
3. Opening a read-only environment with a non default value for the com.sleepycat.je.util.FileHandler.level property resulted in a NullPointerException. In addition, output to the je.info file was disabled for read only environments. This has been fixed. [#18334]
4. Fixed a bug in `Database.preload()` which would cause a latch on a database root node to be held in the face of the cache filling or preload taking too much time. [#18396]
5. Fixed a bug where the cache usage was not correctly calculated when using the `-XX:+UseCompressedOops` JVM option. [#18462]
6. Replicated applications running with the property `je.cleaner.threads` set to a value greater than 1, where the application does not fit in cache, can experience a deadlock which shows the following stacktraces. This has been fixed. [#18475]

       "Cleaner-1":
       at com.sleepycat.je.cleaner.UtilizationProfile.putFileSummary(UtilizationProfile.java:846)
       - waiting to lock <0xf00db530> (a com.sleepycat.je.cleaner.UtilizationProfile)
       at com.sleepycat.je.cleaner.UtilizationProfile.flushFileSummary(UtilizationProfile.java:835)
       at com.sleepycat.je.cleaner.UtilizationTracker.evictMemory(UtilizationTracker.java:95)
       at com.sleepycat.je.dbi.EnvironmentImpl.specialEviction(EnvironmentImpl.java:2379)
       at com.sleepycat.je.evictor.PrivateEvictor.startBatch(PrivateEvictor.java:96)
       at com.sleepycat.je.evictor.Evictor.evictBatch(Evictor.java:306)
       at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:245)
       - locked <0xf0222028> (a com.sleepycat.je.evictor.PrivateEvictor)
       at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:273)
       at com.sleepycat.je.dbi.EnvironmentImpl.criticalEviction(EnvironmentImpl.java:2360)
       at com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:502)

         ...

       Cleaner-4:
       at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:229)
       - waiting to lock <0xf0222028> (a com.sleepycat.je.evictor.PrivateEvictor)
       at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:273)
       at com.sleepycat.je.dbi.EnvironmentImpl.criticalEviction(EnvironmentImpl.java:2360)
       at com.sleepycat.je.dbi.CursorImpl.criticalEviction(CursorImpl.java:295)
       at com.sleepycat.je.Cursor.beginMoveCursor(Cursor.java:2704)
         ...
       at com.sleepycat.je.rep.vlsn.VLSNIndex.getLTEFileNumber(VLSNIndex.java:675)
       at com.sleepycat.je.rep.impl.node.GlobalCBVLSN.getCleanerBarrierFile(GlobalCBVLSN.java:100)
       at com.sleepycat.je.rep.impl.node.RepNode.getCleanerBarrierFile(RepNode.java:1287)
       at com.sleepycat.je.rep.impl.RepImpl.getCleanerBarrierStartFile(RepImpl.java:1156)
       at com.sleepycat.je.cleaner.UtilizationProfile.getBestFileForCleaning(UtilizationProfile.java:289)
       - locked <0xf00db530> (a com.sleepycat.je.cleaner.UtilizationProfile)
       at com.sleepycat.je.cleaner.FileSelector.selectFileForCleaning(FileSelector.java:225)
       at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:202)
       - locked <0xf0035cd0> (a com.sleepycat.je.cleaner.FileProcessor)
       at
       	    com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:140)

7. Replicated applications with a high level of cache eviction can experience a deadlock where the checkpointer thread displays the following type of stacktrace. This has been fixed. [#18475]

       "Checkpointer" daemon prio=3 tid=0x08518000 nid=0x1d waiting on condition [0xedca8000]
       - parking to wait for  <0xf0211680> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
       at com.sleepycat.je.tree.IN.latch(IN.java:332)
                 ...
       at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:245)
       - locked <0xf01809c0> (a com.sleepycat.je.evictor.PrivateEvictor)
                 ...
       at com.sleepycat.je.rep.vlsn.VLSNTracker.flushToDatabase(VLSNTracker.java:366)
       - locked <0xf018e510> (a com.sleepycat.je.rep.vlsn.VLSNTracker)
                 ...
       at com.sleepycat.je.rep.impl.RepImpl.preCheckpointEndFlush(RepImpl.java:656)
       at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:490)

8. Fixed a bug which caused an IllegalArgumentException to be thrown when parsing various JE parameters if the default language was not English (e.g.if it was set to Turkish). Thanks to "Mike" on OTN for reporting this. [#18504]
9. Fixed a bug in `DbCacheSize` that caused a slight overestimation of memory used by the Btree. This utility now prints out detailed memory usage for each level in the Btree. [#18520]
10. A replicated environment could very rarely experience this exception, which was due to an incorrect assertion check. The assertion has been fixed. [#18524]

        com.sleepycat.je.EnvironmentFailureException: (JE 4.0.97) Locker:
        com.sleepycat.je.txn.BasicLocker UNEXPECTED_STATE: Unexpected internal
        state, may have side effects.

        at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:376)
        at com.sleepycat.je.txn.LockManager.lock(LockManager.java:262)
        at com.sleepycat.je.txn.BasicLocker.lockInternal(BasicLocker.java:134)
        at com.sleepycat.je.txn.Locker.nonBlockingLock(Locker.java:487)
        at com.sleepycat.je.tree.MapLN.isEvictable(MapLN.java:174)
        at com.sleepycat.je.tree.BIN.evictInternal(BIN.java:1004)
        at com.sleepycat.je.tree.BIN.evictLNs(BIN.java:965)
        at com.sleepycat.je.evictor.Evictor.evictIN(Evictor.java:820)
           ...

11. Fixed a bug where a Replica could fail to detect that it had lost its connection to the Master when its network cable was unplugged. As a result, an election would not be initiated by the Replica. [#18554]
12. The directions in com.sleepycat.je.rep.RollbackProhibitedException and com.sleepycat.je.util.DbTruncateLog were not clear enough about how to handle a RollbackProhibitedException, and have been improved. DbTruncateLog itself had a bug which made it quietly skip the log truncation. [#18558].
13. Fixed a bug that causes an internal `RefreshException` to be thrown when calling `com.sleepycat.persist.EntityStore.getPrimaryIndex`. This occurs only under these conditions:
    - on a Replica node in a replicated environment, and
    - if the primary index is created on the Master after the entity store has been opened on the Replica.
    [#18594] (4.0.101)
14. Fixed a bug that causes an `NullPointerException` to be thrown when calling `com.sleepycat.persist.EntityStore.getPrimaryIndex`. This occurs only under these conditions:
    - on a Replica node in a replicated environment, and
    - if the primary key uses a sequence, and
    - if the primary index has not been created on the Master, and
    - if another primary index with a sequence has been previously created on the Master.
    [#18594] (4.0.102)
15. Fix a bug that causes the following exception under rare conditions when calling Database.sync for a deferred write database.

         java.lang.IndexOutOfBoundsException
         at com.sleepycat.bind.tuple.TupleInput.readBoolean(TupleInput.java:186)
         at com.sleepycat.je.cleaner.PackedObsoleteInfo.countObsoleteInfo(PackedObsoleteInfo.java:60)
         at com.sleepycat.je.log.LogManager.serialLogInternal(LogManager.java:671)
         at com.sleepycat.je.log.SyncedLogManager.serialLog(SyncedLogManager.java:40)
         at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:388)
         at com.sleepycat.je.recovery.Checkpointer.logSiblings(Checkpointer.java:1285)

    [#18567] (4.0.102)

### Changes in 4.0.92

### Features

1. Normal operation of JE HA requires that at least a simple majority of electable nodes be available to form a quorum for election of a new Master, or when committing a transaction with default durability requirements. The number of electable nodes (the Electable Group Size) is obtained from persistent internal metadata that is stored in the environment and replicated across all members. See Replication Group Life Cycle for details.

   Under exceptional circumstances, a simple majority of nodes may become unavailable for some period of time. With only a minority of nodes available, the overall availability of the group can be adversely affected.

   To deal with this exceptional circumstance, especially if the situation is likely to persist for an unacceptably long period of time, a mechanism has been added by which you can modify the way in which the number of electable nodes, and consequently the quorum requirements for elections and commit acknowledgments, is calculated. See `ReplicationMutableConfig.ELECTABLE_GROUP_SIZE_OVERRIDE` and Replication Guide appendix, Managing a Failure of the Majority [#18022]

2. A number of improvements have been made to the JE Dynamic MBeans.
   - JEMonitor has a new operation, getEnvConfig() which dumps the configuration of the running environment.
   - Replication network throughput information is now presented in an easier to understand bytes or messages per second format.
   - The Monitoring JE with JMX and JConsole how-to has been expanded.
   - It is now possible to display and export incremental, rather than cumulative stats though the JConsole plugin, making it easier to detect trends in the statistics display.
   [#17913], [#18194]

### API Changes

1. Added `getGroupName()` to `com.sleepycat.je.rep.util.ReplicationGroupAdmin`, which returns the group name. `ReplicationGroupAdmin.getMasterSocket()` was inadvertently public and has been made private. [#17841]
2. The following methods in `com.sleepycat.je.rep.ReplicatedEnvironmentStats`:

   > getProtocolAvgReadNanos()
   > getProtocolAvgWriteNanos()
   > getProtocolReadNanos()
   > getProtocolWriteNanos()

   were intended to provide throughput information for a replicated JE environment, but required some computation from the user. They have been replaced with methods that present throughput in a more forthright manner:

   > getProtocolBytesReadRate()
   > getProtocolBytesWriteRate()
   > getProtocolMessageReadRate()
   > getProtocolMessageWriteRate()

   [#17913]
3. A replication group can rollback a committed transaction in rare circumstances, if there is network partitioning or there are master failures when transactions have been committed with the type of durability that requires no replica acknowledgments. In such cases, a node which was previously a master may have transactions committed to disk which were not propagated to the group and which are not contained by the current master. By default, JE handles this case transparently. A new property has been added to limit the number of transactions that can be automatically rolled back, as a safeguard. See `com.sleepycat.je.rep.ReplicationConfig.TXN_ROLLBACK_LIMIT` and the new `com.sleepycat.je.rep.RollbackProhibitedException`. [#17926]

4. Several new classes and methods, and a bug fix were added to make it easier to track replication group changes when using the com.sleepycat.je.rep.monitor package.

   `com.sleepycat.je.rep.monitor.GroupChangeEvent` was being generated whenever an electable node joined a replication group. Instead, this event should only be generated when the replication group composition changes.

   The class `com.sleepycat.je.rep.monitor.GroupChangeEvent` now defines a new enumeration: `GroupChangeType` and a new method `GroupChangeEvent.getChangeType()`. Together these changes make it possible to identify the type of structural change (whether a node was added or removed) and the affected node.

   The class `com.sleepycat.je.rep.monitor.MonitorChangeListener` now defines two new notify methods: `MonitorChangeListener.notify(JoinGroupEvent)` and `MonitorChangeListener.notify(LeaveGroupEvent)`. These new methods, along with the new events: `com.sleepycat.je.rep.monitor.JoinGroupEvent` and `com.sleepycat.je.rep.monitor.LeaveGroupEvent`, permit the listener to track the dynamic state of electable nodes as they join and leave the replication group. You will need to supply implementations for the new notify methods if you have an existing class that implements the `MonitorChangeListener` interface.

   This problem was originally reported here on the OTN forum. [#18006]

5. It is our intent to let the JE user set java.util.logging properties programmatically. JE logging output is most easily specified by setting the `com.sleepycat.je.util.ConsoleHandler.level` and `com.sleepycat.je.util.FileHandler.level` through the standard java.util.logging configuration file or MBean, but the java.util.logging API does not directly support setting handler levels. JE now accepts the properties `com.sleepycat.je.util.ConsoleHandler.level` and `com.sleepycat.je.util.FileHandler.level` as properties to the je.properties file or the EnvironmentConfig.setConfigParam() method to support programmatic setting of these values. See the Logging chapter for more information. [#18017]
6. The maximum value for log file size (je.log.fileMax) on the Android platform was increased from 1MB to 10MB.

### General Changes

1. `Database.preload(PreloadConfig)` no longer throws `NullPointerException` if null is passed for an argument. Instead, it uses a default `PreloadConfig`. [#17784] (4.0.72)
2. Fixed a bug on Android that appears as a NullPointerException when using a SerialBinding. A workaround was added for a Harmony (Dalvik) bug that occurs when the ObjectInputStream.readClassDescriptor is overridden. [#17903]
3. In certain cases, the JE MBeans did not return exception information correctly. For example, calling getDatabaseStats() operation with an invalid database name would return this message:

   > `java.rmi.UnmarshalException: Error unmarshaling return; nested exception is java.lang.ClassNotFoundException: com.sleepycat.je.DatabaseNotFoundException ....`

   instead of a proper message explaining the problem. This has been fixed. [#17913]
4. Fixed an inefficiency which resulted in extra computation on all transactional operations when there are multiple lock tables. The inefficiency became noticeable when using hundreds of lock tables. [#17944]
5. Fixed a performance issue during highly concurrent read operations by hundreds of threads, caused by contention on a `java.util.concurrent.locks.ReentrantReadWriteLock` taken during construction of a `ThreadLocal`. The ThreadLocal is only used during stats collection, and the implementation now makes initialization lazy and conditional. [#17944]
6. Fixed a problem with `Database.close` when the `EnvironmentConfig.ENV_IS_LOCKING` property is set to `false`. This caused `Environment.truncateDatabase`, `removeDatabase` and `renameDatabase` to throw `LockTimeoutException` when the database is opened and closed prior to performing the truncate, remove or rename operation. It also caused a resource/memory leak (a single lock was not released by `Database.close`) that could impact applications that frequently open and close databases. Again, this only impacts applications that have explicitly set `EnvironmentConfig.ENV_IS_LOCKING` to false. [#17985]
7. Fixed a bug that caused `PreloadStats.getNLNsLoaded` to return a non-zero number, even when `PreloadConfig.setLoadLNs(false)` was specified. Note that no LNs were loaded -- only the returned stat was incorrect. [#18021]
8. A minor improvement was made to JE cache management. The calculated amount of memory will be slightly more accurate, especially for applications that write with random key values. The change could, in principle, prevent an OutOfMemoryError, but the improvement is small enough that it is unlikely to have a noticeable impact on most applications. [#18035]
9. Fixed an issue on Android where ClassNotFoundException could sometimes be returned. Possible cases are when a custom key comparator, or a SerialBinding is used. The problem was caused by a bug in the Harmony (Dalvik) implementation of Class.forName() when used with the thread's context class loader. [#18163]
10. Fixed a bug on Android where a full Checkpoint or Environment.sync() might cause later errors to appear. This problem was caused by a bug in the Harmony (Dalvik) implementation of java.util.IdentityHashMap which is described in http://issues.apache.org/jira/browse/HARMONY-6419. [#18167]
11. The JConsole plugin would exit if an non-numeric value was entered in the graphing interval text box. This has been corrected to instead return an exception message. [#18194]
12. Fixed a problem in JE HA where in the unusual case of a node transitioning from a Master to a Replica state, the node could sometimes fail with an com.sleepycat.je.EnvironmentFailureException as it tried to establish a replication stream with the new master. A sample stack trace is enclosed below:

        Replica unexpected exception com.sleepycat.je.EnvironmentFailureException: ...
        com.sleepycat.je.EnvironmentFailureException: (JE 4.0.80)
        node4(3):/tmp/scaleDir4/env Rep stream not sequential.
        Current VLSN: 94,338,473 next log entry VLSN: 94,338,515
        UNEXPECTED_STATE: Unexpected internal state, may have side effects.
         at com.sleepycat.je.rep.impl.node.Replay.replayEntry(Replay.java:342)
         at com.sleepycat.je.rep.impl.node.Replica.doRunReplicaLoopInternalWork(Replica.java:433)
         at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:353)
         at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:295)
         at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:914)

    [#18212]
13. Fixed a problem where cleaned and deleted log files could accumulate in the log cleaner's backlog, or list of files to be cleaned. This occurs when multiple cleaner threads are configured. The impacts of this problem are:
    1. The `EnvironmentStats.getCleanerBacklog` stat is incorrect, which could lead the application to unnecessarily increase the number of cleaner threads.
    3. If `EnvironmentConfig.CLEANER_MAX_BATCH_FILES` is set to a non-zero value, log cleaning is disabled when the number of deleted files in the backlog reaches this limit.
    [#18179]
14. Fixed a bug that can cause LogFileNotFound (loss of log information) when using temporary databases (DatabaseConfig.setTemporary(true) or StoreConfig.setTemporary(true)). This is known to occur only under heavy concurrent cache eviction and log cleaning. [#18227]

### Utility Changes:

1. Added a utility named com.sleepycat.je.rep.util.DbGroupAdmin which presents the functionality of the com.sleepycat.je.rep.util.ReplicationGroupAdmin class in a command line utility. It lets the user both display the composition of a replication group and remove nodes from the group. DbGroupAdmin should be used as part of JE HA configuration checklist. [#17841]
2. `DbCacheSize` now supports `-measure` to measure the actual amount of cache size used, without the use of `-data`. When `-data` is not specified, `-measure` will cause only the keys and internal nodes to be kept in cache. This allows measuring the amount of memory necessary to hold the internal nodes for a data set, without allocating enough memory to hold the entire data set, including leaf nodes. [#18036]

### Examples:

1. Added a new example to illustrate the use of RMI for request forwarding in a replicated system. See the javadoc for je.rep.quote.StockQuotesRMIForwarding.

### Changes in JE 4.0.71

File Format
New Features
API
Performance and General Changes
Direct Persistence Layer
Utility Changes
Documentation, Installation and Integration

### Log File On-Disk Format Changes:

JE 4.0.103 has moved to on-disk file format 7.

The change is forward compatible in that JE files created with release 3.3 and earlier can be read when opened with JE 4.0.103. The change is not backward compatible in that files created with JE 4.0 cannot be read by earlier releases. Note that if an existing environment is opened read/write, a new log file is written by JE 4.0 and the environment can no longer be read by earlier releases.

Changes in API and exception handling create a binary incompatibility between JE 4.0 and JE 3.X. Users are advised to recompile their applications when moving to JE 4.0.

### New Features:

1. This release features JE High Availability, which permits the use of JE with replication. See the High Availability Guide for an introduction to the product, and the com.sleepycat.je.rep javadoc for API specifications.
2. JE now provides two JMX MBean implementations for monitoring a running JE application. A JConsole plugin is also provided for accessing the new JEMonitor and RepJEMonitor classes. Environment statistics can be collected and graphed, and other state information can be perused. See the how-to for more information.
3. It is critical that invalid files are not added to a backup set, since then both the live environment and the backup will be invalid. Two new classes now support verification of log files prior to making a backup.

   The `com.sleepycat.je.util.LogVerificationInputStream` class enables log verification programmatically as files are being copied. The `com.sleepycat.je.util.DbVerifyLog` class is a simple command line utility for inclusion in backup scripts. [#16476]

4. Cursor and Database record retrieval has been optimized to significantly reduce I/O when `DatabaseEntry.setPartial` is used to suppress return of the data item and the `READ_UNCOMMITTED` isolation mode is used. See "Using Partial DatabaseEntry Parameters"in the Cursor javadoc for more information. [#16859]
5. The `EnvironmentConfig.setNodeName(String nodeName)` and `EnvironmentConfig.getNodeName()` methods have been added to match those same methods in `ReplicationConfig`. Setting the `EnvironmentConfig` node name will cause java.util.logging messages and thread names to include the user-specified name. This was previously only true for replicated environments, and has been extended to work for non-replicated environments, as a debugging aid when using multiple environments in the same JVM. The feature was originally requested on the OTN forum. [#17629] (4.0.70)

### API Changes:

1. The `close()` method may now be called repeatedly on `Database`, `Cursor`, `Environment`, `JoinCursor`, and `EntityStore` instances, and `abort()` may be called repeatedly on `Transaction` instances. Whereas they used to throw an exception if `close()` or `abort()` was called multiple times, they now return silently on subsequent calls. [#16214]
2. `XAEnvironment.getXATransaction()` has been removed from the Javadoc. This is an internal entrypoint and should not be used by general users. `getXATransaction()`'s behavior has been changed so that it will no longer return a `Transaction` instance for a transaction which is prepared, but not yet committed or rolled back. The only user operations which can be performed on prepared, but not yet committed/rolled-back transactions are `XAEnvironment.commit()` or `rollback()`. [#16375]
3. Added the `DatabaseConfig.setUseExistingConfig` and `DatabaseConfig.getUseExistingConfig` methods. This allows a user application to open a Database readonly and determine its existing DatabaseConfig (e.g. whether the database has sorted duplicates or not). [#16504]
4. Several major improvements have been made to the JE exception hierarchy. In most cases, no source code changes to applications are required. However, users are strongly encouraged to read the new exception documentation and follow the recommended approach for exception handling. In one case, a compatibility flag must be set if `DeadlockException` or `LockNotGrantedException` is explicitly caught, and the use of these exceptions is not changed as described below. Also, if an instance of `DatabaseException` is being created directly by the application, this must be changed as described below.
   - `DatabaseException` is now a runtime exception rather than a checked exception. In other words, it now extends `RuntimeException` rather than `Exception`. Since all other JE exceptions extend `DatabaseException`, all JE exceptions are now runtime exceptions. This change allows catching and handling JE exceptions at the level of the call chain most appropriate for a given situation, without having to declare `DatabaseException` in the throws clause of every method or wrap it in a runtime exception. In some cases JE exceptions should be handled by the immediate caller of a JE method, but in other cases should be handled at a higher level. Using runtime exceptions makes it easier for applications to make the appropriate choice. No application changes are required, although many users may wish to remove `DatabaseException` from the throws clause of methods where it is no longer needed, or remove the use of runtime exception wrappers. [#16704]
   - The JE exception hierarchy has been expanded and all exceptions are now classified as either operation failures (base class `OperationFailureException`) or environment failures (base class `EnvironmentFailureException`). For both types of failures, the general approach for handling the exception has been defined in the javadoc for these base classes. To support handling all exceptions in a general way, the `Transaction.isValid` and `Environment.isValid` methods have been added. [#16704]
   - `RunRecoveryException` has been deprecated and replaced by `EnvironmentFailureException`; however, application code that catches `RunRecoveryException` need not be changed immediately. `EnvironmentFailureException` extends `RunRecoveryException`, so existing code that catches `RunRecoveryException` will now catch `EnvironmentFailureException`. However, please be aware that `EnvironmentFailureException` is now thrown for more error cases than `RunRecoveryException` was in JE 3.3 and earlier. Applications are strongly encouraged to use the new approach for handling `EnvironmentFailureException`, which is described in the javadoc for this class. [#16704]
   - The `DatabaseException` class is now abstract. This ensures that a specific exception, having well defined rules for handling it, is always thrown. Applications that are throwing `DatabaseException` directly must be changed to throw a different exception, and not a JE exception. Although JE exceptions have public constructors, these are excluded from the javadoc and marked as Internal Use Only. JE exceptions should only be created and thrown by JE. [#16704]
   - Several improvements have been made to the exception hierarchy for lock conflicts (lock timeouts, transaction timeouts, deadlocks, etc). New exceptions and the new base class `LockConflictException` are available. Now, `LockConflictException` should be caught to handle lock conflicts in a general manner, instead of catching `DeadlockException`. New exceptions are now thrown as follows:
     - `LockTimeoutException` is now thrown when a lock timeout occurs, rather than `DeadlockException`.
     - `TransactionTimeoutException` is now thrown when a transaction timeout occurs, rather than `DeadlockException`.
     - `LockNotAvailableException` is now thrown when a lock conflict occurs for a no-wait transaction, rather than `LockNotGrantedException`.
     These three new exceptions are subclasses of `LockConflictException`. `DeadlockException` is also now a subclass of `LockConflictException`, but is not currently thrown by JE because true deadlock detection is not used in JE. Currently, lock timeouts are used instead. When true deadlock detection is added to JE in the future, `DeadlockException` will be thrown. `LockNotGrantedException` has been deprecated and replaced by `LockNotAvailableException`. Unless `EnvironmentConfig.LOCK_OLD_LOCK_EXCEPTIONS` is set to true (which enables the old exception behavior) the following changes must be made to JE applications that upgrade from JE 3.3 or earlier.
     1. All occurrences of `DeadlockException` must be replaced with `LockConflictException` or one of its non-deprecated subclasses (`LockTimeoutException`, `TransactionTimeoutException`, or `LockNotAvailableException`). It is strongly recommended to replace it with `LockConflictException`, since catching this exception will catch true deadlocks in the future and other types of lock conflicts. All lock conflicts all typically handled in the same way, which is normally to abort and retry the transaction.
     2. All occurrences of `LockNotGrantedException` must be replaced with `LockNotAvailableException`. `LockNotGrantedException` has been deprecated because it misleadingly extends `DeadlockException`.
     [#16264]
   - To go along with the lock conflict exception changes above, example code is now provided that shows the canonical approach for implementing a transaction retry loop. Adopting this approach is especially important for compatibility with the use of replication. Please see the Performing Transaction Retries section in the javadoc for `com.sleepycat.je.LockConflictException` (for the base API) and `com.sleepycat.persist.EntityIndex` (for the DPL). [#14573]
   - In addition, many other new exceptions have been defined so they may be specifically caught and handled. For example, this allows detecting a secondary constraint violation by catching `SecondaryConstraintException`, or one of its more specific subclasses, rather than by catching `DatabaseException`. Please see the javadoc for the following new exceptions. Note that some new exceptions were added for replication support.
     - `DatabaseExistsException`
     - `DatabasePreemptedException`
     - `DeleteConstraintException`
     - `DuplicateDataException`
     - `EnvironmentNotFoundException`
     - `ForeignConstraintException`
     - `LockConflictException`
     - `LockNotAvailableException`
     - `LockPreemptedException`
     - `LockTimeoutException`
     - `LogOverwriteException`
     - `LogWriteException`
     - `SecondaryConstraintException`
     - `SecondaryIntegrityException`
     - `SecondaryReferenceException`
     - `SequenceExistsException`
     - `SequenceIntegrityException`
     - `SequenceNotFoundException`
     - `SequenceOverflowException`
     - `ThreadInterruptedException`
     - `TransactionTimeoutException`
     - `UniqueConstraintException`
     - `VersionMismatchException`
     - `XAFailureException`
     [#16704]
   - The JE javadoc has been updated to specify all exceptions that are thrown by each method, including the standard Java runtime exceptions (IllegalStateException, UnsupportedOperationException, and IllegalArgumentException). [#16704]
   See the `DatabaseException` javadoc for an overview of all JE exceptions.
5. The `Environment.getLockStats()` method and `LockStats` class are deprecated. All of the lock stats may be obtained using `Environment.getStats()`. The same get methods which exist on `LockStats` are now present on `EnvironmentStats`. [#16792]
6. Added the `Database.compareKeys(DatabaseEntry, DatabaseEntry)` and `Database.compareDuplicates(DatabaseEntry, DatabaseEntry)` methods which may be used for comparing two `DatabaseEntry` instances using either the btree comparator or the duplicate comparator. [#16816]
7. The `com.sleepycat.je.ForeignKeyDeleteAction`, `com.sleepycat.je.LockMode`, and `com.sleepycat.je.OperationStatus` classes have been changed from classes to Java enums. This is a compatible change and does not require a recompilation unless you use `LockMode.DIRTY_READ`. `LockMode.DIRTY_READ` was previously deprecated and has now been removed from the code base. Use of the `LockMode.DIRTY_READ` field in class files compiled with earlier versions of JE will cause `java.lang.NoSuchFieldError` to be thrown at runtime.

   In addition, the following previously deprecated methods, classes, and constants have been removed:

   `com.sleepycat.collections.StoredCollections.dirtyReadCollection`,
   `com.sleepycat.collections.StoredCollections.dirtyReadList`,
   `com.sleepycat.collections.StoredCollections.dirtyReadMap`,
   `com.sleepycat.collections.StoredCollections.dirtyReadSet`,
   `com.sleepycat.collections.StoredCollections.dirtyReadSortedMap`,
   `com.sleepycat.collections.StoredCollections.dirtyReadSortedSet`,
   `com.sleepycat.collections.StoredContainer.isDirtyReadAllowed`,
   `com.sleepycat.collections.StoredContainer.isDirtyRead`,
   `com.sleepycat.je.CursorConfig.DIRTY_READ`,
   `com.sleepycat.je.CursorConfig.setDirtyRead`,
   `com.sleepycat.je.CursorConfig.getDirtyRead`,
   `com.sleepycat.je.TransactionConfig.setDirtyRead`,
   `com.sleepycat.je.TransactionConfig.getDirtyRead`,

   [#16984]

8. The `SecondaryCursor.dupSecondary()` method has been deprecated and replaced by `SecondaryCursor.dup()`. The `SecondaryDatabase.openSecondaryCursor()` method has been deprecated and replaced by `SecondaryDatabase.openCursor()`. The `SecondaryDatabase.getSecondaryConfig()` method has been deprecated and replaced by `SecondaryDatabase.getConfig()`. The `SecondaryDatabase.openSecondaryCursor()` method has been deprecated and replaced by `SecondaryDatabase.openCursor()`. The `cloneConfig()` methods for `DatabaseConfig`, `EvolveConfig`, and `StoreConfig` have been deprecated and replaced by `clone()`. The `clone()` methods for `CheckpointConfig`, `CursorConfig`, `EnvironmentConfig`, `JoinConfig`, `PreloadConfig`, `SecondaryConfig`, `SequenceConfig`, `StatsConfig`, `TransactionConfig`, and `VerifyConfig` have been added. [#16985]
9. All `public void setFoo()` methods in `com.sleepycat.je.*Config` have been changed to return `this` instead of `void`. This allows multiple set calls to be placed on one line, similar to `StringBuffer` calls. For example, it is now possible to do the following:

       EnvironmentConfig envConf = new EnvironmentConfig();
       envConf.setAllowCreate(true).setTransactional(true);

   This will require users to recompile their code. [#17021]
10. In JE 4.0.70, an additional change was made so that all setter methods in the DPL `StoreConfig` `EvolveConfig` now return `this` rather than having a `void` return type. This change requires that applications using the DPL be recompiled. [#17021] (4.0.70)
11. Previous versions of JE would allow conflicting settings in `TransactionConfig` and `EnvironmentConfig` for `TxnNoSync`, `TxnWriteNoSync`, and `TxnSync`. `TransactionConfig` no longer allows setting more than one of `TxnNoSync`, `TxnWriteNoSync`, or `TxnSync` to `true`. An `IllegalArgumentException` will be thrown if the settings of these configuration parameters are conflicting. [#17705]
12. The 4.0 release provides support for replicated environments. JE defines a new class: `Durability`, as a single consistent way to specify durability for both standalone and replicated environments. Accordingly, the methods: `setTxnNoSync, getTxnNoSync, setTxnWriteNoSync, getTxnWriteNoSync` in EnvironmentMutableConfig and methods `setNoSync, getNoSync, setWriteNoSync, getWriteNoSync` in TransactionConfig are deprecated, the methods `setDurability, getDurability` should be used in their place. A new `Transaction` method, `commit(Durability)` has also been provided for use in both standalone and replicated environments.
13. Removed any code which uses NIO for IO to the file system. Deprecated the `je.log.useNIO` and `je.log.directNIO` parameters. If `EnvironmentConfig.LOG_USE_NIO` or `EnvironmentConfig.LOG_DIRECT_NIO` is used by the application, a deprecation warning will result. If these parameters are set, they will have no effect on JE, and no warning (other than the deprecation warning) will be given to the user.
14. JE 3.X and earlier provided various proprietary JE properties named "java.util.logging.XXX" which were meant to be set either in the je.properties file or in the EnvironmentConfig class. These properties configured JE's use of the java.util.logging framework, but did not obey the conventions of standard java.util.logging naming and configuration management. These proprietary properties have been removed. Instead, JE's use of java.util.logging can now be configured in the standard fashion through the configuration properties described by the java.util.logging package.

    To display all logging output to the console, set com.sleepycat.je.util.ConsoleHandler.level = ALL as a java.util.logging property. By default, JE records logging output of level INFO or higher is recorded in <envdir>/je.info. To change the level of output displayed in the file, set com.sleepycat.je.util.FileHandler.level to the desired level. The logging output is particularly helpful in a replicated application. An example of the output can be found here.

15. The following `Environment` configuration properties now allow specifying a time unit. In the past, values could only be expressed as microseconds. For these properties, the default unit has always been microseconds and that has not changed. For more information about specifying units with configuration properties, see the Time Duration Properties section of the `EnvironmentConfig` class. These properties are:
    - `EnvironmentConfig.LOCK_TIMEOUT`
    - `EnvironmentConfig.TXN_TIMEOUT`
    - `EnvironmentConfig.LOG_FSYNC_TIMEOUT`
    - `EnvironmentConfig.ENV_BACKGROUND_SLEEP_INTERVAL`
    - `EnvironmentConfig.COMPRESSOR_WAKEUP_INTERVAL`
    - `EnvironmentConfig.COMPRESSOR_LOCK_TIMEOUT`
    - `EnvironmentConfig.CHECKPOINTER_WAKEUP_INTERVAL`
    - `EnvironmentConfig.CLEANER_LOCK_TIMEOUT`

    In addition, the following new methods have been added to allow specifying a unit along with a time duration. The equivalent older methods, without a unit argument, have been deprecated. The new methods are:

    - `EnvironmentConfig.setLockTimeout(long,TimeUnit)`
    - `EnvironmentConfig.getLockTimeout(TimeUnit)`
    - `EnvironmentConfig.setTxnTimeout(long,TimeUnit)`
    - `EnvironmentConfig.getTxnTimeout(TimeUnit)`
    - `Transaction.setLockTimeout(long,TimeUnit)`
    - `Transaction.getLockTimeout(TimeUnit)`
    - `Transaction.setTxnTimeout(long,TimeUnit)`
    - `Transaction.getTxnTimeout(TimeUnit)`
16. All methods that return a new cursor in a replicated environment now allow passing a null `Transaction`. This includes `Database.openCursor`, `EntityIndex.entities`, `EntityIndex.keys`, and `StoredContainer.storedIterator`. Before, a null `Transaction` was prohibited in a replicated environment unless read-uncommitted isolation was configured. Note that a null transaction has always been allowed (and still is) in a non-replicated environment. [#16513] (4.0.70)
17. The `Transaction.commit` and `CurrentTransaction.commitTransaction` family of methods no longer throw `LockPreemptedException`. Note that `LockPreemptedException` is only applicable in replicated environments. [#16513] (4.0.70)
18. An `IllegalArgumentException` is now thrown if `LockMode.RMW` is passed to the `Database.get` and `Database.getSearchBoth` methods, and a null `Transaction` is also passed. Without a transaction it is not possible to use these methods to perform a read-modify-write operation that initially acquires a write lock, so the combination of parameters does not make sense. [#16513] (4.0.70)
19. IllegalStateException is now thrown if Transaction.getCommitToken() is called before the transaction has finished. Previously, it would return a null commit token. [#17377] (4.0.70)
20. `Database.preload(PreloadConfig)` no longer throws `NullPointerException` if null is passed for an argument. Instead, it uses a default `PreloadConfig`. [#17784] (4.0.72)

### Performance and other General Changes

1. Fixed a bug which could cause a deadlock if a Database handle is closed while another thread is still using it. Removed code which automatically closes cursors that are still open at Database.close() time. [#15413]
2. If a call to `com.sleepycat.je.Transaction.commit()` throws an exception because the transaction has open cursors, a follow-on call to `Transaction.abort()` could fail with the following stack trace:

           Transaction 3 has been closed. java.lang.IllegalStateException: Transaction 3 has been closed.
               at com.sleepycat.je.txn.Txn.checkState(Txn.java:1580)
               at com.sleepycat.je.txn.Txn.abortInternal(Txn.java:819)
               at com.sleepycat.je.txn.Txn.abort(Txn.java:805)
               at com.sleepycat.je.Transaction.abort(Transaction.java:90)
               at com.sleepycat.je.txn.TxnEndTest.testTxnClose(TxnEndTest.java:

   This was a bug and has been fixed so the call to abort() will complete successfully. [#16214]
3. Improved synchronization on `Database` instances obviating the need to use separate `Database` handles in high-concurrency environments. Previously, this practice was recommended, but is no longer necessary. [#16346]
4. Improved internal concurrency by using shared (vs exclusive) latches on internal nodes of the B+Tree. Added a new stat to Database and Environment stats to record the number of "relatches" (latch upgrades) that have occurred on both the particular Database and overall for the Environment. [#16346]
5. When opening different environments in the same process, if recovery for one of the environments took a long time, the other environment openings would block. Changed the behavior such that opening additional environments is not blocked by another environment's recovery. [#16350]
6. Improved JE write performance when doing synchronous transaction commits on certain Linux filesystems, including ext3.

   In JE, when the durability specified for a transaction dictates force-writing to disk (for example, a `commitSync()` call), it calls `fsync()`. On modern disk drives an `fsync()` will take on the order of several milliseconds. This is a relatively long time compared to the time required to do a `write()` call, which only moves data to the operating system's file cache.

   JE's group commit mechanism seeks to reduce the number of fsyncs required by issuing one fsync for batches of multiple transaction commits. For example, if a thread T1 requires an `fsync()`, and JE determines that no `fsync()` is in progress, it will execute that `fsync()` immediately. If, while T1 is executing the `fsync()`, some other thread(s) require an fsync(s), JE will block those threads until T1 finishes. When T1's fsync() completes, a new `fsync()` executes on behalf of the blocked thread(s).

   The past JE group commit implementation assumed that the underlying platform (OS + file system combination) allow IO operations like `seek()`, `read()` and `write()` to execute concurrently with an `fsync()` call (on the same file, but using different file descriptors). On Solaris and Windows this is true. Hence, on these platforms, a thread which is performing an `fsync()` does not block another thread performing a concurrent `write()`. But, on several Linux file systems, ext3 in particular, an exclusive mutex on the inode is grabbed during any IO operation. So a `write()` call on a file (inode) will be blocked by an `fsync()` operation on the same file (inode). This negates any performance improvement which might be achieved by group commit.

   The JE group commit code has been improved to batch `write()` and `fsync()` calls, rather than just `fsync()` calls. Just before a `write()` call is executed, JE checks if an `fsync()` is in progress and if so, the write call is queued in a (new) Write Queue. Once the `fsync()` completes, all pending writes in the Write Queue are executed.

   This change in behavior is enabled by default and may be disabled by setting the `je.log.useWriteQueue` configuration parameter to `false`. The size of the Write Queue (i.e. the amount of data it can queue until any currently-executing IO operations complete) can be controlled with the `je.log.writeQueueSize` parameter. The default for `je.log.writeQueueSize` is 1MB with a minimum value of 4KB and a maximum value of 32MB. The Write Queue does not use cache space controlled by the `je.maxMemory` parameter. [#16440]

7. Trace messages are now written to the log at recovery and shutdown time. This applies mostly as a field support aid. [#16452]
8. Fixed a memory leak in applications which set EnvironmentConfig.setLocking(false) and used multiple environments and shared caches. JE could continue to maintain a reference to internal data structures after an `Environment` is fully closed. This report was originally mentioned in this forum post. [#16453]
9. Fixed a circular static initialization dependency between `LockType` and `LockUpgrade`. This was discovered by a user whose use of reflection caused a change in the class loading order. [#16496]
10. Fixed a bug that causes incorrect secondary key values to be created when a partial DatabaseEntry is passed to a Cursor or Database put() method, if secondaries are configured. Before, the partial entry was passed to the SecondaryKeyCreator. Now, the resulting merged data is passed to the key creator. Thanks to archie172 for reporting this on OTN. [#16929]
11. A bug in which the `DbPrintLog` utility would sometimes display negative numbers for the size of the key/data bytes when using the `-S` option has been fixed. A `-SC` option has also been added which prints the summary information in CSV format. [#17196]
12. Fixed a bug where com.sleepycat.je.util.DbPrintLog could throw a NullPointerException. [#17456]
13. Fixed a bug that would allow the Total Memory Usage stat to become negative on certain architectures. [#17462]
14. Removed a concurrency hot spot in TxnManager.thread2Txn, which has a positive influence on highly concurrent applications with many transactions . [#17542]
15. Two new environment configuration parameters are available to control the use of "proactive migration" in the log cleaner:
    - `EnvironmentConfig.CLEANER_FOREGROUND_PROACTIVE_MIGRATION`
    - `EnvironmentConfig.CLEANER_BACKGROUND_PROACTIVE_MIGRATION`
    See the javadoc for these parameters for more information. By default, proactive migration is now disabled. In earlier releases, proactive migration was unconditionally enabled for both foreground and background activities. This sometimes caused operation delays and long checkpoints, and these problems should now be reduced. But because of this change in behavior, in rare cases an application may now need to configure more cleaner threads, or enable proactive migration, in order to prevent a cleaner backlog. [#17634]
16. In rare cases requiring a combination of delete and repeated insert operations and environment recoveries, Database.count() could be incorrect, usually by a small number of records. Note that Database.count() does have the general caveat that is it not guaranteed to be accurate when there concurrent updates. This miscount has been fixed. [#17770] (4.0.61)
17. In JE 4.0.60, read operations such as Database.get(), when executed on a replicated environment, and used with a null transaction parameter, did not properly obey the default environment replica read consistency policy. This has been fixed. [#16513] (4.0.70)
18. Fixed a bug that sometimes caused a NullPointerException in a replicated environment, when a database was renamed on the Master. If a secondary database was renamed, the app running on the Replica could see a NullPointerException rather than the intended DatabasePreemptedException. [#17015] (4.0.70)
19. Fixed a bug where a LogFileNotFoundException could be seen in an environment which had repeated inserts and deletes of records with the same key and repeated environment recoveries. [#17879] (4.0.70)
20. A bug could cause replicated environments which contain databases with duplicates to see the following exception with this distinctive stacktrace:

        (JE 4.0.60)... LOG_FILE_NOT_FOUND: Log file missing, log is likely
         invalid. Environment is invalid and must be closed.
        	at com.sleepycat.je.tree.ChildReference.
        fetchTarget(ChildReference.java:147)
        	at com.sleepycat.je.tree.DIN.getDupCountLN(DIN.java:155)
        	at com.sleepycat.je.dbi.CursorImpl.lockDupCountLN(CursorImpl.java:2596)
        	at com.sleepycat.je.tree.Tree.insertDuplicate(Tree.java:2705)
        	at com.sleepycat.je.tree.Tree.insert(Tree.java:2652)
        	at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1076)
        	at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1769)
        	at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1726)
        	at com.sleepycat.je.Cursor.putNotify(Cursor.java:1659)
        	at com.sleepycat.je.Cursor.putLN(Cursor.java:1609)
        	at com.sleepycat.je.DbInternal.putLN(DbInternal.java:125)
        	at com.sleepycat.je.rep.impl.node.Replay.applyLN(Replay.java:738)
        	at com.sleepycat.je.rep.impl.node.Replay.replayEntry(Replay.java:483)
        	at com.sleepycat.je.rep.impl.node.Replica.doRunReplicaLoopInternalWork(Replica.java:414)
        	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:341)
        	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:283)
        	at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:886)

    The problem would only occur in a fairly tight timing window, and has been fixed. [#17879] (4.0.70)
21. A replicated environment which had been running in a replication group and restarted as a replica could see this exception at startup time:

         (JE 4.0.60) ... GroupCBVLSN: 231,655 is outside VLSN range:
         first=231,700 last=583,909 sync=583,909 txnEnd=583,909
        UNEXPECTED_STATE_FATAL: Unexpected internal state, unable to continue.
        Environment is invalid and must be closed. Problem seen replaying entry ...
          at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:392)
          at com.sleepycat.je.rep.impl.node.GlobalCBVLSN.recalculate(GlobalCBVLSN.java:179)
          at com.sleepycat.je.rep.impl.node.RepNode.recalculateGlobalCBVLSN(RepNode.java:650)
          at com.sleepycat.je.rep.impl.node.Replay.replayEntry(Replay.java:444)
          at com.sleepycat.je.rep.impl.node.Replica.doRunReplicaLoopInternalWork(Replica.java:414)
          at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:341)
          at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:283)
          at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:886)

    This has been fixed. [#17885] (4.0.70)
22. `Database.preload(PreloadConfig)` no longer throws `NullPointerException` if null is passed for an argument. Instead, it uses a default `PreloadConfig`. [#17784] (4.0.72)

### Direct Persistence Layer (DPL), Collections and Bind packages

1. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an enum class. Before, the confusing message `Persistent class has non-persistent superclass: java.lang.Enum` was reported. Thanks to Lowell for reporting this problem on OTN. [#15623]
2. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an interface. Before, a NullPointerException was reported. Thanks to Tony Clifton for reporting this problem on OTN. [#15841]
3. Report a meaningful IllegalArgumentException when @Persistent or @Entity is incorrectly used on an inner class (a non-static nested class). Before, the confusing message `No default constructor` was reported. [#16279]
4. Entity subclasses that define secondary keys must now be registered prior to storing an instance of the class. This can be done in two ways:

   - The `EntityModel.registerClass` method may be called to register the subclass before opening the entity store.
   - The `EntityStore.getSubclassIndex` method may be called to implicitly register the subclass after opening the entity store.

   Failure to register the entity subclass will result in an `IllegalArgumentException` the first time an attempt is made to store an instance of the subclass. An exception will not occur if instances of the subclass have previously been stored, which allows existing applications to run unmodified in most cases.

   This behavioral change was made to increase reliability. In several cases, registering an entity subclass has been necessary as a workaround, as described here, for example. The requirement to register the subclass will ensure that such errors do not occur in deployed applications. [#16399]

5. When a secondary index database is opened, it is now auto-populated only if it did not previously exist, rather than if it is empty. This prevents repeated auto-population of the secondary from occurring when the secondary keys for all entities in the primary index happen to be null. This problem was reported by jhalex on OTN. Note that the workaround mentioned in the OTN thread -- to create a dummy record that contains a non-null value for each secondary key -- is no longer necessary.

   To go along with this change, the `EntityStore.truncateClass` method now removes all secondary databases rather than truncating them, so that they will be auto-populated if they are accessed again. The primary database is truncated, as before. [#16399]

6. The `com.sleepycat.collections.TransactionRunner.handleException` method has been added to allow overriding the default transaction retry policy. See the javadoc for this method for more information. [#16574]
7. Fixed a bug that causes an assertion to fire or a NullPointerException (when assertions are disabled) from the EntityStore constructor. The problem occurs only when the previously created EntityStore contains an entity with a secondary key definition in which the key name has been overridden and is different than the field name. The key name can be overridden with the name property of the SecondaryKey annotation. Below are examples of the assertion and exception.

       java.lang.NullPointerException
           at com.sleepycat.persist.impl.ComplexFormat.checkSecKeyMetadata(ComplexFormat.java:1143)
           at com.sleepycat.persist.impl.ComplexFormat.evolveFieldList(ComplexFormat.java:1428)
           at com.sleepycat.persist.impl.ComplexFormat.evolveAllFields(ComplexFormat.java:1245)
           at com.sleepycat.persist.impl.ComplexFormat.evolve(ComplexFormat.java:1019)
           at com.sleepycat.persist.impl.Evolver.evolveFormatInternal(Evolver.java:440)
           at com.sleepycat.persist.impl.Evolver.evolveFormat(Evolver.java:248)
           at com.sleepycat.persist.impl.PersistCatalog.(PersistCatalog.java:357)
           at com.sleepycat.persist.impl.Store.(Store.java:180)
           at com.sleepycat.persist.EntityStore.(EntityStore.java:165)

       java.lang.AssertionError
           at com.sleepycat.persist.impl.ComplexFormat.evolveFieldList(ComplexFormat.java:1327)
           at com.sleepycat.persist.impl.ComplexFormat.evolveAllFields(ComplexFormat.java:1245)
           at com.sleepycat.persist.impl.ComplexFormat.evolve(ComplexFormat.java:1019)
           at com.sleepycat.persist.impl.Evolver.evolveFormatInternal(Evolver.java:440)
           at com.sleepycat.persist.impl.Evolver.evolveFormat(Evolver.java:248)
           at com.sleepycat.persist.impl.PersistCatalog.(PersistCatalog.java:357)
           at com.sleepycat.persist.impl.Store.(Store.java:180)
           at com.sleepycat.persist.EntityStore.(EntityStore.java:165)

   Thanks to Lukasz Antoniak for reporting this bug accurately and clearly on OTN. [#16819]
8. Key cursors have been optimized to significantly reduce I/O when the `READ_UNCOMMITTED` isolation mode is used. See `EntityIndex.keys` for more information. [#16859]
9. Report a meaningful IllegalArgumentException when NULLIFY is used with a @SecondaryKey and the field is a primitive type. Before, the confusing message `Key field object may not be null` was reported. Thanks to Erick for reporting this problem on OTN. [#17011]
10. Enum fields may now be used as DPL keys, including primary keys, secondary keys, and fields of composite key classes. Comparators are supported for composite key classes containing enum fields.

    A bug was also fixed that prevented enum values from being inserted before existing values in an enum declaration. Now, new values may be inserted before existing values in the declaration, as well as appended to the end of the declaration. However, note that renaming and deletion of enum values is not supported. [#17140]

11. Fixed a bug that prevented the use of custom key comparisons (composite key classes that implement `Comparable`) for secondary keys defined as `ONE_TO_MANY` or `MANY_TO_MANY`. An example stack trace is below.

        java.lang.IllegalArgumentException: Key class is not a simple
          type or a composite key class (composite keys must include @KeyField
          annotations): java.util.Set
        at com.sleepycat.persist.impl.PersistKeyBinding.(PersistKeyBinding.java:38)
        at com.sleepycat.persist.impl.Store.getKeyBinding(Store.java:1294)
        at com.sleepycat.persist.impl.Store.setBtreeComparator(Store.java:1312)
        at com.sleepycat.persist.impl.Store.getSecondaryConfig(Store.java:1115)
        at com.sleepycat.persist.impl.Store.openSecondaryIndex(Store.java:666)
        at com.sleepycat.persist.impl.Store.openSecondaryIndexes(Store.java:636)
        at com.sleepycat.persist.impl.Store.getPrimaryIndex(Store.java:384)
        at com.sleepycat.persist.EntityStore.getPrimaryIndex(EntityStore.java:259)

    [#17207]
12. Improve performance of `StoredCollection.removeAll`. This method no longer iterates over all records in the stored collection. Thanks to ambber on OTN for suggesting and testing this improvement. [#17727]
13. A hot upgrade of an application (without taking down the replication group) is now allowed when persistent DPL classes have been changed. Such changes take place when the DPL schema has been updated and class evolution is required. Before, to deploy such a change, the entire replication group had to be bought down. For more information see "Upgrading a Replication Group" in the javadoc package summary for the `com.sleepycat.persist.evolve` package. [#16655] (4.0.70)

### Utility Changes:

1. Changed DbPrintLog -S so that it can display values in the trillions. Fixed a bug that didn't allow -S or -q to come before any other flags on the command line. [#16370]
2. Fixed a bug which caused an `InvocationTargetException` to be displayed (masking the real exception) when invoking a utility with the je.jar file in this way:

         java -jar je.jar <UtilityName>

   [#16649]

### Documentation, Installation and Integration:

1. JE support in Android has been improved by the addition of a `lib/je-android.M.N.P.jar` file. Rather than requiring the Android application developer to modify and recompile the JE sources so that references to `javax.transaction.xa.*` classes are removed, the jar file can be simply included in the project's `libs` directory. The JE and Android directions reflect these changes. (4.0.70)
2. The Monitoring JE with JConsole and JMX directions had a small typo. The sentence "(e.g. using -DJEMonitor-true on the command line)" has been corrected to say "-DJEMonitor=true". (4.0.70)