Berkeley DB Library Version 11.2.5.2 Change Log

Changes between 11.2.5.2.36 and 11.2.5.2.42
Changes between 11.2.5.2.28 and 11.2.5.2.36
Database or Log File On-Disk Format Changes
New Features
Database Environment Changes
Concurrent Data Store Changes
Access Method Changes
SQL API Changes
C API Changes
Tcl-specific API Changes
C#-specific API Changes
Replication Changes
Locking Subsystem Changes
Logging Subsystem Changes
Memory Pool Subsystem Changes
Mutex Subsystem Changes
Transaction Subsystem Changes
Test Suite Changes
Utility Changes
Configuration, Documentation, Sample Apps, Portability and Build Changes
Example Changes
Miscellaneous Bug Fixes
Deprecated Features
Known Bugs

This is the changelog for Berkeley DB 11g Release 2 (library version 11.2.5.2).

Changes between 11.2.5.2.36 and 11.2.5.2.42

  1. Fixed a bug that could cause the last page number stored on the metadata page to be wrong after rolling forward a db->compact operation that freed more pages than will fit in a single log record. [#20646]

  2. Fixed a bug that could cause recovery to fail if DB->compact moved the meta data page of a HASH subdatabase. [#20708]

  3. Fixed a bug in reporting handle lock modes that could cause undetectable deadlock in the SQL API. [#20722]

  4. Fixed a bug in the SQL API that could cause a corrupted environment to not run recovery automatically. [#20767]

  5. Fixes to ADO.NET: solution files will be in DOS format and appropriate compiler flags are specified for debug build. [#20942]

  6. Fixed a bug in the Java API where EnvironmentConfig.setCreateDir would fail to configure the environment. [#21127]

Changes between 11.2.5.2.28 and 11.2.5.2.36

  1. Updated the JDBC version shipped with Berkeley DB to support Oracle Java Embedded Client. [#19814]

  2. Fixed several memory leaks in the Online Backup API. [#19850]

  3. Fix a bug in the SQL API when using a blob field with a lot of content and multiple concurrent connections to the database. [#19945]

  4. Update EID_MASTER to be public static final so that it will be exposed in Java docs. [#20184]

  5. Fixed a bug where BFile module crashes when using BFile handle for SQL expressions interface on 64 bit platforms. [#20193]

  6. Fixed a bug where, on systems without FTRUNCATE, db_verify will return an error for truncated heap databases. [#20195]

  7. Let ADO.NET solution building skip SQLite Designer and Linq by default. [#20218]

  8. Fixed a bug that could cause BDB to run out of avaliable mutexes when renaming many databases. [#20228]

  9. Fixed a bug where the metadata page in hash databases would not be flushed to disk. [#20265]

  10. Fixed a bug where printlog would fail on in-memory heap databases. [#20269]

  11. Fixed a bug that would cause verify to call the wrong compare function if there are user defined compare functions used and the database has multilevel off page sorted duplicate trees. [#20284]

  12. Fixed a bug where two processes accessing the same table and one of those drops the table and recreates, the second process can crash. [#20319]

  13. Fixed a bug where it was possible to panic a heap database without an error message being printed. [#20320]

  14. Fixed a bug where it would fail to put records with overflow keys into hash duplicate database. [#20329]

  15. Fixed a bug where multiple Replication Manager processes would sometimes not all conform to replication-group-aware log archiving. [#20342]

  16. Fixed a bug in the Java API where getting a Stat object could cause a segfault. [#20377]

  17. Fix a bug where cache page eviction statistics could be inaccurate. [#20410]

  18. Fix a bug where extending an almost-full region could cause a seg fault. [#20414]

  19. Fix a bug where multiple long running transactions across checkpoints could cause Log Sequence errors on client systems. [#20421]

  20. Fix a bug where aborting an exclusive transaction followed by an auto-commit read operation could cause an assert failure. [#20567]

  21. Detect obsolete MVCC buffers earlier by refreshing stale old_reader LSNs. [#20570]

Database or Log File On-Disk Format Changes

  1. Existing database file formats were unchanged in library version 11.2.5.2. However, a new database file format, "heap", was introduced.

  2. The log file format changed in library version 11.2.5.2.

New Features

  1. Replication Manager now manages Group Membership. This allows sites to be added to and removed from the replication group dynamically. Replication Manager also now automatically keeps track of the group size (nsites). [#14778]

  2. Initial allocations for various non-pagebuffer (mpool) system resources may now be specified, as well as a total maximum of memory to use, rather than specifying a maximum value for each resource. [#16334]

  3. Implemented Berkeley DB globalization support architecture to enable localized and stripped error and output messages. [#16863]

  4. Added a new access method, DB_HEAP. Heap aims for efficient use (and re-use) of disk space. Keys in a heap database are automatically generated by BDB, it is recommended that one or more secondary indexes be used with a heap database. For full details on DB_HEAP, see the Programmer's Reference Guide. [#17627]

  5. Added a compatible mode for 32bit and 64bit Windows environment. [#18225]

  6. For the SQL API, concurrency between read and write transactions can now be enabled using "PRAGMA multiversion". Added several pragmas that can be used to configure the Berkeley DB datastore. [#18521]

  7. Add several new pragmas to provide in-process support for replication in the SQL API. [#18528]

  8. The Berkeley DB X/open compliant XA resource manager has been restored, including support for multi-threaded servers. [#18701]

  9. Improved the ability to recover from an application crash on connections through the SQL API. Berkeley DB will try to automatically clean up locks, mutexes and transactions from the failed process. [#18713]

  10. Add support for sequence usage in the SQL API using SQLite custom functions. [#19007]

  11. Add a pragma in the SQL API to allow execution of a cache trickle command. [#19202]

  12. Add a pragma in the SQL API to allow configuration of DB_SYSTEM_MEM environments. [#19249]

  13. The new db_env_set_win_security(SECURITY_ATTRIBUTES *) function allows an application to specify the particular Microsoft Windows security attributes to be used by Berkeley DB. This helps support applications which reduce their privileges after opening the environment. [#19529]

Database Environment Changes

  1. None

Concurrent Data Store Changes

  1. None

Access Method Changes

  1. Modified the queue access method so that it only uses latches on the metadata page rather than a latch and a lock. This was done to improve performance. [#18749]

  2. Fixed several bugs that could cause an update running with MVCC to get the wrong version of a page or improperly update the metadata last page number. [#19063]

  3. The database open code will no longer log the open and close of the master database in a file when opening a sub database in that file. [#19071]

  4. Fixed a bug where an error during an update to a hash database with DB_NOOVERWRITE set could return DB_KEYEXIST rather than the correct error. [#19077]

  5. Fixed a bug that could cause the wrong page number to be on a root or metadata page if DB->compact moved the page and the operation was later rolled forward. [#19167]

  6. Fixed a bug that could cause the close of a secondary index database to fail if the transaction doing the open aborted. [#19169]

  7. Fixed a bug that could prevent an update to a primary recno or queue database with DB_NOOVERWITE set. [#19230]

  8. Fixed a bug when an update to a database with DB_NOOVERWRITE set could incorrectly return DB_KEYEXIST rather than the correct error (e.g., DB_LOCK_DEADLOCK). [#19345]

  9. Fixed a bug preventing the use of the set_re_len and set_re_pad methods with a RECNO database when configuring with --disable-queue. [#19367]

  10. Fixed a bug in DB->compact on BTREE databases that did not check if the last page in the database could be moved to a lower numbered page. [#19394]

  11. Fixed a bug that could cause a Log Sequence Error when recovering the deallocation of a multiple page overflow chain. [#19474]

  12. Fixed a bug that could cause a diagnostic assertion if MVCC was in use and multiple levels of a btree needed to be split. [#19481]

  13. Fixed a few error paths that could cause a Panic with an "unpinned page returned" error. [#19493]

  14. Fixed a bug that closed a race condition that under heavy mult-threaded appending to a queue database could cause some records to be lost. [#19498]

  15. Fixed a bug that might cause DB->compact to mis-estimate the size of an overflow record when merging two pages. This may cause the page to have more data than desired. [#19562]

  16. Fixed a bug in DB_ENV->fileid_reset that did not update the fileid's on the metadata pages of subdatabases if the database file was not in native byte order. [#19608]

  17. Fixed a bug that caused the first directory specified in the create of a partitioned database to get too many partitions. [#20041]

  18. Fixed a bug that could leave deleted pages from a HEAP database in the buffer cache. [#20309]

  19. Fixed a bug that could cause a hang or improperly report an empty queue when the queue record numbers wrapped around at 2^32. [#20956]

SQL API Changes

  1. Fixed a race condition that would cause a corruption error in one process when two processes created the same SQL database. [#18929]

  2. Fixed a bug that would cause a constraint violation when updating the primary key with the same value. [#18976]

  3. Overwriting an old backup with a new backup using the SQL online backup API will no longer double the size of the database. [#19021]

  4. Implemented index optimizations for indexes on large values. [#19094]

  5. Fixed a bug that could cause an undetected deadlock between a thread which moved a metadata or root page via a DB->compact operation and another thread trying to open the database if the old page was being removed from the file. [#19186]

  6. Fix a bug in the BDBSQL_FILE_PER_TABLE option, to allow absolute path names. [#19190]

  7. Add a pragma to allow configuration of DB_SYSTEM_MEM environments. [#19249]

  8. Exclusive transactions will now block new transactions and will prevent existing transactions from making forward progress. [#19256]

  9. Fixed a bug that would cause assert error when opening an in-memory hash database with thread count configured when compiled with --enable-diagnostic. [#19357]

  10. Upgrade the bundled version of SQLite to 3.7.6.2 [#19376]

  11. Fixed a performance bug with the cache victim selection algorithm when there were multiple cache regions. [#19385]

  12. Fixed a bug which could cause two SQL threads to have an undetected deadlock when opening or closing tables. [#19386]

  13. Fix a bug that could cause a hang when deleting a table if there are multiple connections to a database from different processes. [#19419]

  14. Fixed a bug which could cause multiple threads performing DB->compact on the same database file to overrun the in-memory freelist, which could potentially lead to memory corruption. [#19571]

  15. Fixed a bug in DB->compact that could cause a loop if an attempt to move a sub-database meta data page deadlocked. [#20028]

C API Changes

  1. Fixed a bug where encryption could not be enabled for individual databases in an encrypted environment. [#18891]

  2. Removed two unused error codes, DB_NOSERVER_HOME and DB_NOSERVER_ID. [#18978]

  3. Added a DB_DBT_READONLY flag so that users can pass in a non-usermem key (DB_DBT_USERMEM) for get operations. [#19360]

  4. Fixed a bug in DB/DBC->get/pget that the partial flags are silently ignored with positional flags and return inconsistent DBT. [#19540]

  5. Fixed a bug which prevented items from being deleted on a secondary database. [#19573]

  6. Fixed a bug to correctly handle the DB_BUFFER_SMALL case on delete operations when compression is enabled. [#19660]

Tcl-specific API Changes

  1. None.

C#-specific API Changes

  1. Added support for partial put/get in the C# API. [#18795]

  2. Fixed a bug in compare delegate for secondary db. [#18935]

Replication Changes

  1. Replication Manager now allows differing ack policies at different sites throughout the group, and supports dynamic changes to the ack policy. (The ack policy in force is determined by the current master.) [#14993]

  2. Replication Manager "channels" feature allows applications to share repmgr's communication facilities. [#17228]

  3. Add example program for RepMgr "channels" feature: ex_rep_chan. [#17387]

  4. Replication Manager now allows dynamic changes to a site's "electability" (changes between zero and non-zero priority). This feature should be used with care, because electability changes can in boundary cases invalidate durability guarantees granted for previous transactions. [#17497]

  5. Changed election criteria so that later group transactions won't get overwritten by earlier generations with more log. [#17815]

  6. Added changes to master lease checks that result in improved performance when using master leases. [#18960]

  7. A log write failure on a replication master will now cause a panic since the transaction may be committed on some clients. [#19054]

  8. Fixed a few memory leak conditions on error paths. [#19131]

  9. Change lease code so that zero priority sites do not count in lease guarantees since they cannot be elected. [#19154]

  10. Repmgr rerequest processing is moved from a dedicated thread to heartbeat messages. Repmgr clients using heartbeats can now detect and rerequest missing final master log records without master activity. [#19197]

  11. Repmgr statistics are now included in full statistics output for an environment. [#19198]

  12. Fix an inefficiency in mixed version elections. We now check if an election is won via the EID instead of priority. [#19254]

  13. Changed election LSNs to use the last txn commit LSN instead of the end of the log. [#19278]

  14. Create replication internal database files in the environment home directory rather than the data directory so that they are in the same location as the other internal replication files. [#19403]

  15. Fix a bug that was preventing repmgr from calling an election when starting a site with the DB_REP_ELECTION flag. [#19546]

  16. Fixed a bug which could cause a segfault at a replication master if a named in-memory database was being created around the same time as a client site were synchronizing (in "internal init") with the master. [#19583]

  17. Adjust lease code to consider timeout length when retrying. [#19705]

  18. Fixed a bug that could cause a crash in replication groups of more than 10 sites, with multiple processes sharing each DB environment concurrently. [#19818]

  19. Fix a bug where an assertion failure could happen if pages in a database were deallocated during a client internal initialization.[#19851]

  20. Fix a bug where an internal initialization of a queue database with non-contiguous extent files could return an error. [#19925]

  21. The 2SITE_STRICT replication configuration parameter is now turned on by default. It can be turned off via a call to DB_ENV->rep_set_config(). [#19937]

  22. Repmgr heartbeats can now help detect a duplicate master without the need for application activity. [#19950]

Locking Subsystem Changes

  1. Fixed a bug where an updater supporting DB_READ_UNCOMMITED might downgrade its lock too soon if there was an error during the update. [#19155]

  2. Fixed a bug where transaction timeouts could have been specified in a database environment where the locking subsystem was disabled. [#19582]

  3. Fixed a bug in a diagnostic assertion that was improperly triggered by the removal of a sub-database. [#19683]

  4. Fixed a bug that would cause DB_ENV->failcheck to free locks for a locker associated with a database handle after the thread that opened the handle exited. [#19881]

Logging Subsystem Changes

  1. Enhanced recovery so that it will not output extra checkpoint or transaction id recycle log records if there was no activity since the last checkpoint. [#15330]

  2. Log checksums can now be disabled using the compile argument --disable-log-checksum. This will give a performance increase at the risk of undetectable corruption in the log records, which would make recovery impossible. [#19143]

  3. Fixed a bug that could cause a page that should have been removed from the end of a file still be in the copy of the file in a hot backup. [#19996]

Memory Pool Subsystem Changes

  1. Fixed a bug in MPOOLFILE->get that did not permit the DB_MPOOL_DIRTY flag to be used with other flags. [#19421]

Mutex Subsystem Changes

  1. Fixed a bug when the mutex region needs to be larger than 4GB, the region size was incorrectly adjusted to be slightly too small to fit the mutexes. [#18968]

  2. Fixed a performance problem with hybrid shared latches in which a request for exclusive access would busy-wait (rather than put itself to sleep) if the latch were held by a shared reader. This also fixed the timeout handling of hybrid mutexes. In some cases the timeout would not be honored, resulting in delays for the replication "read your writes" feature which were longer than requested. [#18982]

  3. Fixed the timeout handling of the pthreads mutexes used by the replication "read your writes" feature. When a timeout occurred there was a race condition which might result in a hang. [#19047]

Transaction Subsystem Changes

  1. Fixed a leak of log file ids when a database is closed before the end of a transaction that references it. [#15957]

  2. Fixed a bug that would cause a panic if a child transaction performed a database rename, then aborted, and then the parent transaction committed. [#18069]

  3. Fixed a bug where we released the metadata page lock too early if a non-transactional update was being done. [#19036]

  4. Removed the possibility that checkpoints will overlap in the log, decreasing the time to recover. [#19062]

Test Suite Changes

  1. Require Tcl 8.5 or greater.

Utility Changes

  1. Added a new utility, db_tuner, which analyzes the data in a btree database, and suggests a reasonable pagesize. [#18910]

  2. Fixed some bugs in log_verify when there are in-memory database logs and subdb logs. [#19157]

  3. Modified db_hotbackup to not read from the file system as required on non-UNIX systems. Also provided the db_copy function for this purpose. [#19863]

  4. Fixed db_hotbackup so that when -d/-l or -D is not specified, DB_CONFIG is used to determine the locations of the databases and logs in the source environment. [#19994]

Configuration, Documentation, Sample Apps, Portability and Build Changes

  1. Changed SQL API library built on *nix to link with libpthreads when necessary. [#19098]

  2. Added CPPFLAGS into our --enable-jdbc configuration. [#19234]

  3. Added encryption support into the Windows CE build project for SQL API. [#19632]

  4. Fixed a bug in the STAT_INC_VERB() dtrace probe that was causing compiler warnings. [#19707]

  5. Fixed a bug that could cause a trap in db_dump using salvage mode if a page was found that was not associated with any database in the file. [#19974]

  6. On Cygwin, circumvented a bug in libtool that is exposed when building the BDB SQL API in a directory path containing whitespace characters. [#19812]

Example Changes

  1. Update repmgr C, C#, C++, Java examples(ex_rep_mgr, ex_rep_gsg_repmgr, ex_rep_chan, excs_repquote, excxx_repquote, excxx_epquote_gsg, repquote, repquote_gsg) with their related API changes for group membership. [#19586][#19622]

  2. Port ex_rep_chan, ex_rep_gsg_repmgr,ex_rep_gsg_simple, excxx_repquote_gsg_repmgr, excxx_repquote_gsg_simple to Window.[#19890]

Miscellaneous Bug Fixes

  1. Fixed a bug where memory copied from the Java API could leak if flags were not correctly configured. [#19152]

Deprecated Features

  1. None

Known Bugs

  1. Rollback of a dropped table in the SQL layer contains a mutex leak, which can consume all mutex resources if enough rollbacks of table drops are performed. [#20077]

  2. The DB_CONFIG configuration parameters which specify path names currently do not support names containing any whitespace characters. [#20158]

  3. On systems without FTRUNCATE, db_verify will return an error for truncated heap databases. This is a bug in db_verify, the database has been truncated correctly and can be used in the future. [#20195]

  4. An application using queue extents which is append mostly could see a decrease in the buffer pool hit rate due to the failure to remove pages from closed extents from the buffer pool. [#20217]