Berkeley DB XML 2.3.11 Change Log

Changes since Release 2.3.10 are described here. The change logs for releases 2.3.10 and 2.3.8 are also included, detailing the changes from 2.2.13.

Release 2.3.11 is a patch release for 2.3.10. For detailed information about the 2.3 release itself, see the 2.3.8 Change Log.

It recommended that all users move to this release. Also see the documentation included in your download package or on our website.


2.3 Release Overview

The major focus areas for release 2.3 include:

Changes in BDB XML 2.3.11

This section describes changes in BDB XML 2.3.11 relative to release 2.3.10.

Upgrade Requirements

No upgrade is required for containers created using release 2.3.10.

General Functionality Changes

Previously published patches:
  1. Fixed a bug where certain runtime errors during query execution when using modules could cause a double free
  2. Fixed a bug which incorrectly reported an error for fractional seconds when the seconds filed was "59"
  3. Fixed a problem with XmlIndexLookup where a GT lookup that happens to start with the last entry in the index might return results when it should return none
  4. Improved query performance for queries consisting of a simple path and predicate
  5. Improved index utilization for user defined functions that are called more than once in the query
  6. Fixed certain predicates containing "or" and an unoptimizable sub-expression that did not optimize correctly
  7. Fixed several problems with XmlModify including a SEGV that can occur when removing a last child element that has leading text [#15615]
  8. Fixed a bug in statistics calculation for substring indexes that could result in a crash [#15823]
  9. Fixed a bad exception that might occur when inserting schema-invalid XML [#15824]
  10. Added an index optimized case- and diacritic-insensitive version of the fn:contains()function called dbxml:contains()
New (unpublished) fixes and changes:
  1. Fixed crash in XmlModify removing nodes [#15848]. This was reported on the OTN forum threads here and here
  2. Fixed a bug where a query of transient content inside a transaction could hang [#15905]. This bug also had implications for replication.
  3. Fixed an assertion failure that could occur when querying empty CDATA sections [#15906]
  4. Fixed a very old bug that could result in incorrectly removing a node index entry during a document update. No user ever reported this issue [#15943]
  5. Fixed a bad memory reference and possible crash (never observed) when inserting new content into an existing document [#15974]. This was reported in a forum thread
  6. Fixed a bug that caused documents to seem to disappear if renamed. The content would remain and appear in query results but the name would be removed from the name index [#15993]
  7. Added the (missing) XmlValue constructor for an explicit type to the PHP API [#15996]
  8. Added (missing) XmlResults.size() to the PHP API
  9. Fixed a problem where deadlock exceptions thrown at the wrong time could result in trying to close the same DB cursor twice which could either throw a fatal abort exception or SEGV [#16002]

Changes in BDB XML 2.3.10

This section describes changes in BDB XML 2.3.10 relative to release 2.3.8.

Upgrade Requirements

Upgrade is required for containers created using release 2.2.13. See the 2.3.8 Change Log. Containers creating using 2.3.8 do not require upgrade.

General Functionality Changes

  1. Fixed Windows-mostly problem where dynamic_cast (RTTI) was being used by SWIG-generated code, and RTTI was not enabled in the build. The fix was to use static casting, as it's safe in these cases, rather than enabling costly RTTI. The symptom was a mysterious failure (exception thrown from the C++ runtime library) in Java when instantiating a class that extends either XmlInputStream or XmlResolver [#15280].
  2. Fixed a problem where some code that should have been conditionalized based on the Berkeley DB version was not. The result is failed compilation of CompactCommand.cpp [#15283].
  3. Fixed a bug where namespaces and global variables defined in the XmlQueryContext were not carried forward to the context of a module [#15277].
  4. Fixed a bug in XmlModify where an assertion failure (debug mode) was triggered when removing element nodes in a certain sequence [#15291].
  5. Fixed python build problems in 2.3 which caused the setup.py script to be a no-op on Unix, and even when fixed, there was a compilation problem, again Unix-specific [#15295].
  6. Fix metadata indexing so that a default index on metadata will work [#15300].
  7. Fixed dbxml shell to not reset the base uri in its default context. This interfered with arguments to fn:collection() [#15299].
  8. Fixed a race condition in initializing function signatures. This only manifested when queries were parsed in parallel [#15298].
  9. Fixed an optimisation bug that meant that using an XQuery module would sometimes cause an assert to be triggered [#15290].
  10. Fixed a bug where namespaces and global variables defined in the XmlQueryContext were not carried forward to the context of a module [#15277].
  11. Enhanced error handling in the case of opening missing or partially constructed containers [#15322].
  12. Fixed a bug in XmlModify where some incorrect logic could lead to an incorrect query, assertion failure or SEGV. This bug would only occur after an XmlModify step was performed. It never occurs on the first modification step [#15321].
  13. Added code to detect, and throw an exception for document/container mismatches, where a document obtained from one container is used for an update or delete operation on another container [#15320].
  14. Fixed a bug that caused incorrect query plans to be generated for predicates that used the "or" operator in conjunction with indexes [#15328].
  15. FIxed a problem where writing an attribute with an empty value resulted in a null pointer reference (SEGV). This could happen via XmlEventWriter, which was used by the 2.3 upgrade code [#15335].
  16. Fixed an XmlModify problem in 2.3.8 where inserting a new element with content would result in an exception thrown indicating that an operation was attempted on an invalid context [#15333].
  17. Added -fno-strict-aliasing flag to Java library build if using gcc to avoid over-optimization of some constructs that SWIG uses for casting pointers to/from jlong. The symptom was an exception thrown indicating that there is a null or empty value [#15307].
  18. Fixed a bug that resulted in the starts-with() function always using a substring index, without considering using a prefix operation on equality index [#15337].
  19. Fixed an optimisation problem that caused document-uri() to fail under certain conditions [#15336].
  20. Fixed the Java example modifyDocument.java to close cleanly.
  21. Fixed a bug in the adding of Processing Instructions to a container.

Changes in BDB XML 2.3.8

This section describes changes in BDB XML 2.3.8 relative to release 2.2.13.

Upgrade Requirements:

  1. Changed database format to provide better node storage performance and scalability. Users must read documentation on container upgrade before performing an upgrade, or there is risk of data loss.

New Features:

  1. Conformance to Final Recommendations of XQuery 1.0 and XPath 2.0, dated January 23, 2007.
  2. Use of the XQilla library replaces the use of XQuery and Pathan libraries.
  3. Additional classes and related interfaces to allow direct pull and push access to XML content :

API Changes:

Unless otherwise noted, the API additions apply to all language bindings, and all bindings use the same method name.
  1. Added new interface classes to enable input and output of XML content via push and pull event interfaces, XmlEventReader, XmlEventWriter, and XmlEventReaderToWriter [#11037].
  2. XmlManager:
  3. XmlContainer:
  4. XmlResults:
  5. XmlModify:
  6. XmlQueryContext::
  7. XmlDocument:
  8. XmlException:
  9. XmlValue:
  10. virtual XmlInputStream *XmlResolver::resolveModuleLocation() method -- allows resolution of as XQuery module namespace to a list of locations when there is no location URI.
  11. virtual XmlInputStream *XmlResolver::resolveModule() method -- allows resolution of XQuery modules referenced in a query.
  12. Added DBXML_WELL_FORMED_ONLY flag to use to force the use of the well-formed scanner for documents. This scanner will make no attempt to retrieve schema or DTDs, even if referenced. The Java equivalent for this is XmlDocumentConfig.setWellFormedOnly() [#14055].

Changes That May Require Application Modification:

  1. Changed database format to provide better node storage performance and scalability [#13771].
  2. Fixed a bug in the choice of index used for comparisons. Users with decimal indexes may find that they need to change them to double indexes before they will work correctly [#15093].
  3. Changed the default base URI to be "dbxml:/" (from "dbxml:"), so that is compliant with RFC 2396 (URI specification). Containers that have been opened using an absolute unix path may now have to use aliases to reference them. See the URI rule changes below [#13881].
  4. Changed how BDB XML URIs are resolved against a base URI to be conformant with the URI specification. This affects naming of containers in the arguments to fn:collection() and fn:doc(). Here are the current rules. Notes: by default, there is a base URI of "dbxml:/", and "container_alias" is either a relative path to a container, or an alias for that container, set using XmlContainer::addAlias().
    New style dbxml URIs
     dbxml:/container_alias/document
     dbxml://xxx/container_alias/document (NB xxx is the URL authority, and is ignored)
     dbxml:///container_alias/document
     dbxml:////absolute/path/to/container/document
     dbxml:/C:/windows/container/document
    With base URI
     dbxml:/ + container_alias/document = dbxml:/container_alias/document
     dbxml:/container_alias + container_alias2/document2 = dbxml:/container_alias2/document2
     dbxml:/container_alias/document + /container_alias2/document2 = dbxml:/container_alias2/document2
     dbxml:///container_alias/document + /container_alias2/document2 = dbxml:///container_alias2/document2
     dbxml://xxx/container_alias/document + /container_alias2/document2 = dbxml://xxx/container_alias2/document2
     dbxml:/container_alias/document + document2 = dbxml:/container_alias/document2
    Backwards compatibility mappings
     dbxml:container_alias/document -> dbxml:/container_alias/document
    
    [#14234].
  5. Replaced the Pathan and XQuery libraries with a single library called XQilla, which implements both XPath 2.0 and XQuery 1.0. This simplifies the build and removes redundant code [#13880].

General Functionality Changes:

  1. Added several new XQuery extension functions that expose access to BDB XML functionality, as well as documentation for BDB XML XQuery extensions. New extensions are:
  2. Fixed a bug where the default namespace for an element in no namespace was not redeclared, even though the element was in the middle of a document that already had a default namespace declared. [#13872]
  3. Fixed a bug where a variable without a prefix was using the default namespace from the statically known namespaces [#13868]
  4. Fixed bug where an attempt to open a transactional container that fails due to a version mismatch would result in errors from Berkeley DB regarding "Locker not found" and eventually an environment PANIC, requiring recovery. The same issue could arise if the open were to fail for other reasons as well [#13962].
  5. Fixed a bug where using XmlContainer::updateDocument() on a document returned from a query that used lazy evaluation could result in an assertion failure [#13933].
  6. Fixed a bug where index entries were incorrectly deleted when updating a document either via XmlModify or updateDocument. The problem only occurred for containers without node indexes and when there were multiple, identical index entries being added/deleted during the update [#14173].
  7. Fixed a problem where the Java XmlInputStreamWrap class, which implements java.io.InputStream did not honor the InputStream contract to return -1 at EOF. Instead it returned 0 [#14216].
  8. Fixed a number of places where database errors, such as DB_ERR_DEADLOCK, were getting lost and not properly passed to the caller [#14311][#14212] [#14743][#15087].
  9. Fixed a bug where errors from dbxml_dump were masked, silently resulting in partial dump files if the container had any corruption [#14388].
  10. Changed dbxml_load so that it will not attempt to load into an existing container, as doing so may corrupt the container [#14381].
  11. Fixed various problems in example code [#14392].
  12. Fixed double-delete crash that could occur when using XmlModify to remove and add attributes on the same node, when the node originally has more than one attribute [#14503].
  13. Fixed a problem where leading whitespace in content passed to XmlModify steps could result in exceptions or bad updates [#14629].
  14. Fix a situation where memory could leak if an application were to use another library that includes the Xerces-C library in conjunction with BDB XML [#14451].
  15. Fixed bug where node storage containers could mangle DOCYTPE declarations using PUBLIC [#14725].
  16. Fixed a bug in date/time comparisons [#14949].
  17. Changed index syntax names to match the documentation, as well as be case-insensitive. The old (incorrect) names (e.g. year vs gYear) still work [#14961].
  18. Modified index storage to be more efficient for non-string types, and fixed bugs in the ordering of index entries with certain types (e.g. boolean). Also removed index types of QName and NOTATION, because namespace bindings are not available during indexing, and removed index type of anyURI, because all comparisons on anyURI values are done after casting to the string type. Upgrade will automatically replace these types with an equivalent string index [#14371].
  19. Changed document ids to be 64-bit (were 32) [#15069].
  20. Fixed a bug in fn:lowercase() and fn:uppercase() where they did not handle Unicode characters well [#14382].
  21. Used support in Berkeley DB 4.5 to fully support CDS in BDB XML (deadlock-free, single-writer, multiple-reader concurrency, not transactional). [#14568].
  22. Fixed XmlResults::next/previous iteration to be consistent with the semantics of the underlying object. The previous() call would not move the cursor if it was positioned on the last value in the result set [#15145].
  23. Modified node allocation algorithm for node ids used in partial modifications to better utilize the node id space [#15154].
  24. Fixed a problem where non-content (empty) documents could not be deleted [#15159].
  25. Fixed a problem where node equality check (XmlValue::equals) would fail incorrectly when comparing across XmlDocument objects. Added XmlDocument::equals method to non-C++ APIs [#15212].
  26. Fixed a problem where adding indexes to an encrypted container would not encrypt the new index databases [#15253].
  27. Added code to attempt to identify mismatched versions of Berkeley DB and BDB XML, especially in non-C++ APIs such as Java, Python, and PHP.

Utility Changes:

  1. Modified dbxml shell to use NULL as the default path for the DbEnv so that the DB_HOME environment variable will be used [#15139].
  2. Added commands to dbxml shell:

Java-specific Functionality Changes:

  1. Fixed a Java-specific problem where using XmlDocument.setContentAsXmlInputStream() could lead to a double-delete of the underlying native memory for the XmlInputStream during finalization [#15126].
  2. Fixed Java GettingStarted examples to use XmlContainer.delete() rather than XmlContainer.close(), and call delete() methods on all objects that need it to release container resources [#14668].
  3. XmlContainer.close() is now mapped to XmlContainer.delete(), so it is no longer possible to aggressively close the underlying databases without releasing container references by explicitly deleting BDB XML objects.

Python-specific Functionality Changes:

  1. Added an XmlException class and exception hierarchy. It is described in the file, dbxml/src/python/README.exceptions [#13959].
  2. Modified code generation to enable threads and correctly handle acquisition and release of the Python Global Interpreter Lock (GIL). If threads are not desired, it is possible to compile without them by modifying the source of src/python/dbxml_python_wrap.cpp to disable SWIG_PYTHON_THREADS. This can be done by defining SWIG_PYTHON_NO_THREADS [#14077].

PHP-specific Functionality Changes:

  1. Fixed PHP XmlManager->createContainer() and openContainer() methods to use flags arguments passed in [#14617].
  2. Brought PHP interface up to date with respect to the current API, adding missing interfaces [#13889].
  3. Added XmlException to PHP interface when compiled under PHP5, which supports exceptions. This allows PHP5 scripts to catch XmlException [#15245].

Tcl-specific Functionality Changes:

None

Configuration, Documentation, Portability and Build Changes:

  1. Bundled version 4.5.20 of Berkeley DB, which includes Multi-Version Concurrency Control (MVCC), allowing snapshot semantics for read-write concurrency. This will be quite useful for concurrent BDB XML applications (read/write concurrency).
  2. XQuery and Pathan libraries have been replaced with a single library, XQilla. This library is not currently available from any other location.
  3. Added dependencies to example Windows project files [#13863]. This showed up as a problem in parallel builds using Visual Studio .NET 2005.
  4. Modified Unix configuration and build to allow the BDB XML library (libdbxml*) to built in a directory other than dbxml/build_unix. The buildall.sh script uses the default build locations [#14772].
  5. Consolidated all projects into the single, BDBXML_all.{dsw,sln} files for Windows build, and included all Berkeley DB utilities [#14098].