Adding new records

The addAssignments element of the ingestRecords operation allows you to add new records to the data store.

The records to be added are considered totally additive. That is, if a record with the same primary key already exists in the data store, the key-value pair list of the added record will be merged into the existing record. If attribute values with the same name already exist, then the added key-value pairs will be additional values for the same attribute (multi-assign).

Note: The addAssignments element is also used to extend (update) existing records. This usage is described in the following chapter.

ingestRecords request

An add-records request uses the ingestRecords operation with the addAssignments element. The record to be added must have a primary-key assignment. It can have other key-value pair assignments as needed.

Note: If you submit the ingestRecords request after a Transaction Web Service request that starts an outer transaction, the request must specify the outer transaction ID. If no outer transactions have been started, the ID attribute must be omitted in the request.
The basic request format is:
<ingest:ingestRecords 
      xmlns:ingest="http://www.endeca.com/MDEX/ingest/1/0" 
      xmlns:mdex="http://www.endeca.com/MDEX/XQuery/2009/09">
   <ingest:addAssignments>
      <mdex:record>
         <primaryKeyProp>primaryKeyValue</primaryKeyProp>
         <!-- List of other property assignments -->
      </mdex:record>
   </ingest:addAssignments>
</ingest:ingestRecords>
For example, this request adds one record (with the primary key P123) to the data store.
<ingest:ingestRecords 
      xmlns:ingest="http://www.endeca.com/MDEX/ingest/1/0" 
      xmlns:mdex="http://www.endeca.com/MDEX/XQuery/2009/09">
   <ingest:addAssignments>
      <mdex:record>
         <partID>P123</partID>
         <color type="mdex:string">red</color>
         <price type="mdex:double">19.99</price>
      </mdex:record>
   </ingest:addAssignments>
</ingest:ingestRecords>

The primary key of the record is the partID primary-key attribute (which in this case must already exist in the data store of the Oracle Endeca Server). The request also creates the color and price attributes, which previously did not exist in the data store.

Success response

An ingestRecordsResponse for a successful add-records request looks like this example:
<ingest:ingestRecordsResponse 
      xmlns:ingest="http://www.endeca.com/MDEX/ingest/1/0">
   <ingest:numPropertiesCreated>2</ingest:numPropertiesCreated>
   <ingest:numRecordsAffected>1</ingest:numRecordsAffected>
   <ingest:numRecordsDeleted>0</ingest:numRecordsDeleted>
</ingest:ingestRecordsResponse>

The sample response shows that one record was created and that two attributes (the color and price attributes) were also created. The partID attribute was not created because it already existed in the data store.

Failure response

On failure, a SOAP fault is returned. The ingestFault and errorDetail elements should contain the error that caused the failure.

For example, assume that one of the record assignments contained a mismatched attribute element that looked like this:
<partNum type="mdex:int">24869</price>
The errorDetail element would return an error similar to this:
'request' cannot be parsed as XML.
Reason: Unable to fetch resource:Expected end of tag 'partNum'

In this example, the reason for the error is that the </partNum> ending tag was not found (because </price> was mistakenly used instead).

State of the data ingest process on failure

The Data Ingest Web Service uses an all-or-nothing insertion strategy for each batch of records. This means that if at least one record in a batch is considered invalid by the Dgraph process of the Oracle Endeca Server, then all of the records are rejected. For example, if a batch of 1000 records contains 999 valid records and 1 invalid record, then the 999 valid records (and the invalid record) are not loaded into the data store.

If the data ingest process is interrupted (for example, by the ETL client or the Dgraph process crashing), then the current batch (i.e., the batch that was being processed when the interruption occurred) is not loaded into the data store. However, all previous valid batches have been loaded into the data store. For example, if 5000 batches are to be loaded and an interruption occurs during batch 3500, then batch 3500 is not loaded into the data store, but the previous 3499 batches will be present in the data store.

Standard attribute assignments and creations

When adding standard attributes, the operation works as follows for the new attribute (i.e., the attribute to be added):
  • If the new attribute already exists in the data store but with a different type, an error is thrown and the new attribute is not added.
  • If the new attribute already exists in the data store and is of the same type, no error is thrown and nothing is done.
Standard attribute names must use an NCName format. The standard attribute name is used as the element name for the assignment, in this format:
<propertyName type="propertyType">property value</propertyName>
For example, assigning a standard attribute named ItemID would look like this:
<ItemID type="mdex:int">247</ItemID>

Standard attributes are created as needed when non-existent attributes are specified for a record. The PDR for the attribute will use the system default settings, which is explained in Default values for new attributes in the data store. Note that you cannot disable this automatic creation of attributes.