Overview of the Data Ingest Web Service

The Data Ingest Web Service loads data into a running data store on the Oracle Endeca Server and can also update existing records.

The Data Ingest Web Service therefore allows you to use a data integration platform, such as Integrator, to load data into an application.

You can access the Data Ingest Web Service WSDL at the following URL:
http://localhost:<port>/ws/diws/<DataStore>?wsdl
where the localhost and port are the host and port of the running Oracle Endeca Server, and DataStore is the name of the Endeca data store.
The Data Ingest Web Service enables performing these tasks:

The Data Ingest Web Service is able to modify a record multiple times in a single transaction (any combination of create, add assignments, delete assignments, and delete record).

The service returns a response indicating the number of records, standard attributes, or managed attribute values that were added or removed as a result of the request. In addition, error messages are returned via a fault mechanism.

The data is sent by an ETL client (such as Integrator) via a program that is running on the client. Typically, ETL client programs written by users use stubs generated from the Data Ingest WSDL and calls from the ETL tool's SDK.

Interaction with transactions

Any request to the Data Ingest Web Service can contain an optional element OuterTransactionId that specifies the ID of an outer transaction (if it has been started by the Transaction Web Service).

This element must be specified as the first element in the request only if a request made by the Data Ingest service is started after a request to start an outer transaction has been made by the Transaction Web Service.

If no outer transactions have been started, the OuterTransactionId should not be specified in the request, or the value of this element should be empty. (If the attribute's value is empty, the request ignores the element and interprets it as not specified.)

About Integrator

Integrator is a high-performance data integration platform that lets you extract source records from a variety of source types (from flat files to databases) and send those records to either the Data Ingest Web Service or the Bulk Load Interface, both of which in turn load the records into the data store.

The records are loaded into the data store via one of the four custom connectors that communicate with the Data Ingest Web Service or a connector that uses the Bulk Load Interface.

For details on Integrator, see the Oracle Endeca Information Discovery Integrator Guide, Oracle Endeca Information Discovery Integrator Server Guide, and Oracle Endeca Information Discovery Integrator Components Guide.

Data Ingest API

The Data Ingest API is a framework that provides ETL developers with a flexible mechanism to load records from an ETL data source to a running data store. Because it is defined by WSDL documents, the Data Ingest API is language-agnostic. That is, it can be used with any programming language that has Web services support. Thus, the API lets developers choose their favorite development environment (Java, Visual Studio .NET, etc.) on which to write their components.

The Oracle Endeca Server API Reference is the documentation generated from the WSDL and XSD files that describe a Web service. This reference provides API-level information about Web services that are packaged with the Oracle Endeca Server. The Oracle Endeca API Reference is located in the doc directory of the Oracle Endeca Server installation.

About the Bulk Load Interface

Besides the Data Ingest API, the Bulk Load Interface is available to ingest records into an Endeca data store. The Bulk Load API exists in the form of a collection of Java classes in a single endeca_bulk_load.jar file, which is shipped in the Endeca Server's apis directory. For information on the Bulk Load API, see Bulk Load API.