9 Using Microsoft Products as Sources

Warehouse Builder enables you to source data stored using Microsoft products. This chapter discusses using Microsoft Excel and Microsoft SQL Server as sources.

Using Excel Spreadsheets as Sources

Scenario

A company stores its employee data in an Excel file called employees.xls. This file contains two worksheets: employee_details and job_history. You need to load the data from the employee_details worksheet into a target table in Warehouse Builder.

Solution

To load data stored in an Excel file into a target table, you must first use the Excel file as a source. Warehouse Builder enables you to source data stored in a non-Oracle source, such as Microsoft Excel, using the Heterogeneous Services component of the Oracle Database.

Figure 9-1 describes how the Oracle Database uses Heterogeneous services to access a remote non-Oracle source.

Figure 9-1 Heterogeneous Services Architecture

Description of Figure 9-1 follows
Description of "Figure 9-1 Heterogeneous Services Architecture"

The Heterogeneous Services component in the database communicates with the Heterogeneous Services agent process. The agent process, in turn, communicates with the remote database.

The agent process consists of agent-generic code and a system-specific driver. All agents contain the same agent-generic code. But each agent has a different driver depending on the type of data being sourced.

Case Study

This case study shows you how to use an Excel file called employees.xls as a source in Warehouse Builder.

Step 1: Install ODBC Driver for Excel

To read data from Microsoft Excel, you must have the ODBC driver for Excel installed.

Step 2: Delimit the Data in the Excel File (Optional)

To source data from an Excel file, define a name for the range of data being sourced:

  1. In the employee_details worksheet, highlight the range that you want to query from Oracle.

    The range should include the column names and the data. Ensure that the column names confirm to the rules for naming columns in the Oracle Database.

  2. From the Insert menu, select Name and then Define. The Define Name dialog box is displayed. Specify a name for the range.

Step 3: Create a System DSN

Set up a System Data Source Name (DSN) using the Microsoft ODBC Administrator.

  1. Select Start, Settings, Control Panel, Administrative Tools, Data Sources (ODBC).

    This opens the ODBC Data Source Administrator dialog box.

  2. Navigate to the System DSN tab and click Add to open the Create New Data Source dialog box.

  3. Select Microsoft Excel Driver as the driver for which you want to set up the data source.

    Click Finish to open the ODBC Microsoft Excel Setup dialog box as shown in Figure 9-2.

    Figure 9-2 ODBC Microsoft Excel Setup Dialog Box

    Description of Figure 9-2 follows
    Description of "Figure 9-2 ODBC Microsoft Excel Setup Dialog Box"

  4. Specify a name for the data source. For example, odbc_excel.

  5. Click Select Workbook to select the Excel file from which you want to import data.

  6. Verify that the Version field lists the version of the source Excel file accurately.

Step 4: Create the Heterogeneous Services Initialization File

To configure the agent, you must set the initialization parameters in the heterogeneous services initialization file. Each agent has its own heterogeneous services initialization file. The name of the Heterogeneous Services initialization file is initSID.ora, where SID is the Oracle system identifier used for the agent. This file is located in the ORACLE_HOME/hs/admin directory.

Create the initexcel.ora file in the ORACLE_HOME/hs/admin directory as follows:

HS_FDS_CONNECT_INFO = odbc_excel
HS_AUTOREGISTER = TRUEHS_DB_NAME = hsodbc

Here, odbc_excel is the name of the system DSN you created in Step 3. excel is the name of the Oracle system identifier used for the agent.

Step 5: Modify the listener.ora file

Set up the listener on the agent to listen for incoming requests from the Oracle Database. When a request is received, the agent spawns a Heterogeneous Services agent. To set up the listener, modify the entries in the listener.ora file located in the DATABASE_ORACLE_HOME/network/admin directory as follows:

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = excel)
      (ORACLE_HOME = C:\oracle11g\product\11.1.0\db_1)
      (PROGRAM = hsodbc)
    )
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = C:\oracle11g\product\11.1.0\db_1)
      (PROGRAM = extproc)
      )
  )

  1. For the SID_NAME parameter, use the SID that you specified when creating the initialization parameter file for the Heterogeneous Services, which, in this case, is excelsid.

  2. Ensure that the ORACLE_HOME parameter value is the path to your Oracle Database home directory.

  3. The value associated with the PROGRAM keyword defines the name of the agent executable.

Remember to restart the listener after making these modifications.

Note:

Ensure that the initialization parameter GLOBAL_NAMES is set to FALSE in the database's initialization parameter file. FALSE is the default setting for this parameter.

Step 6: Create an ODBC Source Module

Use the following steps to create an ODBC source module:

  1. From the Project Explorer, create an ODBC source module.

    On the navigation tree, ODBC is listed within the Non-Oracle node under the Databases node.

  2. You can provide the connection information for the source location either at the time of creating the module, or while importing data into this module.

  3. To provide connection information while creating the module, on the Connection Information page, click Edit and provide the following details:

    Ensure that the service name you provide is the same as the SID_NAME you specified in the listener.ora file.

    Provide the host name and the port number using the Host Name and Port number fields respectively.

    Because you are not connecting to an Oracle database, you can provide dummy values for user name and password. The fields cannot be empty.

    The Schema field can be left empty because you will not be importing data from a schema.

Step 7: Import Metadata from Excel Using the Metadata Import Wizard

Use the Metadata Import Wizard to import metadata from the Excel file into Warehouse Builder. Select Tables as the Filter condition. The wizard displays all the worksheets in the source Excel file under the Tables node in the list of available objects.

  1. Select employee_details and use the right arrow to move it to the list of selected objects.

  2. Click Finish to import the data.

    The data from the employee_details worksheet is now stored in a table called employee_details in the ODBC source module.

Step 8: Create a Mapping to Load Data Into the Target Table

In the Warehouse Builder console, expand the module that contains the target table. Use the table called employee_details in the ODBC source module as a source to load data into the target table. Figure 9-3 displays the mapping used to load data into the target table.

Figure 9-3 Mapping to Load Data Into the Target Table

Description of Figure 9-3 follows
Description of "Figure 9-3 Mapping to Load Data Into the Target Table"

Step 9: Deploy the Mapping

Use the Control Center Manager or Design Center to deploy the mapping you created in step 8. Ensure that you first deploy the source module before you deploy the mapping.

Troubleshooting

This section lists some of the errors that you may encounter while providing the connection information.

Error

ORA-28546: connection initialization failed, porbable Net8 admin error

ORA-28511: lost RPC connection to heterogeneous remote agent using
SID=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=localhost)(PORT=1521)))(CONNECT_DATA=(SID=oracledb)))

ORA-02063: preceeding 2 lines from OWB###

Probable Cause

Providing the same SID name as that of your database.

Action

Provide an SID name different from the SID name of your database.

Error

ORA-28500: connection from ORACLE to a non-Oracle system returned this message: 
[Generic Connectivity Using ODBC][H006] The init parameter <HS_FDS_CONNECT_INFO> 
is not set. Please set it in init<orasid>.ora file.

Probable Cause

Name mismatch between SID name provided in the listener.ora file and the name of the initSID.ora file in ORACLE_HOME/hs/admin.

Action

Ensure that the name of the initSID.ora file and the value provided for the SID_NAME parameter in listener.ora file is the same.

Tip:

Ensure that you restart the listener service whenever you make changes to the listener.ora file.

Using SQL Server as a Source

Scenario

Your company has data that is stored in SQL Server and you would like to import this into Warehouse Builder. Once you import the data, you can perform data profiling to correct anomalies, and then transform the data according to your requirements by using mappings.

Solution

In Warehouse Builder, you can connect to non-Oracle data sources. Once connected, you can import metadata just as from any Oracle data source.

Case Study

To connect to SQL Server and import metadata, refer to the following sections:

  1. "Creating an ODBC Data Source"

  2. "Configuring the Oracle Database Server"

  3. "Adding the SQL Server as a Source in Warehouse Builder"

  4. What's Next

If you encounter problems implementing this solution, see "Troubleshooting".

Creating an ODBC Data Source

You must create an ODBC data source to connect to the SQL Server. To do this, you must set up a System Data Source Name (DSN):

  1. Select Start, Control Panel, Administrative Tools, Data Sources (ODBC).

    This opens the ODBC Data Source Administrator dialog box.

  2. Navigate to the System DSN tab and click Add to open the Create New Data Source dialog box.

  3. Select SQL Server as the driver for which you want to set up the data source.

  4. Click Finish to open the Create A New Data Source to SQL Server Wizard.

  5. In the Name field, specify a name for the data source. For example, sqlsource.

  6. In the Server field, select the server to which you want to connect and click Next.

  7. Specify whether the authentication should be done at the Operating System level or at the server level. Click Next.

  8. Select the database file and click Next.

  9. Accept the default values in the next screen and click Finish.

  10. Test the data source to verify the connection.

Configuring the Oracle Database Server

Next, you must configure Oracle Database to connect to SQL Server. Warehouse Builder can then use this configuration to extract metadata from the SQL Server. There are two steps involved in this:

Creating a Heterogeneous Service Configuration File

You must create the heterogeneous file in the ORACLE_HOME\hs\admin directory. The naming convention for this file should be as follows:

  • Must begin with init

  • Must end with the extension .ora

  • Must not contain space or special characters

For example, you can name the file initsqlserver.ora.

Enter the following in the file:

HS_FDS_CONNECT_INFO = sqlsource
HS_FDS_TRACE_LEVEL = 0

Here, sqlsource is the name of the data source that you specified while creating the ODBC data source.

Editing the listener.ora file

You must add a new SID description in the listener.ora file. This file is stored in the ORACLE_HOME/network/admin directory.

Modify the file as shown:

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = sqlserver)
      (ORACLE_HOME = c:\oracle10g\oracle_home)
      (PROGRAM = hsodbc)
    )
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = c:\oracle10g\oracle_home)
      (PROGRAM = extproc)
      )
  )

The SID_NAME parameter must contain the name of the configuration file you created in the previous step. However, it must not contain the init prefix. For example, if the configuration file you created in the previous step was initsqlserver.ora, then the value of the SID_NAME parameter should be sqlserver.

ORACLE_HOME must point to the Oracle home location of your database installation.

The value associated with the PROGRAM keyword defines the name of the executable agent, which, in this case, is hsodbc.

Restart the listener service after making these modifications.

Adding the SQL Server as a Source in Warehouse Builder

The final step involves adding an ODBC module in Warehouse Builder, and importing the data from the SQL server into this module.

To add an ODBC source module in Warehouse Builder:

  1. Within a project in the Project Explorer, navigate to the Databases, Non-Oracle node.

  2. Right-click ODBC and select New.

  3. Create a new ODBC module using the Create Module Wizard.

  4. You can provide the connection information for the source location either at the time of creating the module, or while importing data into this module.

  5. In the Edit Location dialog box, make sure that you enter User Name and Password within double quotation marks ("). For example, if the user name is mark, enter "mark".

  6. For Service Name, enter the SID name you provided in the listener.ora file. Also select the schema from which you wish to import the metadata.

To import metadata into the ODBC module:

  1. Right-click the module and select Import.

  2. Import the metadata using the Import Metadata Wizard.

    The tables and views available for import depend on the schema you selected when providing the connection information.

What's Next

After you successfully import metadata into Warehouse Builder, you can use the data profiling functionality to check the quality of the data. Or you can skip data profiling and proceed with designing a mapping to extract, transform, and load the data.

Troubleshooting

Some of the errors that you may encounter while providing the connection information are listed here:

Error

ORA-28500: connection from ORACLE to a non-Oracle system returned this message: 
[Generic Connectivity Using ODBC][Microsoft][ODBC Driver Manager] Data source name 
not found and no default driver specified (SQL State: IM002; SQL Code: 0)

ORA-02063: preceding 2 lines from OWB_###

Probable Cause

Creating the DSN from the User DSN tab.

Action

Create the DSN from the System DSN tab.

Error

ORA-28500: connection from ORACLE to a non-Oracle system returned this message: 
[Generic Connectivity Using ODBC][Microsoft][ODBC SQL Server Driver][SQL 
Server]Login failed for user 'SA'. (SQL State: 28000; SQL Code: 18456)

ORA-02063: preceding 2 lines from OWB_###

Probable Cause

The user name and password in the Edit Location dialog box are not enclosed within double quotation marks.

Action

Enter the user name and password within double quotation marks.

Tip:

Make sure that you restart the listener service whenever you make changes to the listener.ora file.