Skip Headers

Oracle® Argus Mart Installation Guide Release 1.0 Part Number E38590-01 |
Home | Book List | Contents | Index | Contact Us

Previous | Next
| View PDF

# 7 Extracting, Transforming, and Loading Data

This chapter describes the steps required to run the Extract, Transform, and Load (ETL) process using the Oracle Data Integrator Studio and Oracle Data Integrator Console.

The following table illustrates some of the terms along with the name of the Load Plan that has been used to refer the different types of ETL in the later sections of this chapter:

Table 7-1 Describing ETL Types

Type of ETL | Description | Name of the Load Plan
--- | --- | ---
Initial ETL | The Initial ETL process involves full load of data from Argus Safety and DLP to AM. It can be described as the first ETL run that is executed for a fresh setup. | LP_INI_AM
Incremental ETL | The Incremental ETL brings changed case data, from last ETL run till start of current ETL run, from Argus Safety and DLP. The LM/CFG data is reloaded only if any change in record(s) is identified. Dictionary data is always reloaded in case of an Incremental ETL. If a new enterprise is added, the Incremental ETL loads complete data of the new enterprise along with delta data of other enterprises. | LP_INC_AM

The following figure depicts your progress in the complete installation process:

Figure 7-1 Installation Progress: Extracting, Transforming, and Loading Data

This chapter comprises the following sub-sections:

- Managing ETL Process: Oracle Data Integrator Studio

- Managing ETL Process: Oracle Data Integrator Console

## Managing ETL Process: Oracle Data Integrator Studio

This section describes the steps required to manage the ETL process using the Database Integrator Studio.

This section comprises the following sub-sections:

- Running the Initial ETL

- Stopping the Initial ETL

- Restarting the Initial ETL

- Processing a Failed ETL

- Running the Incremental ETL

### Running the Initial ETL

To run the Initial ETL, execute the following steps:

1. Open the Oracle Data Integrator Studio and click Connect to Repository. This displays the Oracle Data Integrator Login window, as depicted in the following figure:

   Figure 7-2 Oracle Data Integrator Login Window

2. In the Oracle Data Integrator Login window:

   1. Select the ODI Work Repository name from the Login Name drop-down list.

   2. Enter the name of the ODI user in the User field.

   3. Enter the password for the ODI user in the Password field. The password for the SUPERVISOR user was specified by you in the Creating Master Repository section while configuring the ODI settings.

   4. Click OK. This displays the Oracle Data Integrator Screen.

3. Select the Operator tab in the left pane.

4. Expand the Load Plans and Scenarios section, as highlighted in the following figure:

   Figure 7-3 Load Plans and Scenarios

   The LP_INI_AM option in this section represents the load plan for the initial ETL process for AM.

5. Right-click the LP_INI_AM option. This displays a menu, as shown in the following figure:

   Figure 7-4 Executing the Initial ETL

6. Click Execute. This displays the Start Load Plan window, as shown in the following figure:

   Figure 7-5 Start Load Plan Window

7. In the Start Load Plan window:

   1. Select CTX_ARGUSMART from the Context drop-down list.

   2. Select LA_AM from the Logical Agent drop-down list.

   3. Select the desired log level from the Log Level drop-down list.

   4. Click OK. This displays the Information dialog box with the Load Plan Started confirmation message, as shown in the following figure:

      Figure 7-6 Load Plan Started Confirmation Message

8. Click OK.

   You can verify the status of the ETL process by navigating to the Load Plan Executions section and expanding the Load Plan folder. You can view the status of the Load Plan in Green color with tilted s, which signifies that the ETL session is in progress, as highlighted in the following figure:

   Figure 7-7 Status of the Load Plan

### Stopping the Initial ETL

To stop the initial ETL, execute the following steps:

1. Right-click the Load Plan, which you want to stop, in the Load Plan folder of the Load Plan Executions section. This displays a menu, as shown in the following figure:

   Figure 7-8 Stopping the Initial ETL

2. Select Stop Normal. This displays the Stop Load Plan dialog box, as depicted in the following figure:

   Figure 7-9 Selecting the Physical Agent

3. Select PA_AM from the Physical Agent drop-down list.

4. Click OK. This stops the execution of the Load Plan.

   You can verify the status of the ETL process by navigating to the Load Plan Executions section and expanding the Load Plan folder. You can view the status of the Load Plan in Red color with the X symbol, which signifies that the ETL session is not in progress, as highlighted in the following figure:

   Figure 7-10 Stopped Initial ETL Session

### Restarting the Initial ETL

Restarting the Initial ETL process enables you to start the ETL process from the last execution step where it was stopped or failed.

To restart the Initial ETL, execute the following steps:

1. Right-click the Load Plan, which you want to restart, in the Load Plan folder of the Load Plan Executions section. This displays a menu, as shown in the following figure:

   Figure 7-11 Restarting the Initial ETL

2. Click Restart. This displays the Restart Load Plan dialog box, as shown in the following figure:

   Figure 7-12 Restart Load Plan Dialog Box

3. Select PA_AM from the Physical Agent drop-down list.

4. Select the required log level from the Log Level drop-down list.

5. Click OK. This displays the Information dialog box with the Load Plan restarted message, as depicted in the following figure:

   Figure 7-13 Loan Plan restarted Confirmation Message

6. Click OK.

   This adds another Load Plan, with the same name as that of the stopped ETL, in the Load Plan folder of the Load Plan Executions section. However, this instance of the ETL Process is in Green color with a tilted S, which signifies that the ETL is in progress, as highlighted in the following figure:

   Figure 7-14 Restarted Load Plan

   Once the ETL process is complete, the Load Plan is displayed in Green color with a completed symbol, as highlighted in the following figure:

   Figure 7-15 Completed Load Plan

### Processing a Failed ETL

If an ETL process fails, you have the option of continuing the process from the failed step or executing it again from the beginning of ETL.

This section explains the steps to continue a failed ETL from the failed step and to execute it again from the beginning of ETL.

This section comprises the following sub-sections:

- Continuing the Failed Initial ETL

- Restarting the Failed Initial ETL

#### Continuing the Failed Initial ETL

The process to continue the failed Initial ETL from the failed step is exactly the same as that of the process of restarting the Initial ETL after stopping it.

See the Restarting the Initial ETL section for the step-by-step procedure to continue the failed Initial ETL from the failed step.

#### Restarting the Failed Initial ETL

The process to restart the failed Initial ETL from the beginning is exactly the same as that of the process of running the Initial ETL.

However, before restarting the ETL you must log on to the Oracle SQL Developer using the Argus ETL User credentials and execute the following statements:

EXEC pkg_sm_stage_util.p_set_cmn_profile_value ('DATABASE', 'ODI_ETL_STATUS', '0');EXEC pkg_sm_stage_util.p_set_cmn_profile_value ('DATABASE', 'ETL_SM_ITERATION_NUMBER', NULL);

COMMIT;

To verify the successful execution of these statements, you can execute the following Select statements:

- SELECT * FROM rm_cmn_profile_global WHERE KEY = 'ODI_ETL_STATUS';

  The entry for the Value column must be 0 after executing this statement, as depicted in the following figure:

  Figure 7-16 Select Statement 1 to Verify Successful Execution

- SELECT * FROM rm_cmn_profile_global WHERE KEY = 'ETL_SM_ITERATION_NUMBER';

  The entry for the Value column must be blank after executing this statement, as depicted in the following figure:

  Figure 7-17 Select Statement 2 to Verify Successful Execution

See the Running the Initial ETL section for the step-by-step procedure to restart the failed Initial ETL from the beginning of ETL.

### Running the Incremental ETL

You can perform all the actions for the Incremental ETL such as running, stopping, and restarting the incremental ETL using the steps given for the Initial ETL.

However, while running the Incremental ETL, you must right-click LP_INC_AM instead of LP_INI_AM (which is used in case of Initial ETL) from the Load Plans and Scenarios section and select Execute, as highlighted in the following figure:

Figure 7-18 Running the Incremental ETL

For step-by-step information related to the Incremental ETL tasks such as Running, Stopping, and Restarting the Incremental ETL, refer to the Oracle Argus Mart Administrator's Guide.

## Managing ETL Process: Oracle Data Integrator Console

This section describes the steps required to manage the ETL process using the Database Integrator Console.

This section comprises the following sub-sections:

- Running the Initial ETL

- Stopping the Initial ETL

- Restarting the Initial ETL

- Processing a Failed ETL

- Running the Incremental ETL

### Running the Initial ETL

To run the Initial ETL, execute the following steps:

1. Open the Oracle Data Integrator Console. This displays the Oracle Data Integrator Console Sign In window, as depicted in the following figure:

   Figure 7-19 Oracle Data Integrator Sign In Window

2. In the Oracle Data Integrator Sign In window:

   1. Select the ODI Work Repository name from the Repository drop-down list.

   2. Enter the name of the ODI user in the User Id field.

   3. Enter the password for the ODI user in the Password field.

   4. Click Sign In. This displays the Oracle Data Integrator Console Screen, as shown in the following figure:

      Figure 7-20 Oracle Data Integrator Console Screen

3. Select the Management tab in the left pane.

4. Expand the Runtime folder and navigate to Runtime > Scenarios/Load Plans > LP_INI_AM, as highlighted in the following figure:

   Figure 7-21 Scenarios/Load Plans

   The LP_INI_AM option in this section represents the load plan for the initial ETL process for AM.

5. Click Execute, as highlighted in the following figure:

   Figure 7-22 Executing the Initial ETL

   This displays the Execute Load Plan window, as shown in the following figure:

   Figure 7-23 Execute Load Plan Window

6. In the Execute Load Plan window:

   1. Select LA_AM from the Logical Agent drop-down list.

   2. Select CTX_ARGUSMART from the Context drop-down list.

   3. Select the desired log level from the Log Level drop-down list.

   4. Click Execute. This displays the Information dialog box with the Load Plan Execution submitted successfully confirmation message, as shown in the following figure:

      Figure 7-24 Load Plan Started Confirmation Message

7. Click OK.

   You can verify the status of the ETL process by expanding the Load Plan Executions folder in the Sessions/Load Plan Executions section. You can view the status of the Load Plan in Green color with tilted s, which signifies that the ETL session is in progress, as highlighted in the following figure:

   Figure 7-25 Status of the Load Plan

### Stopping the Initial ETL

To stop the initial ETL, execute the following steps:

1. Select the Load Plan, which you want to stop, by expanding the Load Plan Executions folder of the Sessions/Load Plan Executions section and click Stop, as shown in the following figure:

   Figure 7-26 Stopping the Initial ETL

   This displays the Stop Load Plan Execution dialog box, as depicted in the following figure:

   Figure 7-27 Stop Load Plan Execution Dialog Box

2. Select Normal from the Stop Type drop-down list.

3. Select OracleDIAgent from the Physical Agent drop-down list.

4. Click Stop. This displays the Information dialog box with the Load Plan was Stopped Successfully confirmation message, as depicted in the following figure:

   Figure 7-28 Load Plan Stopped Confirmation Message

5. Click OK.

   You can verify the status of the ETL process by navigating to the Load Plan Executions folder in the Sessions/Load Plan Executions section. You can view the status of the Load Plan in Red color with the X symbol, which signifies that the ETL session is not in progress, as highlighted in the following figure:

   Figure 7-29 Stopped Initial ETL Session

### Restarting the Initial ETL

Restarting the Initial ETL process enables you to start the ETL process from the last execution step where it was stopped or failed.

To restart the Initial ETL, execute the following steps:

1. Select the Load Plan, which you want to restart, in the Load Plan Executions folder of the Sessions/Load Plan Executions section and click Restart, as shown in the following figure:

   Figure 7-30 Restarting the Initial ETL

   This displays the Restart Load Plan Execution dialog box, as depicted in the following figure:

   Figure 7-31 Restart Load Plan Execution Dialog Box

2. Select OracleDIAgent from the Physical Agent drop-down list.

3. Select the required log level from the Log Level drop-down list.

4. Click Restart. This displays the Information dialog box with the Load Plan restarted message, as depicted in the following figure:

   Figure 7-32 Loan Plan restarted Confirmation Message

5. Click OK.

   This adds another Load Plan, with the same name as that of the stopped ETL, in the Load Plan Executions folder of the Sessions/Load Plan Executions section. However, this instance of the Load plan is in Green color with a tilted S, which signifies that the ETL is in progress.

### Processing a Failed ETL

If an ETL process fails, you have the option of continuing the process from the failed step or executing it again from the beginning of ETL.

This section explains the steps to continue a failed ETL from the failed step and to execute it again from the beginning of ETL.

This section comprises the following sub-sections:

- Continuing the Failed Initial ETL

- Restarting the Failed Initial ETL

#### Continuing the Failed Initial ETL

The process to continue the failed Initial ETL from the failed step is exactly the same as that of the process of restarting the Initial ETL after stopping it.

See the Restarting the Initial ETL section for the step-by-step procedure to continue the failed Initial ETL from the failed step.

#### Restarting the Failed Initial ETL

The process to restart the failed Initial ETL from the beginning is exactly the same as that of the process of running the Initial ETL.

However, you need to execute certain steps before restarting the Failed Initial ETL, refer to the Restarting the Failed Initial ETL section for the complete details.

See the Running the Initial ETL section for the step-by-step procedure to restart the failed Initial ETL from the beginning of ETL.

### Running the Incremental ETL

You can perform all the actions for the Incremental ETL such as running, stopping, and restarting the incremental ETL using the steps given for the Initial ETL.

However, while running the Incremental ETL, you must select LP_INC_AM instead of LP_INI_AM (which is used in case of Initial ETL) from the Load Plans folder of the Scenarios/Load Plans section and click Execute, as highlighted in the following figure:

Figure 7-33 Running the Incremental ETL

For step-by-step information related to the Incremental ETL tasks such as Running, Stopping, and Restarting the Incremental ETL, refer to the Oracle Argus Mart Administrator's Guide.