Monitoring the data transfer

You check the progress of each step of the data transfer, including whether each step is successful, by querying the ETL_STATUS table in the destination account. The results of the query include:

• The steps that have been or are being performed.
• The status of each step.
• The number of PERSON_IDs that have been processed for the transformation steps.

You can check the progress at any time during and after the data transfer process.

The imports of the data and vocabulary accounts are completed sequentially. Each table is imported separately, so that in case of failure, resuming the process imports only the tables that have not been successfully imported.

Best practices for monitoring the data transfer

• After starting the data transfer, check to make sure that the Import Person step completes.

  The time to complete the Import Person step varies, depending on the size of the data. For example, a data set with patients in the low millions might take a few minutes, though the time varies depending on many factors. If you are importing using files, you can use metrics gathered from the file export to estimate the time needed for the file import.

  The import of other tables with patients in the low millions might take fifteen minutes for each table.

• If you are working on a network that limits your connection time, determine when your connection will time out so that you can reconnect and resume the data transfer.
• Oracle recommends checking the status of the data transfer at least every two to three hours to make sure that the process has not been interrupted.

To monitor the data transfer:

1. In SQL*Plus, connect to the Empirica Healthcare Analysis database using the destination account.
2. Query the ETL_STATUS table in the destination account:

   select step_name, to_char(start_date, 'HH24:MI:SS') start_tm, to_char(end_date, 'HH24:MI:SS') end_tm, pct_complete pct, next_person_id nxt_id, status from etl_status order by start_date;

   The status of each step appears. In the following example, all steps are completed. If you query the ETL_STATUS table while the data transfer is in progress, the STEP_NAME column lists only the jobs that have completed or started.

   

3. Review the STATUS column to check the progress of the data transfer.