Running partial updates using record output files

The Deployment Template includes a sample CAS crawl pipeline that you can also use to run partial updates.

Make sure that you have updated the AppConfig.xml file to use the sample CAS crawl pipeline located in [appdir]/config/cas_crawl_pipeline. Specifically, you will need to update the configDir in the PartialForge section of the document.

To run the sample partial update pipeline:

  1. Run the load_incremental_cas_crawl_data located in the [appdir]/control/cas directory. This script copies incremental crawl data to the data/partials/incoming directory.
  2. Run the partial_update script located in [appdir]/control.
The partial_update script displays the following informational message when the process is complete:
INFO: Partial update script finished.
The Dgraph should now be updated with the changes from your incremental crawls.