Optimizing Parallel Data Load

Parallel data load includes a “throttle” to limit the number of client threads or server pipelines that are used. A throttle control is important because if the data-file specification matches several hundred files, you should avoid spawning that many threads or pipelines. To control the number of threads or pipelines spawned by a data load request, set the limit using the max_threads grammar in the import data MaxL statement

In general, avoid using more pipelines than you have CPU cores. Additionally, monitor the system I/O bandwidth utilization to help determine how many data loads your I/O subsystem can handle in parallel. See Monitoring Processor and I/O Activity

If you are executing a large number of parallel data loads, performance will likely be improved by setting DLSINGLETHREADPERSTAGE to TRUE in the essbase.cfg file.