Skip Headers

# Multiprocessing

Previous

Next

You can use multiple processes or multiple machines to split the batch job into several smaller jobs. In this example, splitting the batch job across threads enables you to synchronize reads from the cursored stream, and use parallel Units of Work on a single machine.

This leads to a performance increase, even if the machine has only a single processor, because it takes advantage of the wait times inherent in SQL execution. While one thread waits for a response from the server, another thread uses the waiting cycles to process its own database operation.

Example: Fully Optimized illustrates the optimized code for this example. Note that it does not illustrate multiprocessing.

Fully Optimized

    /* Read each batch of employees, acquire a Unit of Work, and register them */
    targetSession.getLogin().useBatchWriting();
    targetSession.getLogin().setSequencePreallocationSize(200);
    targetSession.getLogin().bindAllParameters();
    targetSession.getLogin().cacheAllStatements();
    targetSession.getLogin().setMaxBatchWritingSize(200);

    // Read all the employees from the database into a stream.
    // This requires 1 SQL call, but none of the rows will be fetched.
    ReadAllQuery query = new ReadAllQuery(Employee.class);
    query.useCursoredStream();
    CursoredStream stream;
    stream = (CursoredStream) sourceSession.executeQuery(query);
    //SQL: Select * from Employee. Process each batch
    while (! stream.atEnd()) {
        Vector employees = stream.read(100);
        // Acquire a unit of work to register the employees
        UnitOfWork uow = targetSession.acquireUnitOfWork();
        uow.registerAllObjects(employees);
        uow.commit();
    }
    //SQL: Begin transaction
    //SQL: Update Sequence set count = count + 200 where name = 'SEQ'
    //SQL: Select count from Sequence where name = 'SEQ'
    //SQL: Commit transaction
    //SQL: Begin transaction
    //BEGIN BATCH SQL: Insert into Address (...) values (...)
    //... repeat this 100 times
    //Insert into Employee (...) values (...)
    //... repeat this 100 times
    //END BATCH SQL:
    //SQL: Commit transactionJava optimization