Skip Headers
Oracle Internet Directory Administrator's Guide
10g (10.1.4.0.1)

Part Number B15991-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

9 Using Bulk Tools

As of 10g (10.1.4.0.1), the bulk tools have been rewritten as C executables. The new versions of the tools provide some new features:

In addition to this chapter, bulk tools are also discussed in the following sections of this book:


See Also:

The chapter on Oracle Internet Directory server administration tools in Oracle Identity Management User Reference.


Note:

Stop all instances of Oracle Internet Directory before using bulkload. Before using any of the other bulk tools, either stop all instances of Oracle Internet Directory or disable the entry cache.

This chapter consists of the following sections:

9.1 bulkload

The bulk loader, bulkload, is a bulk management tool. It takes input data in LDIF or SQL*Loader format and loads that data directly into Oracle Internet Directory's schema in the metadata repository. It has three main phases: check, generate and load.

In the check phase, bulkload parses and verifies the LDIF input data for the schema.

In the generate phase, bulkload generates intermediate files in SQL*Loader format.

In the load phase, bulkload can use either of two methods: bulk mode loading or incremental mode loading.

Bulk mode loading is faster than incremental mode loading.

The bulk loader also supports the following features:

At the beginning of the generate phase, the server's orclServerMode is changed from read-write to read-modify. At the end of the generate phase, it is left in the read-modify state so that you cannot add entries to Oracle Internet Directory between the generate and load phases. This is necessary to maintain internal sequence numbers. You are expeced to run the load phase immediately after the generate phase. At the end of the load phase, the servers' orclServerMode is set back to read-write. Using bulkload with the recover option also sets orclServerMode back to read-write.

The bulkload tool generates the following output files in the $ORACLE_HOME/ldap/log directory:

It generates the following output files in the $ORACLE_HOME/ldap/load directory:


Note:

If the applicable password policy has the pwdmustchange attribute set to true, then for every new entry loaded by bulkload, the pwdreset attribute is set to 1 by default. See Chapter 19, "Password Policies in Oracle Internet Directory" for more information.


Note:

If you do not use the bulkload utility to populate the directory, then you must run the oidstats.sql tool to avoid significant search performance degradation.


See Also:


9.1.1 bulkload Command Line Parameters

The bulkload tool accepts parameters in key=value format.

bulkload connect=connect_string
{[check="TRUE"|"FALSE" [restore="TRUE"|"FALSE"] [threads=num_of_threads] 
[file=ldif_file]][generate="TRUE"|"FALSE" [append="TRUE"|"FALSE"] [restore="TRUE"|"FALSE"] [threads=num_of_threads] file=ldif_file]
[load="TRUE"|"FALSE"  [append="TRUE"|"FALSE"] [threads=num_of_threads]]
[index="TRUE"|"FALSE"] [recover="TRUE"|"FALSE"]}
[encode=character_set] [debug="TRUE"|"FALSE"] [verbose="TRUE"|"FALSE"]

Some of the parameter combinations are valid while others are invalid.

You must specify at least one of the following actions when you invoke bulkload: check, generate, load, append, recover, or index.

If check is TRUE, bulkload performs a schema check.

If generate is TRUE, bulkload generates intermediate files.

When using the check or generate action, you must specify the pathname to the LDIF data file.

If load is TRUE, bulkload loads intermediate files.

When append is TRUE, bulkload can perform its actions while the server is up and running.

Use the restore flag only when the LDIF file contains operational attributes, such as orclguid or creatorsname.

Do not specify recover with any other option.

The option combination check index verifies the existing indexes.

9.1.2 Importing an LDIF File by Using bulkload

To import an LDIF file, you use the bulkload utility. This section discusses the tasks to process an LDIF file through bulkload.


See Also:

The bulkload command-line tool reference in Oracle Identity Management User Reference.

This section contains these topics:

9.1.2.1 Task 1: Back Up the Oracle Database Server

Before you import the file, back up the Oracle database server as a safety precaution.


See Also:

Oracle Database Backup and Recovery Basics in the Oracle Database Documentation Library

9.1.2.2 Task 2: Find Out the Oracle Internet Directory Password

To use bulkload, you must provide the Oracle Internet Directory password. The default password is ods, although the system administrator can change it by using the OID Database Password Utility.


See Also:

The oidpasswd command-line tool reference in Oracle Identity Management User Reference

9.1.2.3 Task 3: Check Input for Schema and Data Consistency Violations and Generate the Input Files for SQL*Loader

On UNIX, the bulkload tool usually resides in $ORACLE_HOME/ldap/bin. On Microsoft Windows, this tool usually resides in ORACLE_HOME\ldap\bin.

Check the input file and generate files for the SQL*Loader by typing:

bulkload connect="connect_string" check="TRUE" generate="TRUE" \
   file="path_to_ldif-file_name"

All schema violations are reported in $ORACLE_HOME/ldap/log/bulkload.log. All bad entries are logged in $ORACLE_HOME/ldap/load/badentry.ldif. Use a text editor to fix all bad entries, then re-run bulkload with the check and generate options.

If there are duplicate entries, their DNs are logged in $ORACLE_HOME/ldap/log/duplicateDN.log. This is just for information purpose. The bulkload tool does not generate duplicate data for duplicate entries. It ignores duplicate entries.

When bulkload completes successfully, it generates *.ctl and *.dat files in the $ORACLE_HOME/ldap/load directory to be used by SQL*Loader in load mode. Do not modify these files.

9.1.2.4 Task 4: Load the Input Files

After you have generated the input files, rerun bulkload with the load option. During this step, the *.dat files, which are in Oracle SQL*Loader specific format, are loaded into the database and the attribute indexes are created. The syntax is:

bulkload connect="connect_string" load="TRUE"

9.1.2.5 If Bulk Loading Fails

All loading errors are reported in the $ORACLE_HOME/ldap/log directory. They reside in bulkload.log and in the SQL*Loader-generated files *.bad and bsl_*.log.If bulk load fails, the database might be in an inconsistent state. If this occurs, you should restore the database to its state prior to the bulkload operation. You can do this either by using bulkload with the recover option or by restoring Oracle Internet Directory directory from a backup taken before you invoked bulkload.

9.1.3 More bulkload Examples

The following examples show some uses of bulkload.

9.1.3.1 Example 1. Loading in Bulk Mode

Typically, you load directory data just after installing Oracle Internet Directory. This requires three actions:

  • Check the LDIF file for schema errors and generate the intermediate files

  • Load the data into the Oracle Internet Directory metadata repository.

You perform these actions with command lines similar to these:

bulkload connect="conn_str" check="TRUE" generate="TRUE" file="LDIF_file" 
bulkload connect="conn_str" load="TRUE" 

You can omit the check phase if the LDIF data is from another Oracle Internet Directory node.

9.1.3.2 Example 2. Loading in Incremental or Append Mode

If you must add entries to an Oracle Internet Directory server that already contains data, and the server must be up and running at the same time, then you must use the incremental or append mode. This mode is usually faster than other methods of adding entries to the directory. However, you must ensure that the Oracle Internet Directory LDAP instances are in read-modify mode so that bulkload can append data. You invoke bulkload in incremental or append mode with command lines similar to these:

bulkload connect="conn_str" check="TRUE" generate="TRUE" append="TRUE" \
  file="LDIF_file" 
bulkload connect="conn_str" load="TRUE" append ="TRUE" 

9.1.3.3 Example 3. Index Verification

The bulkload operation can either update indexes or create indexes. Sometimes, however, bulkload does not update or create the indexes properly. This is typically due to issues like improper sizing. If this happens, you can use bulkload to verify and re-create all the indexes.

Use the following syntax to invoke bulkload for verification of indexes:

bulkload connect="conn_str" check="TRUE" index="TRUE"

9.1.3.4 Example 4. Index Re-creation

To re-create indexes, use the following syntax:

bulkload connect="conn_str"  index="TRUE"

9.1.3.5 Example 5. Data Recovery

The load phase of bulkload can fail because of issues like improper disk sizing. After such a failure, the directory data might be inconsistent. You can use the recover option to return the directory data to its pre-bulkload state. The syntax is:

bulkload connect="conn_str" recover="TRUE"

9.2 bulkmodify

The bulkmodify tool is useful for modifying the attributes of a large number of entries in an existing directory. It can perform add and replace operations on attribute values. It can operate on a naming context. Using filters, it can also operate selectively on a few entries under a specified naming context.

The bulkmodify tool does not allow add or replace operations on the following attributes:

It does not allow replace operation on the attribute objectclass.

It does not allow add for single-valued attributes.Output from bulkmodify is logged in $ORACLE_HOME/ldap/log/bulkmodify.log.


See Also:

The bulkmodify command-line tool reference inOracle Identity Management User Reference

9.2.1 bulkmodify Command Line Parameters

Bulkmodify accepts parameters in key=value pairs.

bulkmodify connect=connect_string basedn=Base_DN {[add="TRUE"|"FALSE"]|[replace="TRUE"|"FALSE"]} attribute=attribute_name value=attribute_value [filter=filter_string] [size=transaction_size] 
[threads=num_of_threads] [debug="TRUE"|"FALSE"] [encode=character_set] [verbose="TRUE"|"FALSE"]

The number of threads should be from one to six times the number of CPUs.

Select either the add or the replace option. By default both are set to FALSE.

9.2.2 bulkmodify Usage Examples

The following examples show some uses of bulkmodify:

9.2.2.1 Example 1. Adding a Description for All Entries Under a Specified Naming Context

This example adds descriptions for all the entries under "c=us".

bulkmodify connect="connect_str"  basedn="c=us" add="TRUE" \
    attribute="description"  value="US citizen" filter="objectclass=*"

9.2.2.2 Example 2. Adding telephonenumber for All Entries Under a Specified Naming Context with the Same Manager

This example adds telephonenumber for all the entries under "c=us" that have the manager Anne Smith.

bulkmodify connect="connect_str" basedn="c=us" add="TRUE" \
  attribute="telephoneNumber" value="408-123-4567" filter="manager=Anne Smith"

9.2.2.3 Example 3. Replacing an Attribute for All Entries Under a Specified Naming Context

This example replaces pwdreset for all the entries under "c=us".

bulkmodify connect="connect_str"  basedn="c=us" replace="TRUE" \
  attribute="pwdreset" value="1" filter="objectclass=*" 

9.3 bulkdelete

Bulkdelete is useful for deleting the attributes of a large number of entries in an existing directory. Bulkdelete can delete entries specified under a naming context. By default, it deletes entries completely. It removes all traces of an entry from the database. If you use the option cleandb=FALSE, bulkdelete turns all entries into tombstone entries instead of deleting them completely.

Bulkdelete output is logged in $ORACLE_HOME/ldap/log/bulkdelete.log.


See Also:

The bulkdelete command-line tool reference in Oracle Identity Management User Reference

9.3.1 bulkdelete Command Line Parameters

The bulkdelete tool accepts all parameters as key=value pairs.

bulkdelete connect=connect_string {[basedn=Base_DN]|[file=file_name]} [cleandb="TRUE"|"FALSE"] [size=transaction_size] [encode=character_set] [debug="TRUE"|"FALSE"] [threads=num_of_threads] [verbose="TRUE"|"FALSE"]

Select either the basedn or the file option.If cleandb is TRUE, bulkdelete removes entries completely from the database. By default cleandb is set to TRUE. The number of threads should be from one to six times the number of CPUs.

9.3.2 bulkdelete Usage Examples

The following examples demonstrate how to use bulkdelete.

9.3.2.1 Example 1. Delete All Entries Under a Specified Naming Context from Database

This example deletes all the entries under "c=us".

bulkdelete connect="connect_str" basedn="c=us" cleandb="TRUE"  

9.3.2.2 Example 2. Delete Entries Under a Naming Context and Make them Tombstone Entries

This example deletes all the entries under "c=us" and leaves them as tombstone entries.

bulkdelete connect="connect_str" basedn="c=us" cleandb=FALSE

9.3.2.3 Example 3. Delete Entries Under Specified Naming Contexts Given in File and Make them Tombstone Entries.

This example deletes all the entries under given basedn's specified in file and leaves them as tombstone entries.

bulkdelete connect="connect_str" file="file" cleandb=FALSE 

9.4 ldifwrite

The ldifwrite tool is used to dump of data from an Oracle Internet Directory store to a file. Having the data in a file facilitates loading the data into another node for replication or backup storage. As it writes to the output file, the ldifwrite tool performs a subtree search, including all entries below the specified DN, as well as the DN itself. It dumps data in LDIF format. It can also dump entries under a specified replication agreement DN.

The ldifwrite tool can dump entries located by using specified filters.Output from ldifwrite is logged in $ORACLE_HOME/ldap/log/ldifwrite.log.


See Also:

The ldifwrite command-line tool reference in Oracle Identity Management User Reference

9.4.1 ldifwrite Command Line Parameters

The ldifwrite tool accepts all parameters in keyword=value pair format.

ldifwrite connect=connect_string basedn=Base_DN ldiffile=LDIF_Filename [filter=LDAP_Filter] [threads=num_of_threads] [debug="TRUE"|"FALSE"] [encode=character_set] [verbose="TRUE"|"FALSE"]

Use the basedn option to specify the base DN or replication agreement DN.

The number of threads should be from one to six times the number of CPUs.

9.4.2 ldifwrite Usage Examples

The following examples show some uses of ldifwrite.

9.4.2.1 Example 1. Dumping All Entries Under a Specified Naming Context to an LDIF File

This example writes all the entries under "ou=Europe, o=imc, c=us" into the output.ldif file.

ldifwrite connect="connect_str" basedn="ou=Europe, o=imc, c=us" \
     ldiffile=output.ldif 

The ldifwrite tool includes the operational attributes of each entry in the directory, including createtimestamp, creatorsname, and orclguid.

9.4.2.2 Example 2. Dumping Part of a Specified Naming Context to an LDIF File

This example uses the following naming context objects defined in partial replication:

dn: cn=includednamingcontext000001, cn=replication namecontext,
 orclagreementid=000001, orclreplicaid=node replica identifier,
 cn=replication configuration
orclincludednamingcontexts: c=us
orclexcludednamingcontexts: ou=Americas, c=us
orclexcludedattributes: userpassword
objectclass: top
objectclass: orclreplnamectxconfig

In this example, all entries under "c=us" are backed up except "ou=Americas,c=us". The userpassword attribute is also excluded. The command is:

ldifwrite connect="conn_str" \
  basedn="cn=includednamingcontext000001, cn=replication namecontext, \  
  orclagreementid=000001,orclreplicaid=node replica identifier,\ 
  cn=replication configuration" ldiffile="ldif_file_name" 

9.4.2.3 Example 3. Dumping Entries Under a Specified Naming Context to an LDIF File

This example writes all the entries that satisfy LDAP search filter criteria under "ou=Europe, o=imc, c=us" into the output.ldif file.

ldifwrite connect="connect_str" basedn="ou=Europe, o=imc, c=us" filter="uid=abc" \ 
  ldiffile="output.ldif" 

9.5 catalog

The catalog tool is useful for creating indexes for or dropping indexes from existing attributes. The catalog tool makes an attribute searchable.Output from catalog is logged in $ORACLE_HOME/ldap/log/catalog.log.

9.5.1 catalog Command Line Parameters

Catalog accepts all parameters in key=value pair format.

catalog connect=connect_string {[add="TRUE"|"FALSE"]|[delete="TRUE"|"FALSE"]} {[attribute=attribute_name]|[file=file_name]} [logging="TRUE"|"FALSE"] [threads=num_of_threads] [debug="TRUE"|"FALSE"] [verbose="TRUE"|"FALSE"]

Select either the add or the delete option. By default both are set to FALSE.

The number of threads should be from one to six times the number of CPUs.

If logging is TRUE, catalog generates a redo log.

You can specify only one attribute argument on the command line at a time. To add or delete more than one attribute in a single command invocation, use the file option and specify a list of attributes in the file. Use one line per attribute, for example:

description
sn
title

9.5.2 catalog Usage Examples

The following examples show some uses of catalog:

9.5.2.1 Example 1. Changes a Searchable Attribute into a Non-searchable Attribute

This example drops an index from the attribute title.

catalog connect="connect_str" delete="TRUE"  attribute="title" 

9.5.2.2 Example 2. Changes a Non-searchable Attribute into a Searchable Attribute

This example adds an index to the attribute title.

catalog connect="connect_str" add="TRUE" attribute="title"