Skip Headers

Oracle® Content Management SDK Administrator's Guide
10g (9.0.4)
Part No. B10701-02
  Go To Table Of Contents
Contents
Go To Index
Index

Previous Next  

A Oracle Text Reference

Oracle Content Management SDK (Oracle CM SDK) uses Oracle Text to facilitate full-text search and other advanced capabilities. The speed with which results are returned depends on several factors, including the quality of the Oracle Text index used with Oracle CM SDK (IFS_TEXT). In addition, the end user's experience of the "performance" of the search can depend on how much time you let elapse before an in-progress search times out.

This appendix provides information about how to maintain the Oracle Text index to ensure optimal Oracle CM SDK performance, and includes these topics:

Previous names for Oracle Text include Oracle Context and Oracle interMedia Text. Many of the underlying indexes, views, tables, and various PL/SQL packages referred to in much of the administrator and application developer documentation still use Context or interMedia-related nomenclature. For example, the database schema that owns all Oracle Text objects, such as the indexes, is CTXSYS.

For detailed information about Oracle Text, visit the Oracle Technology Network (http://otn.oracle.com/products/text/).

Oracle Text Tablespaces and Disk Utilization

Disk space for Oracle Text is divided among three distinct tablespaces:

Use this information to estimate and plan disk storage needs for your Oracle CM SDK instance.

Creating and Maintaining the Oracle Text Index

At the end of the configuration process for Oracle CM SDK, the Configuration Assistant uses the SQL scripts shown in Table A-1 to create and populate the IFS_TEXT index.

These scripts are located in the following directory:

$ORACLE_HOME/ifs/cmsdk/admin/sql

If the installation process fails and you need to run these scripts manually, you should execute the scripts at a SQL*Plus prompt after logging on as the Oracle CM SDK schema user (for example, IFSSYS).

Table A-1 SQL Scripts for Creating Oracle Text Index

Script Usage
CreateContextFunnelProcedure.sql Creates the procedure used by USER DATASTORE.
GrantContextToIFS.sql Grants the Oracle CM SDK user (schema) privileges on the Oracle Text-specific commands required to maintain the index.
CreateContextPreferences.sql Tablespace and other text preferences are created by the Oracle CM SDK user.
CreateContextIndex.sql Creates the IFS_TEXT index based on the text preferences.

Maintaining the IFS_TEXT Index By Using the Oracle Text PL/SQL Packages

Two PL/SQL procedures are provided with Oracle Text for maintaining the index. Unlike a regular database index, the Oracle Text index is not dynamically updated with each insert or update of information. Rather, the index must be refreshed (or synchronized) periodically, using the Oracle Text stored procedure ctx_ddl.sync_index.

The ctx_ddl.sync_index procedure does not rebuild the entire index, but adds and deletes records that have changed since the last synchronization. Since the changes are incremental, the more frequently this procedure is run, the faster it runs. Over the course of time, however, the index can become fragmented, so a companion procedure (ctx_ddl.optimize_index) is provided to optimize the index.

During Oracle CM SDK configuration, the procedures to sync and optimize the IFS_TEXT index are automatically set up to run periodically in the background, using the DBMS_JOBS package of the Oracle9i Database Server. DBMS_JOBS procedures, which are similar to cron jobs on UNIX systems, are portable across all platforms on which the Oracle9i Database Server runs.

When the Oracle CM SDK schema is created during configuration, two DBMS_JOBS are set up: Sync Job and Optimize Job.


Note:

Sync Job and Optimize Job are only automatically created when a new schema is created. If you are upgrading from an existing schema, these jobs will not be created automatically.

Sync Job

Sync Job will periodically call the 'ctx_ddl.sync_index()' method. This method indexes the documents that were created or updated since the last run. By default, this job is set up to run every 30 minutes.

Optimize Job

Optimize Job will periodically call the 'ctx_ddl.optimize_index()' method. The goal of this job is to optimize the IFS_TEXT index by defragmenting it. By default, this job is run in FULL mode, with a maximum of one hour allocated for the optimization task. The job is set up to run every 24 hours, starting at midnight.

Monitoring DBMS_JOBS

DBMS_JOB log files can be found under the $ORACLE_HOME that hosts the Oracle9i Database Server, in the directory that holds the background process logs. This directory is pointed to by the BACKGROUND_DUMP_DEST configuration parameter of the database server. You can recognize the log trace files by their name pattern, DBNAME_j###_process-id.trc.

Another database configuration parameter, JOB_QUEUE_PROCESSES, determines how many processes are available at any given time to run all background tasks. You may need to increase the value of this parameter if not enough processes are available to run Sync Job and Optimize Job. The default value is 10.

You can also look at the USER_JOBS view to see a list of all the jobs set up by the current schema user. The USER_JOBS view shows details such as the PL/SQL being run by each job, the last time each job was run, and when the jobs are scheduled to be run next. To see the USER_JOBS view, log on to the Oracle CM SDK schema using SQL*Plus.

Changing or Removing the Default DBMS_JOBS

Two SQL files are used to set up and clear DBMS_JOBS in Oracle CM SDK: SetupContextJobs.sql and ClearContextJobs.sql. These files are located in the following directory:

$ORACLE_HOME/ifs/cmsdk/admin/sql

SetupContextJobs.sql is used by the system during configuration to set up Sync Job and Optimize Job. ClearContextJobs.sql is provided for you to remove Sync Job and Optimize Job, in case you want to set up your own DBMS_JOBS.

See the Oracle9i Database Administrator's Guide for information about setting up your own DBMS_JOBS. You can also look at Sync Job and Optimize Job as examples.

Manually Synchronizing and Optimizing IFS_TEXT

To synchronize an existing IFS_TEXT index, use SQL*Plus to connect as the Oracle CM SDK schema user, and enter:

exec ctx_ddl.sync_index('ifs_text');

You can also run the SyncContextIndex.sql script from the $ORACLE_HOME/ifs/cmsdk/admin/sql directory. In addition to synchronizing the IFS_TEXT index, this script will print extra log information to the console.

To optimize an existing IFS_TEXT index, use SQL*Plus to connect as the Oracle CM SDK schema user, and enter:

exec ctx_ddl.optimize_index('ifs_text', 'FAST');

or

exec ctx_ddl.optimize_index('ifs_text', 'FULL', maxtime);

Monitoring Oracle Text Indexing of Oracle CM SDK Documents

Oracle CM SDK provides some utility-type SQL scripts to facilitate interaction with Oracle Text (see Table A-2). Read each .sql file for additional usage details. All scripts are available in:

$ORACLE_HOME/ifs/cmsdk/admin/sql

Table A-2 SQL Scripts for Monitoring Oracle Text Indexing

Script Usage
ViewContextErrors.sql Script that decodes the operating system specific errors that were generated during Oracle Text indexing.
SyncContextIndex.sql Script that synchronizes the Oracle Text index and enables you to monitor the Oracle Text synchronization process. Uncomment the first two lines in the script, which includes call to ctx_output.add_event(), to monitor on a rowid by rowid basis.
ViewDocumentByRowID.sql Script that enables you to view additional information about a document that is indexed by Oracle Text. Use the docid from the Oracle Text log with this script.

Indexing Non-Standard Content Types

Oracle CM SDK does not by default index every file that is loaded into the system, but you can configure it to index any document you choose. To do this, designate the MIME type as "Indexed" on the New Format page (or Edit Format page, if the format already exists) in Oracle CM SDK Manager. The MIME type of a document is determined by its extension.

For example, you may want to index all your .java source code files. To do so:

  1. Use Oracle CM SDK Manager to add the .java MIME type and designate it as "Indexed" on the New Format page.

  2. Insert the files into the repository.

  3. Sync the index using the procedure discussed in "Manually Synchronizing and Optimizing IFS_TEXT".

See Table 7-6 for a list of default Oracle CM SDK formats.

Theme Indexing

Oracle Text can generate themes based on a file's content. By default, Oracle CM SDK disables theme indexing because of the overhead involved with indexing thematic content. See the Oracle Text Application Developer's Guide for more information about themes.

Enabling or disabling Oracle Text theme indexing causes all documents in Oracle CM SDK to be re-indexed, which can take considerable time. To reduce the impact on your user community, be sure to perform either of these tasks after hours or when system usage is low.

Enabling Theme Indexing

Use SQL*Plus to connect as the Oracle CM SDK schema user, and enter:

exec ctx_ddl.set_attribute('ifs_default_lexer', 'index_themes', 'yes');
exec ctx_ddl.set_attribute('ifs_default_lexer', 'theme_language','english');
alter index ifs_text rebuild parameters('replace lexer ifs_global_lexer');

Disabling Theme Indexing

In SQL*Plus, connect as the Oracle CM SDK schema user, and enter:

exec ctx_ddl.set_attribute('ifs_default_lexer', 'index_themes', 'no');
alter index ifs_text rebuild parameters('replace lexer ifs_global_lexer');

The index is rebuilt and theme indexing is disabled.

Modifying the Search Timeout Parameter

The IFS.SERVICE.SESSION.DefaultSearchTimeoutPeriod service configuration parameter specifies the timeout period for a running search that has not yet returned results. The default setting for this parameter (in the default service configurations) is 60 seconds. Increase this value and users will wait longer than a minute before a search times out; decrease the value to shorten the time in which a running search will time out.

Troubleshooting Oracle Text Problems

This section provides Oracle Text troubleshooting information.

Table A-3 Troubleshooting Oracle Text Problems

Problem Probable Cause Corrective Action
Cannot search on document contents of any documents. Documents have not been indexed. Start the database instance and make sure that the Oracle Text indexing jobs are running. See "Creating and Maintaining the Oracle Text Index" for more information.
Cannot search on document contents after upgrading the Oracle CM SDK schema. Oracle Text index was not re-populated. See "Repopulate the Oracle Text Index" in the Oracle Content Management SDK Installation and Configuration Guide for more information.
Server is slow only on content-based search activity. Probable cause #1: Oracle Text tablespaces are on the same disk as other database files.Probable cause #2: Oracle Text indexes have become fragmented. Move the Oracle Text tablespaces to other disks. See the Oracle9i Database Administrator's Guide for more information on moving tablespaces.Regularly optimize the Oracle Text index INDEXEDLOB_I (for upgraded schemas), or GLOBALINDEXEDBLOB_I (for new schemas). See "Manually Synchronizing and Optimizing IFS_TEXT" for more information.
Oracle CM SDK rows show up in the Oracle Text view ctx_user_index_errors. Oracle CM SDK documents are corrupt or do not have the correct extension.
  1. Determine which Oracle CM SDK document is being referred to, based on the err_textkey from ctx_user_index_errors.
    SQL*Plus cmsdk_schema/schema_password
    
    select du.uniquename, vd.name, co.contentsize, cs.id, vd.id
    from odmv_document vd, odm_contentobject co, odmm_contentstore cs, odm_document od, odm_directoryuser du
    where vd.id = od.id
    and od.contentobject = co.id
    and co.content = cs.id
    and du.id = vd.owner
    and cs.id in
    (
    select distinct od.id
    from ctx_user_index_errors cp, odmm_ contentstore od
    where od.rowid = err_textkey
    )
    order by cs.id;
    
    
  2. cd $ORACLE_HOME/ifs/cmsdk/bin

  3. ifslogin system/system_password

  4. ifsshowallpaths -id vd.idreplacing vd.id with the vd.id returned from the Select statement above. This command will display the absolute path of the document.

  5. Examine this document, looking for these problems:

    Is the file damaged in any way?

    Is the file name extension correct for this document?

    Is the character set of the document correct?

  6. If no obvious problems are found, send the document to your Oracle Support Representative for further diagnosis.

Oracle CM SDK rows never get processed and never leave the Oracle Text view ctx_user_pending. Oracle CM SDK documents are corrupt or do not have the correct extension.
  1. Follow the steps in "Oracle CM SDK rows show up in the Oracle Text view ctx_user_index_errors." to determine which Oracle CM SDK documents are being referred to, substituting ctx_user_pending for ctx_user_index_errors and pnd_rowid for err_textkey.
  2. Examine this document, looking for these problems:

    Is the file damaged in any way?

    Is the file name extension correct for this document?

    Is the character set of the document correct?

  3. If no obvious problems are found, send the document to your Oracle Support Representative for further diagnosis.

  4. Delete the document from Oracle CM SDK.

Searching on the contents of new documents stops working. A recent document has caused Oracle Text server to fail.
  1. Log in to SQL*Plus as cmsdk_schema/schema password, and issue:
    select count(*) from ctx_user_pending;
    
    
  2. If there are any rows in that view and the rows are not changing, then a recent document has caused Oracle Text to stop indexing. To determine which Oracle CM SDK documents these rows refer to, see the problem "Oracle CM SDK rows show up in the Oracle Text view ctx_user_index_errors."

  3. Check again to see if there are any rows in ctx_user_pending and, if so, that the rows are changing.

  4. If this does not resolve the issue, contact your Oracle Support representative for further assistance.