E
Working with Oracle Text

This appendix provides detailed information on how Oracle Internet File System (Oracle 9iFS) leverages the indexing done by Oracle Text (formerly called interMedia Text). This section is meant to explain strategies for maintaining the Oracle Text index used by Oracle 9iFS. It also provides experienced DBAs with the information necessary to enable developers to access advanced Oracle Text features using the Oracle 9iFS Java API. This is not meant as a complete reference, but it should save you some time while developing Oracle 9iFS applications. Topics include:

The Oracle Text Index

Immediately following the Oracle Internet File System (Oracle 9iFS) installation process, the Configuration Assistant runs and populates the database schema used by Oracle 9iFS. Oracle 9iFS creates an Oracle Text index named IFS_TEXT.

SQL Scripts for Creating IFS_TEXT Index

The Oracle 9iFS Configuration Assistant uses the following SQL scripts to create the IFS_TEXT index:

CreateContextFunnelProcedure.sql: Creates the procedure used by USER DATASTORE.
GrantContextToIFS.sql: Grants access to some Oracle Text-specific commands to the Oracle 9iFS user.
CreateContextPreferences.sql: Tablespace and other text preferences are created by the Oracle 9iFS user.
CreateContextIndex.sql: Based on these text preferences, IFS_TEXT index is created.

Maintaining the IFS_TEXT Index

Oracle Text relies on the quality of its index for fast and accurate searches. Anyone seeking to improve performance can start with index issues first. If no further improvements can be made to your indexing scheme, the next step is to reevaluate your queries.

When content changes, the index may need to be updated to reflect these changes. Recreating the index from scratch is expensive, and document repositories tend to grow rather than shrink. As it happens, there are ways to balance the need for a fresh index with the need to minimize processing expense during peak periods. Please refer to the Oracle Text documentation for detailed information.

The Oracle Text index is not like a regular index in that updating does not happen within the insert/update transaction. A separate "index refresh" must take place. The refresh is accomplished with the ctx_ddl.sync_index procedure. It can be easily set up as a dbms_job to run at any frequency. This procedure does not rebuild the entire index. It only adds the records that have been inserted or changed (or deleted) since the last sync. Therefore the more frequently it runs, the faster it goes.

The downside to running this frequently is that the index becomes fragmented with eventual performance implications. This can be rectified by occasionally optimizing the index.

Index Maintenance

Indexes for Oracle Text can be created using a number of SQL and PL/SQL procedures. Before the Oracle9i database, Oracle Text was an optional feature, and required a process called CTXSRV to be running for any indexes to be updated. This process is still used, but it is deprecated in the Oracle9i database.

Using the PL/SQL Package

The PL/SQL interface for index maintenance is preferable for a production system:

Sync Existing IFS_TEXT Index

ctx_ddl.sync_index(indexname);

Optimize IFS_TEXT Index

ctx_ddl.optimize_index(indexname, 'FAST'|'FULL', maxtime );

These two commands can run simultaneously. You can run these commands using the Oracle Enterprise Manager Java Console job system. Another option is the DBMS_JOB package, which is described in the Oracle9i Database Administrator's Guide.

Using the Oracle Enterprise Manager Java Console Job System to Schedule Oracle Text Jobs

To use the Oracle Enterprise Manager Java Console job system, you need to make sure it has been configured correctly. Refer to "Enabling the Job System" in the Oracle Enterprise Manager Configuration Guide.

To display the Create Job property sheet, select Create Job from the Job menu.

Text description of the illustration ejob1.gif
On the General tab, enter a job name and select the target database.
On the Tasks tab, select the task: Run SQL*Plus Script.

Text description of the illustration ejob2.gif
On the Parameters tab in the Script Text box, enter the SQL syntax or SQL script for the job. Make sure you use the preferred credentials for your IFSSYS schema.

Text description of the illustration ejob3.gif
On the Schedule tab, enter how frequently you want the job to run and its execution start date and time.

Text description of the illustration ejob4.gif
Select the Submit and Add to Library option and click Save to save the job to the job library and to send the job to the Intelligent Agents at the selected destinations. The job appears in the Active page and the job library. You can modify or submit a saved job at a later time.

Text description of the illustration ejoblibl.gif

For more information about creating and monitoring jobs refer to the Oracle9i Enterprise Manager Administrator's Guide.

Using Oracle 9iFS Manager to Designate Indexing

To illustrate the way Oracle 9iFS works with Oracle Text, consider this example: Imagine you want to index the content of some .java source files. Before placing them into the repository, you notice that, by default, Oracle 9iFS does not consider these files candidates for indexing.

If your goal is to provide content searching on these files, you need them to be indexed by Oracle Text. The most straightforward way would be as follows:

Use the Oracle 9iFS Manager tool to add the .java mimetype and designate it as To Be Indexed, using the Create Format dialog.

Text description of the illustration ecreatf.gif
Insert the files into the repository.
Sync the index.

Monitoring Oracle Text Indexing of Oracle 9iFS Documents

You can view which documents generated errors when they were indexed by Oracle Text by examining $ORACLE_ HOME/9ifs/admin/sql/ViewContextErrors.sql.

If you want to monitor each document as it is indexed by Oracle Text, you can use the procedure ctx_output.add_event() prior to synchronizing the Oracle Text index. See the example, $ORACLE_ HOME/9ifs/admin/sql/SyncContextIndex.sql. For an example of viewing more information about the documents that have just been indexed by Oracle Text, see $ORACLE_ HOME/9ifs/admin/sql/ViewDocumentByRowID.sql.

Additional Features of Oracle Text

The following additional features of Oracle Text provide users viewing options, additional search operators and functions, and a variety of other queries.

Document Services

Oracle Text includes a set of features referred to as Document Services, which grant users the ability to view results in unformatted text, the original document format, or in HTML.

XML Indexing

Metadata formatted in XML is easily indexed and queried using the Oracle Text syntax. Special components called Sectioners are used to limit the scope of a query according to XML tags, with the Autosectioner indiscriminately indexing every XML tag it encounters, and the XML Sectioner requiring more input in return for greater power and flexibility.

For more information about defining sections in XML and news group formatted documents, consult the Oracle9i Text Reference.

Theme Indexing

Oracle Text supports the ability to generate themes based on the file's content. By default, Oracle 9iFS has turned off theme indexing since it takes longer to index the content. To use themes, custom application development is also required because the Oracle 9iFS clients do not expose themes in the user interfaces. See "Document" in the Javadoc for more information regarding custom application development using themes.

Turning On Theme Indexing

In SQL*PLUS, connect as the Oracle 9iFS schema user, and enter:

exec ctx_ddl.set_attribute('ifs_default_lexer', 'index_themes', 'yes'); exec ctx_ddl.set_attribute('ifs_default_lexer', 'theme_language','english'); alter index ifs_text rebuild parameters('replace lexer ifs_global_lexer');

Warning:

This operation may take a long time, since it must re-index all the document content, including themes.

Turning Off Theme Indexing