Oracle8 ConText Cartridge Administrator's Guide Release 2.3 A58165-01 |
Library | Product | Contents | Index

# 2
Administration Concepts

This chapter describes the concepts necessary for understanding system administration for ConText.

The following concepts are discussed in this chapter:

- Administrator Responsibilities
- ConText Roles
- Predefined ConText Users
- ConText Servers
- Personality Masks
- Text Request Queue
- DML Queue
- Services Queue

## Administrator Responsibilities

Administrative responsibilities for ConText can be divided into two functional areas:

- System Administrator
- Database Administrator (DBA)

### System Administrator

The system administrator's main responsibility is managing ConText servers. This includes:

- starting and stopping ConText servers to balance the processing loads of the machines on which the servers are running
- changing the personality masks of ConText servers as needed

### Database Administrator (DBA)

The DBA's main responsibilities include:

- creating and maintaining Oracle users for ConText
- monitoring and administering the ConText queues
- monitoring the ConText data dictionary

## ConText Roles

Three database roles provided with ConText:

- CTXADMIN Role
- CTXAPP Role
- CTXUSER Role

Each ConText user should be assigned one of the ConText roles. The role assigned to a ConText users depends on the tasks performed by the user.

Note: It is not necessary to assign more than one ConText role to a user because the roles are defined hierarchically in the following descending order: CTXADMIN, CTXAPP, CTXUSER. Each higher-level role provides additional privileges in addition to the privileges of the lower-level roles.

See Also: For an example of assigning ConText roles to users, see "Managing Users" in Chapter 3, "Administering ConText". For more information about database roles, see Oracle8 Server SQL Reference

.

The ConText tasks are grouped into the following areas:

- managing ConText servers, including:
  - shut down
  - monitoring ConText server status
  - changing the personality masks for ConText servers
- administering ConText queues, including:
  - removing entries from the queues
  - changing the status of the queues
  - setting the buffer size of the Query pipes
- managing the ConText data dictionary, including:
  - creating/deleting preferences
  - creating/deleting/modifying policies
  - creating/deleting ConText indexes
  - creating/deleting/updating thesauri
- managing linguistics, including:
  - creating custom setting configurations for linguistics
  - enabling setting configurations for linguistics
  - generating linguistic output
- performing queries

### CTXADMIN Role

The CTXADMIN role provides users with the ability to perform all of the ConText tasks.

CTXADMIN should be assigned to all users who perform system and database administration for ConText.

### CTXAPP Role

The CTXAPP role users with the ability to perform the following tasks/actions:

- managing the ConText data dictionary
- managing linguistics
- performing queries

CTXAPP should be assigned to all users who develop text applications.

### CTXUSER Role

The CTXUSER role provides users with the ability to perform ConText queries (text and theme). It should be assigned to all users of a text application.

## Predefined ConText Users

ConText provides two Oracle users which are created during installation:

- CTXSYS User
- CTXDEMO User

### CTXSYS User

CTXSYS is used primarily to perform administration tasks, such as starting ConText servers and administering the ConText queues.

CTXSYS has the following functions and privileges:

- Oracle DBA with all privileges
- owner of the ConText data dictionary
- CTXADMIN role

  Note: Only the CTXSYS user can start ConText servers.

### CTXDEMO User

CTXDEMO is used primarily to run the sample applications provided with ConText.

CTXDEMO has the following functions and privileges:

- owner of the ConText database schema (tables, views, etc.) and data dictionary objects (policies, preferences) used in the sample applications
- CTXAPP role

The sample applications, provided to help application developers develop applications using ConText, include:

- EMP and DEPT tables with text indexes for performing ad-hoc text queries
- SQL scripts for setting up text in a ConText system for performing text queries
- SQL scripts for performing different types of queries
- SQL script for generating linguistic output necessary for advanced document viewing
- SQL scripts for viewing documents by themes or Gists/theme summaries
- Oracle Forms application, distributed with the ConText Workbench, for performing text queries

  See Also: For more information about setting up and using the SQL scripts, see Oracle8 ConText Cartridge Application Developer's Guide. For more information about setting up and using the Oracle Forms application, see Oracle8 ConText Cartridge Workbench User's Guide.

## ConText Servers

ConText servers are shared server processes that handle text operations in SQL requests. ConText server processes mirror their shared Oracle server counterparts, but process only text-related operations. ConText servers can be started using the ctxsrv executable or the ctxctl utility.

ConText servers can only be started from the command-line of the server machine where ConText is installed. In addition, only the CTXSYS user can start ConText servers.

If the server machine can support multiple ConText servers, multiple ConText servers can run at the same time to help balance the processing load. In addition, ConText servers can work with databases installed on either the server machine or on remote machines.

See Also: For more information about ctxsrv and ctxctl, see Chapter 4, "ConText Server Executables and Utilities".

### Text Operations

ConText servers can process five types of text operations:

- text loading
- DDL operations (creating, dropping, and updating text indexes)
- DML operations (updating the text index for a text column when inserts, updates, and deletes are performed on the table)
- Linguistics requests (generating linguistic output for documents)
- text/theme queries

The type of text operations processed by each ConText server is determined by the personality mask assigned to the server.

See Also: For more information about each of the text operations that can be processed by ConText servers, see "Text Operations" in Chapter 6, "Text Concepts". For more information about personality masks, see "Personality Masks" in this chapter.

### Text Request Queue

Requests for text operations are stored in the Text Request Queue. Available ConText servers monitor the queue for text requests. As text operations are submitted, available ConText servers with the appropriate personalities pick up the operations and process them.

See Also: For more information about the database tables and pipes that comprise the Text Request Queue, see "Text Request Queue" in this chapter.

### Server Log

ConText provides a timestamped report for each action performed by a ConText server. These reports are written to the standard output on which the server was started; however, the reports can be directed to a log file if one is specified when the ConText server is started.

See Also: For examples of starting ConText servers, see "Starting ConText Servers" in Chapter 3, "Administering ConText".

### Server Shutdown

A ConText server performs the following tasks before shutting down:

- releases all currently held resources
- cleans up and closes the server's mailboxes
- updates the server process table by removing the corresponding entry for the server from the table

## Personality Masks

The personality mask for a ConText server indicates the text operations that the server can process. When a ConText server is started, it is assigned a personality mask. A personality mask consists of one or more of the following five personalities (corresponding to the five text operations supported by ConText):

- Loader (R) Personality
- DDL (D) Personality
- DML (M) Personality
- Query (Q) Personality
- Linguistic (L) Personality

  Note: For ConText to process text and/or theme queries, at least one ConText server with the Query personality must be running.

In addition to the user-specified personalities, all ConText servers automatically take on a DBA Personality.

The DBA personality does not appear as part of the personality mask because it is automatically assigned to all ConText servers.

See Also: For more information about the text operations supported by ConText, see "Text Operations" in Chapter 6, "Text Concepts".

### Loader (R) Personality

The Loader personality enables a ConText server to scan directories at regular intervals for files to be loaded into columns in the database. When the ConText server detects a new file in a specified directory, it uses the ctxload utility to load the contents of the file into a specified column.

The directories scanned by ConText servers running with the Loader personality, as well as the scanning intervals and the columns to which the text is loaded, are specified by a ConText object, called a source, which can be attached to a column.

See Also: For more information about automated text loading, see "Text Loading" in Chapter 6, "Text Concepts". For an example of setting up ConText servers to load text, see "Using ConText Servers for Automated Text Loading" in Chapter 9, "Setting Up and Managing Text".

### DDL (D) Personality

The DDL personality enables a ConText server to process requests for creating, optimizing, and dropping text indexes. The text index on a column is what allows users to query the text stored in the column.

The DDL personality also enables a ConText server to process DML requests when DML operations are processed in batch mode.

See Also: For more information about batch DML processing, see "DML" in Chapter 6, "Text Concepts".

### DML (M) Personality

The DML personality enables a ConText server to automatically update the text index for a table when changes to the table are made that require the text index to be update. Such changes include inserting new documents, updating existing documents, and deleting existing documents.

Suggestion: When systems have a high volume of text inserts, updates, and deletes, assign the DML personality to multiple servers to better distribute the system load.

### Query (Q) Personality

The Query personality enables a ConText server to process queries for text stored either internally or externally in a text column in a database table. If no running ConText servers have the Query personality, queries submitted to ConText will fail.

Note: The Query personality is not required to use the output generated by the Linguistics. Linguistic output is stored as structured data and, as such, no ConText servers are not required to process queries for this type of information.

### Linguistic (L) Personality

The Linguistic personality allows a ConText server to process requests for the Linguistics and generate linguistic output. Linguistic output includes:

- document themes
- document Gists and/or document theme summaries

  Note: The Linguistic personality is only required to process requests for Linguistics. Once the requests have been processed, Linguistic servers can be shut down or the Linguistic personality can be removed from the personality mask of a running ConText server. For more information about the ConText Linguistics, see Oracle8 ConText Cartridge Application Developer's Guide.

### DBA Personality

The DBA personality allows a ConText server to detect and correct client/server failures and perform system cleanup (recovery).

The DBA personality is automatically assigned to each ConText server during start up, which prevents a single point of failure in a multi-server configuration.

#### ConText Server Monitoring

When a working ConText server detects a failed ConText server, it performs the following DBA actions:

- the working server cleans up the failed server's mailbox and logs the event
- if the failed server was processing a Query request, the working server releases any allocated resources
- if the failed server was processing a DDL request, the working server drops any tables that are not needed for recovery of the requested action
- if the failed server was processing a DML request, the working server places any uncompleted DML requests back in the queue
- if the failed server was processing a Query or DDL request, the working server notifies the waiting client

#### System Recovery

When a table has a text column with an existing ConText index and the table is dropped without first dropping the index and policy for the column, the index tables and policy for the column remain in the system until recovery is performed.

System recover is performed automatically by ConText servers approximately every fifteen minutes.

System cleanup can also be performed manually using the RECOVER stored procedure in the CTX_ADM PL/SQL package.

## Text Request Queue

The Text Request Queue is the logical mechanism ConText servers use to process all text operations, except for text loading.

When a client submits a request for a text operation, such as a query, the request is stored in the Text Request Queue. All available ConText servers scan the Text Request Queue regularly, retrieve pending requests if they have the appropriate personality mask, and perform the requested operations.This diagram illustrates how ConText servers with different personality masks access the Text Request Queue.

ConText servers process text requests in the Text Request Queue in the following order:

  Administration --> Query --> DDL --> DML --> Services

When a ConText server finishes processing a request of any type, it always checks the pipes and queues for the next pending request according to the precedence order.

The Text Request Queue is made up of the following database pipes and tables:

- pipes assigned to each ConText server for performing administrative (DBA) tasks
- pipes for storing Query and DDL requests
- tables for storing DML and Linguistics requests

  See Also: For more information about the tables used to process DML, see "DML Queue" in this chapter. For more information about the tables used to process Linguistic requests, see "Services Queue" in this chapter.

### Administration Pipes

Each ConText server has a dedicated administration pipe for administrative tasks that control its operation (e.g. server shutdown).

### Query Pipe

When a SQL statement or PL/SQL block includes a text query, the system dispatches the query to the query pipe.

The ConText servers notify the calling client when the operation is finished.

### DDL Pipe

ConText dispatches all requests for DDL operations (e.g. CREATE_INDEX, DROP_INDEX, and OPTIMIZE_INDEX) to the DDL pipe.

The ConText servers notify the calling client when the operation is finished.

If a ConText server encounters a problem with a request in the DDL pipe, the error does not affect the pipe or the server processing the pipe. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the pipe.

The CTX_INDEX_ERRORS view can be used to display errored DDL requests.

## DML Queue

The ConText DML Queue stores requests for index update when changes are made to a table containing a text column with an existing index.

When a DML operation is performed (i.e. data in a text table is modified, deleted, or added), a request for an index update is automatically recorded in the DML Queue. Requests are placed in the DML Queue by internal database triggers that are created automatically the first time a ConText index is generated by a ConText server for the text column.

If DML processing is running in immediate mode, the next available ConText server with the DML (M) personality picks up the requests and updates the index as soon as resources and system load allow.

If DML processing is running in batch mode, the requests are stored in the queue until a user explicitly requests DML processing. Then, available ConText servers with the DDL (D) personality pick up all the pending requests and process the requests as one or more batches.

The DML Queue consists of the following internal tables:

- Pending Table (DRQ_PENDING)
- In-Progress Table (DRQ_INPROG)
- Waiting Table (DRQ_WAITING)
- Batches Table (DRQ_BATCHES)

  Note: The DML tables are internal tables and should not be accessed directly. To view the queue, use the queue views or the GUI administration tools provided with the ConText Workbench. To administer the queue, use the CTX_DML package. For more information about DML operations, see "DML" in Chapter 6, "Text Concepts". For more information about the DML queue views, see "ConText Queue Views" in Appendix B, "ConText Views". For more information about the CTX_DML package, see "CTX_DML: ConText Index Update" in Chapter 11, "PL/SQL Packages - Text Management".

### Pending Table (DRQ_PENDING)

This table contains one row for each DML operation (request for index update) that has not yet been picked up by a ConText server. The row indicates the specific cell that has changed in the text table.

Note: Requests are visible in the pending table only after the insert, update, or delete has been committed.

When a request has been picked up by a ConText server with the DML or DDL personality, the row is deleted from the pending table and the server begins the index update. In addition, a row is written to the in-progress table to indicate that the request is being processed.

### In-Progress Table (DRQ_INPROG)

This table contains one row for each DML request being processed by a ConText server.

Once the ConText server finishes processing the request, the row is deleted from the in-progress table, indicating that the appropriate index has been updated to reflect the document modification that generated the request.

### Waiting Table (DRQ_WAITING)

This table contains a row for each DML request for which a duplicate request exists in the pending table. This condition occurs when a DML request for a row has not been picked up by a ConText server and additional requests for the row are issued. This table ensures that DML requests for a row in a text table are processed in order.

### Batches Table (DRQ_BATCHES)

This table contains one row for each batch of DML requests. A batch consist s of up to 10,000 DML requests for an indexed column. If the queue contains more than 10,000 DML requests for a column or requests for different columns, ConText uses multiple batches to process the requests.

For each batch, the table stores information such as the number of rows in the batch, the batch ID, and the ID of the server processing the batch.

If immediate DML is enabled, batches are created automatically as ConText servers with the DML personality pick up requests from the queue. If batch DML is enabled, batches are created by users calling CTX_DML.SYNC.

See Also: For more information about immediate and batch DML, see "DML" in Chapter 6, "Text Concepts"

### Timestamps

Requests in the DML Queue are processed in the order they are received, based on a time-stamp. Rows are inserted into the pending table without a timestamp. At a specified time interval, all unmarked rows within the pending table are marked with a time-stamp. The timestamp is based on the time each change was committed.

### Error Handling

If a ConText server encounters a problem with a request in the DML Queue, the error does not affect the queue or the server processing the queue. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the queue.

The CTX_INDEX_ERRORS view of the Services Queue can be used to display errored DML requests.

### Queue Management

The DML Queue can be enabled/disabled manually to control processing of DML requests. When the DML Queue is disabled, the queue continues to accept requests; however, new requests and any pending requests in the disabled DML Queue are not picked up by ConText servers until the queue is enabled manually.

## Services Queue

The Services Queue is used for processing requests for all ConText services. The Services Queue is designed to be extensible. As additional services are provided by ConText, the Services Queue is the mechanism by which the services will be managed. Currently, the Services Queue supports the following services:

- requests for the ConText Linguistics (theme, Gist, and theme summary generation)
- error handling for index creation and optimization (DDL) and index updating (DML)

When a request is submitted for the Linguistics, the request is stored in the Services Queue. A request is picked up by the first available ConText server with the Linguistic personality and the server generates linguistic output for the specified request.

ConText servers with the Linguistic personality pick requests out of the queue based on the request priority and creation time-stamp. Clients may queue a request and continue to work while the request is being processed.

The Services Queue is asynchronous. Clients that place a request in the queue do not automatically block their subsequent requests while waiting for a reply. However, clients can choose to block their subsequent requests for a specified time when they submit requests.

### Services Queue Table (CTX_SVCQ)

The Services Queue consists of the CTX_SVCQ internal table. This table stores a row for each request for the ConText Linguistics, as well as the request status.

Note: CTX_SVQ is an internal table and should not be accessed directly. To view the queue, use the queue views or the GUI administration tools provided with the ConText Workbench. To administer the Services Queue (e.g. cleaning up entries), use the CTX_SVC package or the GUI administration tools. For more information about the CTX_SVC package, see "CTX_DDL: Text Setup and Management" in Chapter 11, "PL/SQL Packages - Text Management".

### Error Handling

If a ConText server encounters a problem with a request in the Services Queue, the error does not affect the queue or the server processing the queue. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the queue.

The CTX_LING_ERRORS view of the Services Queue can be used to display errored requests for Linguistics.

### Queue Management

The Services Queue can be enabled/disabled manually to control processing of Linguistics requests. When the Services Queue is disabled, requests are still accepted into the queue; however, new requests and any pending requests in the disabled Services Queue are not picked up by ConText servers until the queue is enabled manually.

Prev | Next
| Copyright © 1997 Oracle Corporation. All Rights Reserved. |
Library | Product | Contents | Index