A Planning Your Oracle CM SDK Deployment

Using Oracle Application Server, Infrastructure and Oracle Internet Directory

Oracle CM SDK does not require Oracle Application Server, Infrastructure unless you want to use Oracle Internet Directory for credential management. If you do want to use Oracle Internet Directory with Oracle CM SDK, follow these guidelines:

• Install and configure Oracle Internet Directory on a separate database instance on a separate computer.

  • The only Oracle Internet Directory releases certified for use with Oracle CM SDK 9.0.4 are the releases shipped with Oracle Application Server, Infrastructure (release 9.0.2 and release 9.0.4).

• To configure the OidCredentialManager during Oracle CM SDK configuration, you must know the orcladmin password, the computer name, the port number (389 is the default LDAP port number), and the root Oracle context of the Oracle Internet Directory.

• Decide in advance how you want to map the default Oracle CM SDK system, guest, and scott user accounts to Oracle Internet Directory. During the configuration process for the OidCredentialManager, you can either create new accounts for these users (if Oracle Internet Directory does not already contain accounts with these names), or you can map these accounts to an Oracle Internet Directory account of your choice. (See Chapter 3, "Installation and Configuration" for additional information.)

See the Oracle Application Server 10g Installation Guide and Oracle Internet Directory Administrator's Guide for additional recommendations and requirements.

Multiple-Computer Deployment

Oracle CM SDK is designed to run as middle-tier application server software supported by Oracle Application Server. For optimal performance and ease of administration, the tiers should be located on different physical computers: the database should run on one computer, and the Oracle Application Server and Oracle CM SDK software should run on another computer.

To use Oracle Internet Directory for managing Oracle CM SDK user credentials, you should also install and configure Oracle Application Server, Infrastructure on a third computer that meets the requirements for running Oracle Internet Directory and Oracle Application Server, Infrastructure. See the Oracle Application Server 10g Administrator's Guide for more information.

The following provides an overview of the deployment process:

Single-Computer Deployment

Oracle CM SDK can be installed on a single computer if the computer meets the recommended hardware and software requirements. If your computer does not meet the recommended requirements, performance in this configuration can be less than satisfactory.

The hardware requirements for single-computer deployment can support only two Oracle CM SDK users accessing two protocols concurrently. If you plan to use Oracle Internet Directory for credential management, the computer requires three Oracle home instances. Because of this, Oracle recommends that you use single-computer deployment for development or evaluation purposes only.

1. Install and configure the Oracle 10g Database Server (release 9.0.1.4, 9.2.0.3, or later) or Oracle10g Database Server into one Oracle home.

2. Install and configure Oracle Application Server, A. J2EE and Web Cache into a new, separate Oracle home, accepting all of the default values.

3. Install Oracle CM SDK in the same Oracle home that contains Oracle Application Server.

4. If you are upgrading, copy all custom classes from a previous Oracle 9iFS or Oracle CM SDK installation to the following directory:

   %ORACLE_HOME%\ifs\cmsdk\custom_classes
   

   
   

   Note: If you have a custom class named LINK, you must rename the class before installing Oracle CM SDK.

   
   

5. Using Oracle CM SDK Configuration Assistant, configure Oracle CM SDK following the instructions in Chapter 3, "Installation and Configuration".

Sizing Formulas for Each Middle-Tier Computer

This section provides formulas that you can use to determine specific hardware sizing for each middle-tier computer.

• Number of CPUs

• Required Usable Disk Space

• Total Computer Memory, HTTP as the Primary Protocol

• Total Computer Memory, Non-HTTP Protocol

The following table summarizes the sizing formulas:

roundup(peak concurrent connected users / 250 + 33% headroom)

480MB + (3.6MB * peak concurrent connected users)

480MB + (1MB * peak concurrent connected users * average number of sessions in use by each concurrent connected user) + (3KB * number of objects desired in the Java object cache) + (8MB * number of connections to the database)

Sizing Formulas for the Database Computer

This section provides formula that you can use to determine specific hardware sizing for each database computer.

• Number of CPUs

• Required Usable Disk Space

• Total Computer Memory

The following table summarizes the sizing formulas:

roundup(peak concurrent connected users / 250 + 33% headroom)

4.5GB + total raw file size + (total raw file size * 20%)

64MB + 128MB + database buffer cache + (1MB * number of connections to the database) + (500 bytes * number of documents) + (100KB * peak concurrent connected users)

Memory Requirements: Sample Deployment

Approximate minimum memory overhead on the middle-tier computers for each component are detailed in Table A-7:

Data Types and Storage Requirements

Table A-8 lists the different types of data stored in Oracle CM SDK and describes the purpose of each tablespace. For about creating custom tablespaces, see Appendix B, "Creating an Oracle Database".

Typical tablespace storage space and disk I/O are detailed in Table A-9:

Note the following issues regarding the information in Table A-9:

• I/O rates are highly dependent on the size of the db_block_cache. These measurements were taken on the Oracle-internal Oracle Files implementation, with 8GB db_block_cache, 17 million documents, and 40,000 named users.

• The IFS_MAIN tablespace is the most important tablespace to spread across disks for maximum I/O capacity.

• Disk I/O for the IFS_CTX_I, IFS_CTX_X and IFS_CTX_K tablespaces is largely generated from Oracle Text batch processes (ctx_ddl.sync_index, and ctx_ddl.optimize_index), which are not critical to end-user performance. Therefore, these tablespaces can be on disks with lower I/O capacity, if necessary.

Storing Documents in an Oracle Database

The largest consumption of disk space occurs on the disks that actually contain the documents that reside within Oracle CM SDK, namely the Indexed Medias tablespaces, Non-Indexed Medias tablespaces, and interMedia tablespaces. This section explains how the documents are stored and how to calculate the amount of space those documents require.

As previously mentioned, documents stored in Oracle CM SDK are actually stored in database tablespaces. Oracle CM SDK makes use of the Large Object (LOB) facility of the Oracle database. All documents are stored as Binary Large Objects (BLOBs), which is one type of LOB provided by the database. LOBs provide for transactional semantics much like the normal data stored in a database. In order to accomplish these semantics, LOBs must be broken down into smaller pieces which are individually modifiable and recoverable. These smaller pieces are referred to as chunks. Chunks are a group of one or more sequential database blocks from a tablespace that contains a LOB column.

Both database blocks and chunk information within those blocks (BlockOverhead) impose some amount of overhead for the stored data. BlockOverhead is presently 60 bytes per block, which consists of the block header, the LOB header, and the block checksum. Oracle CM SDK configures its LOBs to have a 32K chunk size.

As an example, assume that the DB_BLOCK_SIZE parameter of the database is set to 8192 (8K). A chunk would require four contiguous blocks and impose an overhead of 240 bytes. The usable space within a chunk would be 32768-240=32528 bytes.

Each document stored in Oracle CM SDK consists of some integral number of chunks. Using the previous example, for instance, a 500K document actually uses 512000/32528=15.74=16 chunks. Sixteen chunks take up 16*32K = 524288 bytes. The chunking overhead for storing this document would then be 524288-512000=12288 bytes which is 2.4% of the original document's size.

The chunk size used by Oracle CM SDK is set to optimize access times for documents. Note that small documents, documents less than one chunk, incur a greater disk space percentage overhead since they must use at least a single chunk.

Another structure required for transactional semantics on LOBs is the LOB Index. Each LOB index entry can point to 8 chunks of a specific LOB object (NumLobPerIndexEntry = 8). In this example, where a 500K document takes up 16 chunks, two index entries are required for that object. Each entry takes 46 bytes (LobIndexEntryOverhead) and is then stored in an Oracle B*tree index, which in turn has its own overhead depending upon how fragmented that index becomes.

The last factor affecting LOB space utilization is the PCTVERSION parameter used when creating the LOB column. For information about how PCTVERSION works, please consult the Oracle9i SQL Reference.

Oracle CM SDK uses the default PCTVERSION of 10% for the LOB columns it creates. This reduces the possibility of "ORA-22924 snapshot too old" errors occurring in read consistent views. So by default, a minimum of a 10 percent increase in chunking space must be added in to the expected disk usage to allow for persistent PCTVERSION chunks.

For large systems where disk space is an issue, Oracle recommends reducing PCTVERSION to 1, in order to reduce disk storage requirements. This can be done at any time in a running system using the following SQL commands:

alter table odmm_contentstore modify lob (globalindexedblob) (pctversion 1);
alter table odmm_contentstore modify lob (emailindexedblob) (pctversion 1);
alter table odmm_contentstore modify lob (emailindexedblob_t) (pctversion 1);
alter table odmm_contentstore modify lob (intermediablob) (pctversion 1);
alter table odmm_contentstore modify lob (intermediablob_t) (pctversion 1);
alter table odmm_nonindexedstore modify lob (nonindexedblob2) (pctversion 1);

To calculate LOB tablespace usage:

1. Calculate the number of chunks a file uses by figuring the number of blocks per chunk, then subtracting the BlockOverhead (60 bytes) from the chunk size to obtain the available space per chunk.

2. Divide the file size by the available space per chunk to obtain the number of chunks, per the following formula:

   chunks = roundup(FileSize/(ChunkSize-((ChunkSize/BlockSize)*BlockOverhead)))
   
   

   For example, if FileSize = 100,000, ChunkSize = 32768, Blocksize = 8192, and BlockOverhead = 60, then:

   chunks = roundup (100000 /(32768 - ((32768 / 8192) * 60)))= 4 Chunks
   
   

3. Calculate the amount of disk space for a file by multiplying the number of chunks times the chunk size, multiplying that result by the PCTVERSION factor, and then adding the space for NumLobPerIndexEntry (8) and LobIndexEntryOverhead (46 bytes). Use the following formula:

   FileDiskSpaceInBytes = roundup(chunks*ChunkSize*PctversionFactor) + roundup(chunks/NumLobPerIndexEntry*LobIndexEntryOverhead)
   
   

   For example, if chunks = 4, ChunkSize = 32768, PctversionFactor = 1.1, NumLobPerIndexEntry = 8, and LobIndexEntryOverhead = 46, then:

   FileDiskSpaceInBytes = roundup (4 * 32768 * 1.1) + (roundup(4/8) * 46) = 144226 FileDiskSpaceInBytes
   
   

4. Calculate the total disk space used for file storage by summing the application of the above formulas for each file to be stored in the LOB, per this formula:

   TableSpaceUsage = sum(FileDiskSpaceInBytes) for all files stored 
   
   

Oracle CM SDK creates multiple LOB columns. The space calculation must be made for each tablespace based upon the amount of content that qualifies for storage in that tablespace.

Oracle CM SDK Metadata and Infrastructure

The Oracle CM SDK server keeps persistent information about the file system and the contents of that file system in database tables. These tables and their associated structures are stored in the Oracle CM SDK Primary tablespace. This tablespace contains approximately 300 tables and 500 indexes. These structures are required to support both the file system and the various protocols and user interfaces that make use of that file system.

The administration and planning tasks of this space should be very similar to operations on a normal Oracle database installation. The administrator of the system should plan for approximately 6K of overhead per document to be used from this tablespace, or about 2% of the overall content. If there is a significant amount of custom metadata, such as categories, this overhead is larger.

The initial disk space allocated for this tablespace is approximately 50MB for a default install. Of this 50MB, 16MB is actually used at the completion of installation. This includes instantiations for all required tables and indexes and the metadata required for the approximately 700 files that are loaded into Oracle CM SDK as part of the install. Different tables and indexes within this tablespace grow at different rates depending on which features of Oracle CM SDK are used in a particular installation.

Oracle Text

When Oracle CM SDK works in conjunction with Oracle Text, it allows users to use powerful search capabilities on the documents stored within Oracle CM SDK. Disk space for these capabilities is divided among three distinct tablespaces for optimal performance.

The Oracle Text Data tablespace contains tables which hold the text tokens (separate words) that exist within the various indexed documents. The storage for these text tokens is roughly proportional to the ASCII content of the document.

The ASCII content percentage varies depending on the format of the original document. Text files only have white space as their non-ASCII content and therefore incur a greater per document percentage overhead. Document types such as Microsoft Word or PowerPoint contain large amounts of data required for formatting that does not qualify as text tokens. The per document percentage on these types of documents is therefore lower. On a system with diverse content types the expected overhead is approximately 8% of the sum of the original sizes of the indexed documents.

Table A-10 offers some general guidelines for the amount of ASCII text in a document for several popular file formats:

The Oracle Text Keymap tablespace contains the tables and indexes required to translate from the Oracle CM SDK locator of a document (the Oracle CM SDK DocID) to the Oracle Text locator of that same document (the Oracle Text DocID). The expected space utilization for this tablespace is approximately 70 bytes per indexed document.

The Oracle Text Index tablespace contains the B*tree database index that is used against the text token information stored in the Oracle Text Data tablespace. This grows as a function of the ASCII content just as the Oracle Text Data tablespace does. On a system with diverse content types the expected overhead is approximately 4% of the sum of the ASCII content of the documents, or approximately 1% of the sum of the total sizes of the indexed documents.

Disk Space Requirements: Sample Deployment

This section details disk space requirements, and offers guidance as to how necessary disk space expands with the addition of documents to the server.

Based on experience running Oracle CM SDK for Oracle Corporation's internal usage, the disk overhead of Oracle CM SDK for a large system (hundreds of gigabytes of file content) is detailed in Table A-11:

See the Oracle Concepts Guide for explanation of the terms Large Object (LOB), tablespace, chunk size, and extents.

Given that a large percentage of the overhead is in LOB overhead, note that the overhead for your Oracle CM SDK instance can vary depending on the average and median sizes of documents.