Oracle Enterprise Manager Administrator's Guide Release 2.2 Part Number A85248-01 |
Library | Service | Contents | Index

# 5
Events

The Event system allows you to monitor your network for specific conditions, such as loss of service or lack of storage, that may occur in your managed environment. You select tests to run on managed targets (databases, nodes, listeners, or other services), then set the threshold parameters for which you want to be notified. You can share events with other administrators, in addition to being able to notify specific administrators when an event condition occurs. For some event tests, you can also choose to execute a fixit job that automatically corrects the problem.

The following topics are discussed in this chapter:

- Event System Overview

- Event Categories and Types

- Event Pane

- Event Menu

- Creating, Modifying, and Registering an Event

- Oracle Event Tests

- Event System Features and Requirements

## What's New in Events

This section discusses the enhancements to the Events System in Enterprise Manager 2.2.

- New operating system event tests

  These event tests monitor operating system-specific metrics for Solaris, HP-UX, Compaq Tru64, IBM AIX and Windows NT. These new events require an 8.1.7 or higher Intelligent Agent. As with other Advanced Events, these events are available with the Oracle Diagnostics Pack.

- Support for duplicate events

  Duplicate events can now be registered against 8.1.7 or higher Agents. Duplicate events are those that contain the same event tests and parameters against the same target. For example, 2 or more administrators can create their own events that register the same Tablespace Full event with the same parameters against the same tablespace.

  You cannot, however, have duplicate event tests within the same Event. For example, if you create an Event called "Database Availability", it cannot have 2 or more of the same Database Up/Down event test against the same database A.

  Refer to the Enterprise Manager release notes for details on duplicate events with earlier versions of Enterprise Manager and the agents.

- Improved Node Up/Down Monitoring

  The Node Up/Down event test is enhanced to distinguish between situations where the target node is unreachable, the agent is down, or the agent is not responding.

- Enhanced Monitoring for Target Subcomponents

  For Events that monitor targets with multiple subcomponents, (e.g. a group of tablespaces), notifications will now be sent each time the status of any subcomponent changes, even if it does not change the overall severity of the Event.

  For example, you register a Tablespace Full event for tablespaces A, B, C. If tablespace A becomes full, the event triggers an Alert. If tablespace B also becomes full, then even though the event is still in Alert due to tablespace A, another notification will be sent due to tablespace B becoming full.

- Error State for Events

  When there are problems evaluating an event (e.g. an archive full event against a database that is NOT running in archivelog mode), then an alert may be sent indicating the event is in error. This error state is denoted by a yellow hexagon. In most cases, events in error are due to user errors and can thus be corrected.

  Some examples of error situations are:

  - registering an Archive Full event against a database running in noarchivelog mode

  - registering an event that monitors segments but specifying a filter that excludes all available segments

  - registering an event that needs to connect to the database but providing the wrong database credential

  Refer to "Interpreting Events" on page 5-8 for more information.

- Enhanced Datablock Corruption event

  When registered against 8.1.7 or higher Agents, the Datablock Corruption event test will now also monitor for ORA-1157 (cannot identify/lock data file string - see DBWR trace file) and ORA-27048 (skgfifi: file header information is invalid) errors.

- Enhanced User-Defined SQL Test

  The User-Defined SQL event test can support pl/sql functions. This means you can select from a pl/sql function and have the return value evaluated against a specified threshold. The actual pl/sql function needs to be defined separately.

- Enhancements to the Event Handler

  To simplify setup and configuration, the Event Handler is now solely available as an external service of the Management Server. Error handling has also been improved.

- Context-Sensitive Help for Event Tests

  When creating/editing events, this feature will allow users to select an individual event test and press the F1 key or click the Help button to access online help for that particular event.

## Event System Overview

The Event system allows you to efficiently monitor a large system. Using the Event system and Intelligent Agents, you can effectively monitor any number of databases, nodes, or other services 24 hours a day, and be alerted when a problem or specific condition is detected. You can also pinpoint only the services you wish to monitor. The Event system can be extended to include other third-party applications that detect events independent of the Intelligent Agents.

In the Event system, event settings are stored based on the administrator registering the event. This allows administrators of large systems to customize their event systems to their preferences and tasks. Administrators receive messages for events for which they have been selected to receive notifications by other administrators.

The Event system includes the following processes:

1. Creating an event by completing the Event property sheet pages. This involves:

   1. Determining the destination and frequency of the event.

   2. Selecting the event tests that you want to run.

   3. Determining the parameters for the event tests.

   4. Specifying a fixit job to be run when an event triggers.

   5. Assigning permissions to allow other administrators to share the event.

2. Saving and modifying an event.

3. Registering, or submitting, an event to a destination.

4. Interpreting and correcting an event occurrence.

   1. Logging information pertinent to your interpretation of the event to the Event log.

   2. Assigning the Event to a different administrator if appropriate.

### Using Events

You need to create and register events, which are simply a group of event tests that you want to run on your managed systems. Oracle Enterprise Manager includes a variety of predefined event tests that you can use when creating events. The event tests are grouped by service type, for instance:

- Database

- Listener

- Web Servers

- Concurrent Manager

- Node

#### Creating Events

You can create events using the predefined event tests that have been installed with Oracle Enterprise Manager. See "Event Categories and Types" on page 5-10 for more information.

The events are created with information entered in the Event property sheet. You determine parameters such as the destination that is monitored, the specific tests to perform, the frequency that the event test is executed, and whether other administrators can share the events. See "Permissions" on page 1-23 for more information. Some event tests have parameters with threshold values that you can customize for your system. See "Event Parameters Page" on page 5-34 for more information. To use the Event system, an administrator must have sufficient privileges to access database objects from the Console. Under most circumstances, full DBA privileges are not required, nor would be appropriate to assign full DBA privileges to every administrator. For this reason, the Event Handler role was created.

##### Enterprise Manager Monitor Role

Beginning with Oracle 8.0.6 databases and higher, the OEM_MONITOR role is created by the Oracle database creation scripts. This role permits access to database functionality within Enterprise Manager. For example, running events against a database (tablespace full, buffer cache hit ratio) or browsing through the objects in a database via the Console Navigator tree. These types of functionality require database credentials on which to perform these operations. Rather than granting the powerful DBA role to the database credentials, many administrators prefer to provide only the necessary privileges required to do these operations. Granting the OEM_MONITOR role to the database credentials, ensures that the user has the minimum sufficient privileges required for these operations.

For database users on 7.3.x databases, you need to define the OEM_MONITOR role manually.

Note: You need to create the OEM_MONITOR role using the SYS account.

Here are the steps you need to perform:

1. Create a role called OEM_MONITOR

       drop role OEM_MONITOR;
       create role OEM_MONTOR:

2. Grant the 'connect' role to OEM_MONITOR

       grant connect to OEM_MONITOR;

3. Grant the system privileges 'Analyze any' & 'Create table' to OEM_MONITOR

       grant analyze any to OEM_MONITOR;
       grant create table to OEM_MONITOR;

4. Create the SELECT_CATALOG_ROLE role as defined in sc_role.sql .

5. Grant the SELECT_CATALOG_ROLE to the OEM_MONITOR role

       grant select_catalog_role to OEM_MONITOR;

You are now ready to grant the OEM_MONITOR role to the database user that will be used as "database preferred credentials" in Enterprise Manager. In addition to granting the OEM_MONITOR role to a user, you must also ensure that the QUOTA for the user account is set to UNLIMITED.

The "Continued Row" event test needs to analyze results into a table so it needs both the "analyze any" and "create table" privileges.

Note: The "analyze any" privilege is used by the "index rebuild" event to compute statistics.

#### Registering Events

Events are registered, or submitted, to specific destinations, such as nodes, listeners, or databases. The status of a registered event is viewed in the Registered page of the Event pane.

The event scripts are executed on nodes with the permissions of the Intelligent Agent. However, some of the database event tests, such as Continued Rows, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Preferred Credentials" on page 1-25 for more information.

The Intelligent Agent is responsible for detecting when a specific event condition has occurred. The Intelligent Agent first notifies a Management Server, which in turn notifies interested administrators either through the Oracle Enterprise Manger Console, or by external means such as Email or Paging.

The Management Server is responsible for registering event information with the appropriate Intelligent Agents on nodes in the network. You determine the frequency that an Intelligent Agent checks an event. See "Frequency" on page 5-32 for details on setting the frequency interval for an event. An exception to this is the Up/Down (node) event test, which is checked at an interval set by the system itself. See "Fault Management Event Tests" on page 5-11 for more information on this event test.

### Event Occurrences

When an alert condition occurs, the Intelligent Agent is responsible for notifying the Management Server. Each event is logged in the repository and can be viewed and acknowledged in the Alerts page of the Console. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.

#### Event Notifications

Events can consist of multiple event tests. If any one of these tests identify a specified condition, the event is triggered and a notification is sent to the Console. If enhanced notification is configured for your system, paging and/or email notifications are sent.

Event notification occurs as follows:

- A notification is sent when the threshold of an event test exceeds the level specified by parameter values. If the event does not have parameters, a notification is sent when the event occurs.

- If the event test condition remains above the threshold specified, a new notification is not sent. If the condition does not exist when the next test is run, the event clears. Notifications are also sent (email/paging) when an event clears.

- If an event test condition changes from warning to critical or critical to warning, a new notification is sent to the Event pane or via E-mail or paging.

- If you acknowledge and move an alert to history, a new notification is not sent to the Alerts page unless a moved warning changes to a critical alert.

#### Notifying Administrators

Enterprise Manager administrators can be notified in various ways, such as electronic mail or paging, depending on the administrator's setups and permissions. You need to set up the notification services and determine the administrators that need to be notified for the events. See "Event Permissions Page" on page 5-35 and "Permissions" on page 1-23 to determine the administrators that receive notifications. See "Notification" on page 1-12 and "Schedule" on page 1-22 to determine how and when an administrator is notified.

If you plan to notify administrators with email or paging, you need to make sure the following are set up properly:

- The settings for the system modem.

- The notification schedules for the administrators.

- The mail and paging services that are used to contact administrators.

- The mail address and paging numbers for each administrator.

#### Interpreting Events

An event is composed of one or more event tests. While an individual event test may result in a different status (For example, some clear, some are in alert), there is a general status for the Event. To determine the general severity for the event, the following rules apply in succession:

1. If the event includes an UpDown event test, and this test triggers, then the general status of the Event is "Unknown" (gray flag)

2. Otherwise, if the event includes a test that reaches an alert state, then the general status of the Event is "Critical" (red flag)

3. Otherwise, if the event includes a test that reaches a warning state, then the general status of the Event is "Warning" (yellow flag)

4. Otherwise, if the event includes a test that is in error, then the general status of the Event is "Error" (yellow hexagon)

5. Otherwise, all tests should be clear, so the general status of the event is "Clear"

You can still see the individual status of each event test in the Event Viewer.

##### Event Colors and Icons

All events return values and some events produce output messages. The events return different icons depending on the severity of the event. These severity levels are determined by parameter thresholds you set for the event tests during event creation. The colors are displayed on the event severity icon that is located:

- Next to the event name listed in the Alerts page.

- On the object in the Group pane if it is part of a group. See Chapter 3, "Groups" for information on the Group pane.

The colors of the event severity icons are:

- Error State (yellow hexagon with an exclamation point)

  An error state indicates there is a problem with the evaluation of the event condition, as opposed to a threshold being met. Examples of error states are: registering an Archive Full event against a database in non-archivelog mode, registering an event that monitors segments but specifying a filter that excludes all available segments.

- Event cleared (green flag)

- Warning (yellow flag)

- Critical (red flag)

- Unknown (gray flag)

  A gray flag represents an "unknown" state where it is not possible for Enterprise Manager to ascertain the event status because the node is unreachable or the Intelligent Agent is not working. The gray flag will appear on the group pane and as the flag for the event in the Alert tab if your event includes at least one up/down event test (any target: node/database/listener). When the gray flag occurs, it will be set for the Event. When you see the event in the Event viewer, the flag for the up/down event test will be gray, and the flags for the other event tests will remain the color of their original state.

  If you have a single db up/down event test in an event, it will also turn gray (at the Event level). When you look at the Event in the Event Viewer, you will see two event tests: db up/down and node up/down (which is always added whenever you have any up/down event test). Both these event tests will have gray flags.

  If you have an event that does not include any up/down event test, then it will remain the same color even though other events are showing gray flags.

  If an UpDown event test is specified for a particular event, and the node to which the event is assigned goes down, the event will return an "unknown" severity (gray flag). The original event severity will be restored when the node comes back up. At least one UpDown event test (node, listener, database, etc.) must be specified for an event in order for the "unknown" severity to be returned. Events not containing an UpDown event test will not provide any node down notification.

##### Note:

Some events, such as Probe and User Blocks events, do not return a warning value because the warning threshold parameter is not used. The event has either occurred or not occurred.

### Correcting Problems

When an event occurs, you need to correct the problem. In some cases, you can create a fixit job that responds to specific event conditions. These situations are noted in the online help for Oracle events.

In other cases, the solution requires careful attention by a system administrator. For example, space management event conditions may require an administrator to increase space requirements and resource management conditions may require an administrator to adjust initialization parameters.

For information on how to correct event conditions, refer to the appropriate documentation. To correct Oracle database problems, refer to the Oracle Server Administrator's, Tuning, and Reference Guides. For network problems, refer to the Oracle networking guides for your system.

## Event Categories and Types

The Oracle event tests for the database, listener, and node destination types are grouped into categories:

- Fault Management event tests

- Space Management event tests

- Resource Management event tests

- Performance Management event tests

- Unsolicited events

Only the UpDown event tests are included with Oracle Enterprise Manager. These fall under the category of `Fault Management event tests'. Additional advanced events for all categories are available with the optional Oracle Diagnostics Pack. Beginning with the Oracle Diagnostics Pack for Enterprise Manager version 2.2, operating system-specific tests are also available for NT and UNIX platforms.

See the online help for Oracle predefined event tests, "Oracle Event Tests" on page 5-40, and the Diagnostic Pack documentation for information on events and their parameters. All the Node events are supported on Unix and Windows NT platforms. For other platforms, see your platform-specific documentation.

### Fault Management Event Tests

This category of event tests monitors for catastrophic conditions on the system, such as a database, node, or listener is down. Immediate action must be taken by the administrator. Examples of event tests available in this category include:

- Alert

- UpDown

Most of the fault management event tests do not require any threshold values because the event test only checks whether the service is up or down. For the Alert event test, the event test checks whether error messages are written into the database alert log file.

The UpDown event tests are provided with the Enterprise Manager base product. These event tests check whether a database, listener, or node is available. With the UpDown event test for databases or listeners, you can use the Startup Database or Startup Listener task as a fixit job to re-start the database or listener. To avoid executing that job when the database or listener is brought down intentionally, you need to remove the event registration.

### Space Management Event Tests

This category of event tests track possible space problems, such as running out of space on a disk or archive device. Examples of space management event tests in this category include:

- Disk Full

- Archive Full

To check for space management events, set a threshold on the free space left. For example, set an alert if the free space on a disk falls below a specific number of bytes. In order to properly choose the threshold value, you need to know the characteristics of the tablespaces. For example, you would want to know whether the tablespaces contain online transaction processing (OLTP) tables or decision support tables. The former usually has a very fast growth rate, while the latter almost never grows.

### Resource Management Event Tests

This category of event tests track possible resource problems, such as exceeding datafile or lock limits. Examples of resource management event tests in this category include:

- Datafile Limit

- Lock Limit

To check for resource management events, set a threshold on the percentage of a resource used. For example, you can set an alert if the percentage of the datafile resource used is greater than a specified value.

### Performance Management Event Tests

This category of event test monitors the system for performance problems, such as excessive disk input/output or library cache miss rate. Examples of events in this category include:

- Disk I/O

- Library Cache

To check for performance management events, set a threshold on a system value. For example, you can set an alert if the library cache miss rate is greater than a specific value. The set of threshold values is system specific, depending on the hardware platform, number of users, and other factors.

### Unsolicited Event Tests

Unsolicited event tests are event tests that have been initiated outside the Enterprise Manager Event system. An event is considered unsolicited if it is raised by a process other than the Oracle Intelligent Agent, but is running on the same node as the Intelligent Agent. These events are usually provided by third-party software. Creating an unsolicited event allows you to integrate and monitor third-party events. Essentially, there are two phases to setting up an unsolicited event:

- Registering interest in an unsolicited event

- Raising the unsolicited event

#### Registering Interest in an Unsolicited Event

To register interest in an unsolicited event, check the Unsolicited Event Test box in the Event General page and complete the information in the succeeding property sheet pages. Select the Unsolicited event in the Test page and complete the Parameters pages. Information on how to fill out the Parameters page is discussed in the next section. After completing the unsolicited event, you can save and submit the event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.

#### Setting the Parameters Property Sheet for Unsolicited Events

Because unsolicited events originate outside the Event system, you may wish to screen only for specific external events. The Parameters page for the Unsolicited Event Test allows you to filter unsolicited events based upon the following field entries:

- Event Name: Name of the event

- Object: Name of the object that the event is monitoring

- Severity: Level of severity of the event

- Message: Message to be displayed in the Console

##### You can enter a wildcard "*" (no filtering) or a character string in these fields to define the filtering mechanism.

##### Event Name

This is the four-part name of the event in the form:

    /vendor/product/category/name

You can enter any character strings but all four parts and the forward slashes (/) are required. The eventname is assumed to be in 7-bit ASCII, so that it never changes regardless of platform or language. The name of the event that fires must match the value specified in this parameter field in order for the unsolicited event to fire.

Enter a "*" if you do not want to filter on an event name.

##### Object

An object can be any service the unsolicited event is monitoring, such as a database, listener, or node name. Important: The object must be a service that the Intelligent Agent has already discovered.

Enter a wildcard character ( "*") if you want to receive unsolicted events from any type of object. Or, you can use the percent sign "%" for partial filtering. For example, ORCL% for events fired on objects whose name starts with "ORCL."

##### Severity

Enter the numeric severity level (-1, 1, 2) corresponding to the severity of the event.

Severity levels map to different flags in the Console Event Pane:

##### Table 5-1 Severity Level/Event Flag Mapping

Flag Color | Severity | Severity Level
--- | --- | ---
Green | Cleared | Severity -1
Yellow | Warning | Severity 1
Red | Critical | Severity 2

##### Message

Message is a quoted text string that is displayed in the Console, such as "File not found." If a message filter is specified, then the event is passed only if the message matches what is specified in the message filter.

You can specify wildcard characters as part of the filter i.e. Message Filter : ERROR%. In this case the unsolicited event should fire on anything with a message that starts with "ERROR:"

#### Raising an Unsolicited Event Test

To raise unsolicited events, users have a choice of a command-line interface (oemevent executable) or an OraTcl verb (orareportevent). The related syntax is as follows:

    oemevent [event_name] [object_name] [severity] [message]
    orareportevent [event_name] [object_name] [severity] [message]

where event_name is the name of the event that triggered the unsolicited event. object_name is the name of the object that the event is monitoring, severity is the level of severity for the event, and message is the text string displayed in the Enterprise Manager Console. For additional details, please refer to the Intelligent Agent User's Guide.

Note that the severity is specified as a string in oemevent and as an integer in orareportevent. Also, note that the event_name must be a four-part string of the form /a/b/c/d, where the different elements may be used to organize the event test within a hierarchy of event tests. For example, /myevents/node/files/filefound may be an event test you developed. It relates to nodes, more specifically space on nodes, and it monitors for the existence of a particular file.

For an example of using OraTcl to trigger a third-party event, see Tcl Script Examples in the Run Tcl job task. For information on OraTcl and event scripts, see the Oracle Intelligent Agent User's Guide.

#### Raising Unsolicited Events Via Enterprise Manager Jobs

Typically, unsolicited events are evaluated and raised by third-party software. Enterprise Manager allows you to implement unsolicted events through the Job system and Tcl. You create a Tcl job and submit it as a periodic job. The Tcl contains logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Since the job is submitted as a periodic job, the underlying test is evaluated periodically like all regular Enterprise Manager event tests. Using techniques like the ones scribed below, users of Enterprise can implement event monitoring specific to their environments.

##### Example 1: Simple Job Raising an Unsolicited Event

It is possible to submit a job with an imbedded OS command task that executes the oemevent program and passes the program all necessary arguments. All users that have registered for the event raised by oemevent, will receive the event notification. The event has to be known to the administrator submitting the job that raises it.

However, the job that raises the event may contain enough logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Such a job may be submitted as a periodic job so that the underlying test is evaluated periodically, which is similar to regular OEM event tests.

Unsolicited events (their logic actually) are evaluated in their own process and within the proper OS security envelope. For this reason, no longer pose security or robustness threats to the system. What we have introduced here is a procedure where one needs to submit a job in order to monitor for an event.

Consider as an example how to implement an event test which triggers when a particular file is found. Lets call this event /myevents/node/files/filefound. The following Tcl script needs to be submitted as the job:

    # event name
    set event_name /myevents/node/files/filefound
    # filename to look for comes at the first (and only) argument
    set file_name [lindex $argv 0]
    # check for the file, and if it's found trigger the event as critical
    if { [file exists $file_name] }  {
       orareportevent $event_name $oramsg(oraobject) 2 "$file_name found"
    }

In order to receive this event, a user needs to register an event with the Unsolicited test selected and configured to filter in events with name /myevents/node/files/filefound. This event should be registered against a node and will trigger against it. The message attached to the event occurrence will contain the values of all parameters passed into orareportevent.

Although this event is fairly straightforward, there are two things wrong::

1. The event does not ever clear - it should, if the file disappears

2. The event will trigger every time the above script is evaluated, even if it has triggered the time before - you will receive the same alert multiple (possibly infinite) times. An event should not trigger unless there is a severity change.

Any other scripting language or executable program can also be used to implement the logic of an unsolicited event test. However, Tcl is preferable because it allows platform-independent implementation and the fact that the code may be sent from the Enterprise Manager Console `on-demand' without requiring anything to be installed on the Intelligent Agent side.

##### Example 2: Unsolicited Events with the Proper Lifecycle

As with regular Enterprise Manager events, unsolicited events could be triggered only once per condition detected and could clear automatically if the condition that triggers the event is no longer met. Events adhering to this operational pattern are said to have a "proper lifecycle."

Typically, scripts that implement unsolicited events are composed of two basic parts:

1. The part that evaluates the event and sets its associated severity

2. The part that handles the event reporting and avoids multiple notifications if the severity does not change

The following Tcl script illustrates this two-part script implementation, as well as a technique that allows proper event lifecycle.

    #---------------------------------------------------------------------
    #
    # Tcl Procedure
    #        orareportevent1
    #
    # Purpose:
    #        Trigger an unsolicited event only previous state is different
    #
    # Arguments:
    #        - event_name: event test to trigger
    #        - severity: new severity
    #        - message: message to be attached to the event report
    #
    #---------------------------------------------------------------------
    proc orareportevent1 {event_name severity message} {
       # define a 'lock' that its contents define the previous event status
       # and figure out the event state during the previous execution
       global oramsg
       append event_lock [tempdir] "/" $oramsg(jobid) ".el"
       if { [file exists $event_lock] } {
          set f [open $event_lock r]
          gets $f previous_severity
          close $f
       } else {
          set previous_severity -1
       }
       # if event test state has changed, trigger the event at new severity
       if { $previous_severity != $severity }  {
          orareportevent $event_name $oramsg(oraobject) $severity $message
          if { $severity == -1 } {
             rmfile $event_lock
          } else {
     	   set f [open $event_lock w]
    	   puts $f $severity
    	   close $f
          }
       }
    }

    #---------------------------------------------------------------------
    #
    # Event Test Name:
    #        /myevents/node/files/filefound
    #
    # Purpose:
    #        Monitor for the existence of a particular file
    #        The test triggers at warning level if the file exists, but
    #        at critical level if the file is larger than the specified
    #        value
    #
    # Arguments:
    #        - filename to look for
    #        - critical file size
    #
    #---------------------------------------------------------------------
    set event_name /myevents/node/files/filefound
    set file_name [lindex $argv 0]
    set critical_filesize [lindex $argv 1]

    if { [file exists $file_name] } {
       # if the file exists calculate its size in Kilobytes
       set file_size [expr [file size $file_name] / 1024]
       if { $file_size > $critical_filesize } {
          # if file is larger than the critical value, trigger as critical
          orareportevent1 $event_name 2 "Size: $file_size Kb"
       } else {
          # if file is smaller than the critical value, trigger as warning
          orareportevent1 $event_name 1 "Filesize: $file_size Kb"
       }
    } else {
       # if file in no longer there, clear the event
       orareportevent1 $event_name -1 "File does not exist"
    }

##### Example 3: An Unsolicited Event Script that Accesses the Oracle Database

This section contains an example of unsolicited event tests. In this case, the test evaluation involves connecting to an Oracle instance and performing some SQL against it.

This example checks the size of a particular table in the database and triggers the event when a set threshold is crossed. There is a warning value and a critical value. The size of the table is measured by counting the number of its rows.

    #---------------------------------------------------------------------
    #
    # Tcl Procedure
    #        orareportevent1
    #
    # Purpose:
    #        Trigger an unsolicited event only previous state is different
    #
    # Arguments:
    #        - event_name: event test to trigger
    #        - severity: new severity
    #        - message: message to be attached to the event report
    #
    #---------------------------------------------------------------------
    proc orareportevent1 {event_name severity message} {
       # define a 'lock' that its contents define the previous event status
       # and figure out the event state during the previous execution
       global oramsg
       append event_lock [tempdir] "/" $oramsg(jobid) ".el"
       if { [file exists $event_lock] } {
          set f [open $event_lock r]
          gets $f previous_severity
          close $f
       } else {
          set previous_severity -1
       }
       # if event test state has changed, trigger the event at the new severity
       if { $previous_severity != $severity }  {
          orareportevent $event_name $oramsg(oraobject) $severity $message
          if { $severity == -1 } {
             rmfile $event_lock
          } else {
     	   set f [open $event_lock w]
    	   puts $f $severity
    	   close $f
          }
       }
    }

    #---------------------------------------------------------------------
    #
    # Event Test Name:
    #        /myevents/database/space/tablesize
    #
    # Purpose:
    #        Monitor the size of a particular database table
    #        The test triggers at warning level when the warning threshold
    #        is crossed and at critical level when the critical threshold
    #        is crossed
    #
    # Arguments:
    #        - table name
    #        - critical threshold
    #        - warning threshold
    #        - username/password for conneting to target (optional)
    #
    #---------------------------------------------------------------------
    set event_name /myevents/database/space/tablesize
    set table_name [lindex $argv 0]
    set critical_threshold [lindex $argv 1]
    set warning_threshold [lindex $argv 2]

    if { $argc == 4 } {
       set connect [format "%s@%s" [lindex $argv 3] $oramsg(oraobject)]
    } else {
       set connect [format "%s/%s@%s" $SMP_USER $SMP_PASSWORD $oramsg(oraobject)]
    }

    if {[catch {oralogon $connect} lda]} {
       append msg "Cannot connect to target." "\n" $oramsg(errortxt)
       orafail $msg
    }

    if {[catch {oraopen $lda} cur]} {
       append msg "Cannot connect to target." "\n" $oramsg(errortxt)
       oralogoff $lda
       orafail $msg
    }

    set sql [format "select count(*) from %s" $table_name]
    if {[catch {orasql $cur $sql}]} {
       append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt)
       oraclose $cur
       oralogoff $lda
       orafail $msg
    }

    if {[catch {orafetch $cur} row]} {
       append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt)
       oraclose $cur
       oralogoff $lda
       orafail $msg
    }

    set current_tablesize [lindex $row 0]

    if { $current_tablesize > $critical_threshold } {
       orareportevent1 $event_name 2 "Table:$table_name #rows:$current_tablesize"
    } elseif { $current_tablesize > $warning_threshold } {
       orareportevent1 $event_name 1 "Table:$table_name #rows:$current_tablesize"
    } else {
       orareportevent1 $event_name -1 "Table:$table_name #rows:$current_tablesize"
    }

A number of OraTcl verbs were used in this script. Refer to the Intelligent Agent User's Guide for details on OraTcl verbs. Note that the preferred credentials, specified in the console, are available to the script writer via the SMP_USER and SMP_PASSWORD Tcl global variables. For jobs against a database, the values of those variables are set to the username and password specified as preferred credentials for that database. Notice that this script allows for an optional overwrite of the preferred credentials via an optional forth input argument.

#### Unsolicited Event Caveats

- To monitor for an unsolicited event against multiple targets on the same node (say the event above against multiple databases on the same node), you need to register for the unsolicited event multiple times so that you may have the flexibility of proper filtering.

- Unsolicited Events cannot be used with Fixit Jobs.

## Event Pane

The Event pane contains the following pages:

- Alerts Page

- History Page

- Registered Page

You can switch between the pages by clicking the tab of each page. The rows in any page can be sorted on any column by clicking the column heading. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.

The Event pane can be hidden or shown by selecting Show/Hide Event Display in the Console View menu. You can also hide or show the pane by clicking on the flag icon in the Console toolbar.

#### Figure 5-1 Event Menu and Pane

Text description of the illustration event2.gif

### Alerts Page

The Alerts page displays event tests that have been triggered.

Severity

Severity of the event occurrence: critical (red flag), warning (yellow flag), clear (green flag), unknown (gray flag), or error state (yellow hexagon).

Event

Name of the event.

Target

Target where the event was triggered.

Time/Date

Time and date of the event occurrence.

Assigned To

Administrators assigned to work on the event occurrence.

Owner

Administrator who owns the event.

#### Viewing Alerts

To view details of an event that has occurred, double-click on the event to display the Event Viewer property sheet. You can enter notes on the nature and progress of the event condition. Note: Comments entered into the log are viewable/editable by admins with the Modify permission. After you have reviewed an event, you can move it to the History page. See "Event Viewer" on page 5-27 for more information.

### History Page

The Event History page displays a history of events that have occurred and have been moved to History by an administrator or cleared by an Intelligent Agent. The Event History page displays the same columns as the Alerts page.

The History page is refreshed automatically each time you move between the History page and the Alerts or Registered page. However, to refresh the event history list while currently viewing the History pane, you must click the Refresh History icon located at the bottom left of the event History page.

To clear all entries in the History page, click the Clear History icon located next to the Refresh History icon.

### Registered Page

The Registered page displays the events that have been registered, or submitted, to monitor test conditions on any network objects. The Registered page contains the following information:

Event

Name of the event.

Target

Target where the event was monitored.

Target Type

Type of event destination: database, node, listener, web server, Concurrent Manager,

Status

Current registration status of the event: Registered, Registration Pending, De-Registration Pending, and Registration Failed.

Owner

Administrator who owns the event.

Note: Under certain circumstances, an event will remain in a Registration Pending state. There are two probable causes:

1. The Intelligent Agent on the node with which you are trying to register the event is down, or the node is not connected to the network.

2. The node with which you want to register the event was defined manually (without using the Intelligent Agent). Connections to a manually defined node will not allow you to utilize remote management functionality such as Jobs or Events. You must first de-register any jobs or events against the node, remove the node from the Console navigator, and then rediscover the node while it is running a 7.3.4 or later Intelligent Agent.

## Event Menu

The Event menu allows you to set up event and administrator information. This menu also provides options to register, track, and view specific events. Menu options are enabled or displayed according to the items selected in the Event pane. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event menu.

##### Note:

When you register or remove an event, there is usually a slight delay while the Intelligent Agent processes the request.

Create Event

Displays the Event property sheet and allows you to create the definition of a new event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.

Edit Event

Displays the definition of the selected event and allow you to edit the event permissions.

Edit Event Occurrence

Displays the definition of an existing event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.

Acknowledge

Acknowledges the selected event in the Alerts page. When an event triggers, an entry is added to the Alerts page. In the severity column, a flag of the appropriate color is displayed along with a pair of eyeglasses. The eyeglasses also appear whenever there is a change in the status of the event (e.g. from `warning' to `critical') If you choose to "acknowledge" this event, then it means you are aware of this event occurrence and hence the eyeglasses will disappear.

Copy to Library

Copy the selected event in the Event pane to the Event Library.

Remove Event

Removes the selected event from the Event pane, effectively de-registering the event. This menu option only appears when an event test is selected in the Registered page.

Move to History

Moves the selected event in the Alerts page to Event History page of the Event pane.

Refresh History

Updates the History pane with the most recent entries.

Clear History

Clears the contents of the Event History page.

Report Event History

Saves the contents of the Event History page to a report.

Report Registered Events

Creates a report of currently registered events.

Report Event Alerts

Creates a report of the events that have triggered.

Event Library

Displays the Event Library dialog. See "Event Library Dialog" on page 5-26 for more information.

### Context-Sensitive Menus

If you select an item in the Event pane with the right mouse button, the context-sensitive menu for that item appears. This menu is a subset of the Event menu plus selection-specific menu options.

### Event Library Dialog

The Event Library dialog displays the events that have been created and saved to the Event Library. The advantage to using the Event Library is that both events and any associated target information can be stored, copied, or modified in the library for future use. When you create an event, you have the option of submitting, saving to the Event Library, or submitting and saving to the Event Library.

This dialog contains the following information:

Event

Name of the event.

Owner

Administrator who created the event.

#### Editing an Event

Select an event and click Edit to display the property sheet for the event. The property sheet allows you to view and modify the events. Editing most event parameters can only be done via the Event Library dialog box.

#### Oracle Event Tests

Several predefined event tests have been installed with Oracle Enterprise Manager. These appear in the Tests page of the Event property sheet. You can add these tests to an event. The tests include:

- Oracle DB UpDown: checks whether a database is up or down.

- Oracle Host UpDown: checks whether a node is up or down.

- Oracle Listener UpDown: checks whether a listener is up or down.

- Data Gatherer UpDown: checks whether the data gatherer is up or down.

##### Note:

Only the UpDown tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.

To view the specific tests assigned to an event, double-click on the event in the Event Library dialog and view the Test page of the Event property sheet. See the online help for Oracle events, "Oracle Event Tests" on page 5-40, or the Diagnostic Pack documentation for information on Oracle events and their parameters.

## Event Viewer

The Event Viewer property sheet displays details on a selected event in the History or Alerts page. When an event triggers, you select the triggered event and bring it up in the Event Viewer. The Event Viewer contains information on why the event triggered. You can also assign the event to the a particular administrator and put instructions for other administrators via the Log page.

You can enter optional comments in the Log page, which is good way to share information about an event with other administrators. Once cleared, events are automatically moved to the History page. The pages of the Event Viewer include:

- General

- Log

- Notification Details

### General

The Event Viewer General page displays statistics and owner information on a selected event. You can display the property sheet for the event with the Show Event Definition option. The following statistics are displayed on the General page:

Target

Service (database, node, etc.) where the event triggered.

Target Type

Type of service, such as database, listener, or node.

Last Updated

Time of last update.

Owner

Administrator that created the event.

Assigned To

List of administrators that are receiving notifications for this event.

Test Name

Event test that was evaluated.

Severity

Severity of the event occurrence: critical, warning, clear, unknown, or error..

Time/Date

Time and date of the event occurrence.

Message

Message generated from the alert. You may position your cursor above the message to see it displayed as balloon text.

### Log

The Event Viewer Log page displays an entry whenever an event is moved to history. An event can be moved manually with the Move to History menu option or automatically when the severity of the event changes.

The Log page also allows comments to be entered on a selected event. Any administrator with permissions to modify the event can add comments in this page. You enter comments in the text box and select the Apply or OK button to add the comment.

The information displayed in the Log page includes:

Type Entry

Text input field allowing you to add comments.

Entry

Comment that has been entered for this event.

Author

Administrator that entered the comment.

Date/Time

The date and time when the comment was entered.

### Notification Details

The Event Viewer Notification Details displays details of email and paging notifications sent for a selected event. The information displayed in Details page includes:

Severity

The severity flag associated with the event occurrence.

Administrator

Administrator that was notified.

Date/Time

The date and time of the notification.

Method

Method of notification: E-mail or page.

Notification Status

Status of the notification, indicating whether the notification was sent, is pending, or has failed.

Message

If the notification failed, this message indicates the reason for notification failure.

## Creating, Modifying, and Registering an Event

Events include the destination type and the event information that you want to monitor. Events can consist of multiple event tests. To create or modify an event:

1. Choose the Create Event option from the Event menu to display the Event property sheet. (you can also display the Event property sheet by opening an event from the Event Library dialog.)

2. Complete or modify the fields in the General, Parameter, and Permissions pages of the property sheet to create a new event. If you modify an event that has been registered, those changes are not used by the registered event.

3. When you have completed the Event property sheet:

   1. Choose Submit, to register the event against the selected destinations. The new event is not saved to the Event Library.

   2. Choose Add to the Library (or Save to Library if editing an event from the Library) to save the event to the Event Library. The event will not be submitted to the target destinations at this time. The new event appears in the Event Library dialog.

   3. Choose Submit and Add to Library (or Submit and Save to Library) to submit the event to the selected destinations and save the event to the Event Library. The new event appears in the Event Library dialog.

If you registered an event, the Intelligent Agent for a destination processes the event and the event appears in the Registered page of the Event pane. Each destination service is listed separately with the event.

##### Note:

There is usually a slight delay between the time the event is registered and the actual notification by the Intelligent Agent.

When threshold values are exceeded for the tests in an event, the event appears in the Alerts page of the Event pane. This notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:

- Unknown (gray flag)

- Event cleared (green flag)

- Warning (yellow flag)

- Alert (red flag)

- Error (yellow hexagon)

Cases where an event notification is Unknown (gray flag) indicate the target or node where the event is registered is unavailable.

##### Warning:

Do not register an UpDown event (included in the Oracle DB Fault event) against the database or node where the Repository schema is stored. If the database containing Repository goes down, the Management Server also shuts down. Hence, the Intelligent Agent cannot inform the Management Server that the database is down.

The property sheet for creating a new event is the same as the property sheet for modifying an event, except that the event name field is read-only. See Figure 5-2, "Event General Page" for an illustration of the Event property sheet.

See "Event Categories and Types" on page 5-10 for more information.

### Modifying a Registered Event

To modify a registered event, perform the following steps:

1. If you need to modify any of the administrator permissions for the event, including enabling or disabling notifypermissions, then you can select the Event from the Registered pane, make the modifications, then click "OK" to accept the changes.

   If the monitored destinations for the event specify multiple targets, e.g. databases A, B, C, then any changes made to the permissions for the Event will apply to all targets, A, B and C.

2. If you need to modify any other event parameter (destination list, parameter values, etc). then first make a copy of the event to the library if you haven't already done so. The "Copy to Library" is an option available from the right-mouse context menu of the registered event. Next, de-register the Event(s) you'd like to modify. Then, go to the Event library, and select and edit the event. Finally 'Submit' the event for registration.

### Event General Page

On the General page, you determine the event name, destination type, description, fixit job, polling frequency, and whether this event should monitor third-party events.

#### Figure 5-2 Event General Page

Text description of the illustration eventgen.gif

Event Name:

Enter an event name.

Destination Type:

Select the destination type you want to monitor from the pull-down list. The types include Database, Listener, Node, or other service that is integrated into the Console.

If the selected Destination Type is "Node", then a second pull-down list of operating systems will appear. If you choose `All', then event tests that apply to all types of nodes, i.e. operating systems, will be available. If you choose a particular operating system, (e.g. Solaris), then additional operating-system specific event tests will be available.

In general, the selection of the Destination Type determines the list of Available Destinations. If your choose "Node" and a particular operating system, such as Solaris, then the list of available destinations will show all Solaris nodes that are running at least an 8.1.7 Intelligent Agent. Any Solaris nodes that use older agents will not be shown.

Event Description:

Enter a description or comment for the event

Fixit Job

A fixit job is designed to correct a problem. For example, you may want the Intelligent Agent to run a job to restart a database when the database instance has shut down unexpectedly. Fixit jobs have been created with the Job system and have been designated as fixit jobs. The jobs must be submitted and running on the same destination that the event is set on. You must select destinations to make fixit jobs available in the pull-down list. See "General Page" on page 4-13 for more information.

Fixit job options are:

- To choose a fixit job, select an existing fixit job from the pull-down list.

- Select <None> if you do not want to use a fixit job.

To turn off a fixit job after an event has been registered, you can copy the registered event to Event Library, remove the event registration, edit the event from the Library to remove the fixit job, and re-submit the event for registration.

##### Note:

Each event must use a unique fixit job on each destination where the event is registered. Also, when a single Intelligent Agent is monitoring multiple databases at a destination, create a separate event and fixit job for each database.

Frequency

Determine the frequency that you want the test to be evaluated on the selected destinations. The frequency determines how often the event condition is checked. For example, if the frequency is set to 60 seconds, the event condition is checked every 60 seconds. The frequency should not be less than 60 seconds. To set the frequency:

- Enter a value in the frequency field or select a value from the pull-down list.

- Select the interval units from the pull-down list.

Consider carefully the value for frequency. Some event evaluations may take more time, particularly if there are large numbers of objects to be monitored. The following list covers some of the potentially longer-running database event tests:

- Continued Row*

- Index Rebuild*

- Chunk Small

- Fast Segment Growth

- Maximum Extents

- Multiple Extents

- Tablespace Full

- SnapShot Log Full

* Continued Row and Index Rebuild are resource expensive because they perform an ANALYZE on the segments they monitor.

See "Event Notifications" on page 5-7 for details on the notification frequency.

Enable Unsolicited Events

Check this box to monitor unsolicited events. See "Unsolicited Event Tests" on page 5-12 for more information.

### Event Tests Page

On the Tests page, you determine the event tests that you want to perform. Event test are arranged hierarchically in a tree list for ease of viewing and selection. As with the Console Navigator, you can expand and compress entries in the tree list.

#### Figure 5-3 Event Tests Page

Text description of the illustration evttest.gif

Available Tests:

Select the event tests in the list you want to perform in this event, then click on the << (Add) button to move the events to the Selected Events list.

Selected Tests:

Select the event tests in the list you want to remove from this event, then click on the >> (Remove) button.

### Event Parameters Page

The parameter settings for the selected event tests are entered in the Parameters page of the Event property sheet. The settings and types of parameters vary according to the event test selected. Some event tests do not have parameters. See the online help for Oracle events and "Oracle Event Tests" on page 5-40 for information on tests and their parameters.

#### Figure 5-4 Event Parameters Page

Text description of the illustration evtparms.gif

#### Parameters

The parameters for an event are displayed when the event is selected in the Selected Tests list. The parameters vary according to the event selected. Some events do not have parameters.

You can accept the default values or change the values for the parameters. To enter parameter values for an event, you can enter a value directly into a parameter field.

Filtering

Filtering is used in events such as Chunk Small and Maximum Extents. Examples of filters are = 'SYSTEM', LIKE '%SMP%', and IN ('SYSTEM', 'TOOLS'). Note that the quotes are single quotes. Use uppercase to match the case of the database object names. If you enter a filter value that does not select any objects or is an incorrect value, the event fails.

### Event Permissions Page

Determine the permissions that you want to assign to the event with the Permissions Page. This allows other administrators to view or modify the event. Notifications are also assigned with this page.

The levels of permission that you can assign to an Enterprise Manager administrator are:

None

Does not allow the administrator to view this event anywhere.

View

Allows the administrator to view the event, inspect event properties, and receive notifications.

Modify

Allows the administrator to modify the event's log (See "Event Viewer" on page 5-27), and edit the event's properties except those reserved for Full permission.

Full

Allows the administrator to delete the event, modify permissions for other administrators, and change the ownership of the event.

Notify

Allows the administrator to receive enhanced event notifications on the objects through paging or email. Other notifications will be routed to that particular administrator's Console. Notify permission cannot be assigned if the administrator's permission level is set to None.

Any permissions assigned on this page supersede any administrator default permissions. See "Permissions" on page 1-23 for more information. Also, the administrator's notification schedule must be set up in order for them to receive the E-mail/page notification.

Enable SNMP Notifications

When checked, permits external notification (SNMP traps) for third-party event tests at the destination where the Intelligent Agent is located.

Show Notification Schedule

Show Notification Schedule displays the notification schedule for the event. The schedule shown on this page is a combined schedule for all administrators that have been given "Notify" privileges for this event. To view administrators assigned to a particular time slot, use the right mouse button to call up the context-sensitive menu, choose the "Remove Recipient" option, and view the list of administrators. To add or remove notifications for an administrator, display the context menu (press the right mouse button) on any time block. The context menu provides options for adding and removing recipients of the notifications.

### Event Progress Page

The Event Progress page displays when you edit an event from the Registered page of the Events pane. This page provides the current registration status for the event selected: Registered, Registration Failed, or Registration Pending. In addition, the destination and time and date when registration was attempted is shown.

When the Progress page is displayed, it shows only the status for the selected event. If the selected event is registered, or had been submitted for registration on other destinations, you can view the status of this event for those destinations by selecting the desired target from the Destination pull-down list. The status of the event displays for that destination. To view the status of this event for all destinations simultaneously, select <All>.

The following options are available on the Progress page:

Destination (pull-down list)

Select the destination of the event you want to view from the pull-down list. Select <All> for all destinations for which this event has either been registered or failed to be registered.

Status

Status for the event: Registered, Registration Pending, or Registration Failed.

Destination

Network destination for the event.

Date/Time

Date and Time the event was submitted for registration.

Show Output

Displays the Event Status Message dialog. This button is active only when you have selected a failed event registration. Selecting this option will allow you to view the reasons for the failure.

Save List

Saves the contents of the list to a text file.

### Example: Creating and Registering an Event

In the following example, a new event for monitoring extents in a database is created and registered at several databases. When creating an event, the General, Tests, Parameters, and Permissions Pages are completed to define the event. You need to register, or submit, an event to run event tests on specific destinations in the network environment.

The following example is used for illustrative purposes. When creating actual events, you must enter accurate values for the polling frequency, and parameter values.

1. To create a new event, select Create Event from the Event Menu to display the General Page of the Event property sheet.

2. Enter an event name in the Event Name field, such as Maximum Extents. Provide a descriptive name so the event can be easily identified from the Event pane.

3. Enter a brief description of and comments about the event in the Description field. The event in this example will monitor Maximum Extents. This event is only available with the Oracle Diagnostics Pack.

4. Select Database from the Destination Type pull-down list.

5. Keep the frequency at 60 seconds.

6. Do not select a fixit job (if any are available). These are jobs that have been created with the Job system and have been designated as fixit jobs.

7. Select a database from the Available Destinations list, then click on the << (Add) button. The database is moved to the Selected Destinations list and will be monitored for the events in the event.

8. Repeat the previous step for several additional databases.

9. Click on the Tests page tab to display the Tests page.

10. Select the Maximum Extents event test. This test checks the number of available extents for table, index, cluster, or rollback segments. Because this test accesses system tables, you must set up Administrator Preferred Credentials with an administrator that has system privileges. See "Preferred Credentials" on page 1-25 for more information.

Note: The Maximum Extents event test is only available with the optional Diagnostics Pack. The standard Up/Down event tests provided with Enterprise Manager do not require parameter settings.

1. Click on the << (Add) button. The event test is moved to the Selected Tests list and will be added to the event.

2. Click on the Parameters page tab to display that page. If you had selected several tests, you would need to select a specific test in the Selected Tests list to display the Parameters for the event test.

3. Click on Warning threshold in the scrolling list, to change the value of the parameter. Enter 3 in the text field.

4. Click on Alert threshold in the scrolling list, to change the value of the parameter. Enter 2 in the text field.

5. Do not change the other parameters. Note that the * in the Tablespace name, Segment name, and Segment type parameter fields signifies all existing values. To monitor a specific tablespace, such as USERS, you would enter = 'USERS' in the Tablespace name field.

6. Click on the Permissions page tab to display that page. Assign permissions to all administrators to allow them to view the event and receive notifications.

7. After you have completed the event property sheet, select the Submit and Add to the Library button at the bottom of the property sheet to submit the event to the selected destinations.

8. If you want to modify the event, you can edit this event in Event Library dialog. In order for new values to affect event monitoring, you need to de-register and then re-register the event.

When an event is submitted, each destination is validated and the Intelligent Agent for each destination processes the event. If the registration is successful, the event appears in the Registered page of the Event pane. Each destination database is listed separately with the event.

If an event condition occurs or threshold values are exceeded for an event test, a notification is sent to the Alerts page of the Console Event pane. If administrators have been selected to be notified by email or paging, an email or page is sent. The event notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:

- Unknown (gray flag)

- Cleared (green flag)

- Warning (yellow flag)

- Critical (red flag)

- Error ( yellow hexagon)

After an event condition is fixed, the event is cleared automatically. There are four event tests that are exceptions to this rule: Alert, Data Block Corruption, Session Terminated, Archiver Hung. Since there is no way to determine automatically that these event conditions have been resolved, these events must be manually cleared.

### Administrator Event Notification

Oracle Enterprise Manager allows you to specify administrators that are notified when a particular event condition occurs. Each administrator can be associated with an email ID and/or a pager number. When using a paging service or email notification, each administrator can be assigned responsibility for specific systems at specific days and times.

For more information on setting up Oracle Enterprise Manager administrators, see "Managing Enterprise Manager Administrators" on page 1-8.

## Oracle Event Tests

This section lists the Event system event tests with their parameters and return values. See "Event Parameters Page" on page 5-34 for information on entering parameter values. A list of event tests with numeric pager event Ids is also provided. See "Numeric Pager Job/Event Ids" on page 5-42 for more information.

Event tests are specified for database, listener, and node services. The event tests are also divided into fault, space, resource, and performance management categories. Only the UpDown event tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.

The event test scripts are written in the Tool Command Language (Tcl) enhanced with Oracle Tcl commands (OraTcl). For information on using Tcl and OraTcl to write event scripts, see Oracle Intelligent Agent User's Guide.

Some of the event tests require special tables in the database. For example, the catblock.sql script needs to be run to use the User Blocks event test.

Some of the database event tests, such as Chain Row, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Enterprise Manager Monitor Role" on page 5-5 and "Preferred Credentials" on page 1-25 for more information.

### Database Fault Management Event Tests

This category of event test tracks severe problems that require immediate attention.

##### UpDown (Database)

This event test checks whether the database being monitored is running. If this event test is triggered, other database events are not ignored.

Parameters

none

Action

The Startup Database job task can be set up as a fixit job for automatically correcting the problem.

##### Note:

If the listener serving a database is down, this event test may be triggered because the Intelligent Agent uses the listener to communicate with the database.

### Node Fault Management Events

This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.

Note: If an Up/Down event test (Database Up/down, Listener Up/Down) has been registered on a node, then the UpDown Node event test is implicitly registered and is triggered when the Intelligent Agent is down, or when the Management Server looses contact with the node.

##### UpDown (Node)

This event checks whether the Intelligent Agent on a node can be accessed from the Console via periodic "heartbeating" between the Management Server and the Intelligent Agent. Since this event does not actually register with the Intelligent Agent, no fixit jobs can be associated with it. If the defaults are accepted, this event will trigger within five minutes of the Intelligent Agent becoming inaccessible and will clear within three minutes of the Intelligent Agent becoming accessible again.

Parameters

none

### Listener Fault Management Events

This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.

##### SQL*Net UpDown (Listener)

This event checks whether the listener on the node being monitored is available.

Parameters

none

Action

The Startup Listener job task can be set up as a fixit job for automatically correcting the problem.

### Data Gatherer Fault Management Events

The Oracle Data Gatherer Alert and UpDown events are both Fault Management events for the node destination type. This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.

##### UpDown (Data Gatherer)

This event checks whether the Intelligent Agent data gathering service on a node can be accessed from the Console. If the Intelligent Agent data gathering service is down, this event is triggered.

Parameters

none

### Numeric Pager Job/Event Ids

The Event Management System provides paging services that notify an administrator with a page when an event has occurred. Alphanumeric pagers provide a brief text message identifying the event. Numeric pagers provide the numeric pager event Ids to identify the event.

For job notifications, you will receive a 6 digit number. The first 3 digits indicate the job-id. The last 3 digits indicate job status.

For event notifications you will receive the event ID with the status code.

For a complete list of pager job/event IDs, see "Paging Status Codes for Numeric Pages" on page 1-18

### General Event System Troubleshooting Procedures

The following procedures will assist you and/or Oracle Support if problems are encountered with the Event subsystem.

#### Obtaining Enterprise Manager Repository Status

If you need to contact Oracle Support, you must provide specific information on the status of your Oracle Enterprise Manager repository. To assist you, a SQL script has been provided that extracts pertinent Repository data and generates a debug log file.

To generate the log file, perform the following procedure:

1. Save oms.log.* for every Management Server connected to the repository. The log files are, by default, located in

   $ORACLE_HOME/sysman/log.

2. Log on as the repository user and connect to the repository using SQL*Plus. For example:

   connect <repository name>/<repository password>

3. Run the following SQL script:

   @$ORACLE_HOME/sysman/admin/evtdbg.sql

4. This SQL script will generate a file called evtdbg.log in the directory where Server Manager or SQL*Plus is invoked.

5. Submit the Management Server log files along with evtdbg.log to Oracle Support.

Note: The size of the evtdbg.log file can be quite large.

## Event System Features and Requirements

Because the Enterprise Manager framework is a three-tier system that can manage a heterogeneous environment, it is important to keep in mind various software version requirements necessary for proper event system operation. The following table lists event system features and associated software version requirements.

##### Table 5-2 Event Features and Their Requirements

Feature Name | Description | Enterprise Manager Version | Required Agent | Management Server/Console Required | Works In Browser
--- | --- | --- | --- | --- | ---
Advanced Events | All events for databases, nodes, listeners. See Enterprise Manager online help for more information. | Diagnostics Pack 1.5.5 and highe.r | All supported agents, latest recommended | For EM 2.x, the Management Server and Console that corresponds with that Pack | yes
Event Handler | Component that allows you to log event information or execute custom commands in response to an event occurrence. See Appendix A, "Enterprise Manager Event Handler" for more information | 2.0.4 and higher/ beta | n/a | 2.0.4 OMS | n/a
|  | 2.1 and higher / production | n/a | 2.1 OMS | n/a
Improved Node Up/Down Monitoring | Enhancement to the Node Up/Down event test. Provides more information on whether or not the node is down, the agent is down, etc. | 2.2 and higher | all supported | 2.2 and higher | yes
User-Defined SQL Test | Allows you to write your own custom SQL to monitor database events | Diagnostics Pack 2.1 and higher | 8.1.6 and higher | 2.1 and higher | yes
Enhanced monitoring for target subcomponents | "For events whose targets involve multiple subcomponents (e.g. monitor tablespace full for ALL tablespaces), information on which subcomponent is in alarm is now provided | "2.2 and higher | 8.1.7 and higher | 2.2 and higher | yes
Context sensitive help for Event tests | "In the Parameters tab of the Event dialog, invoking ""Help"" will bring up information pertinent to the current selected event test | "2.2 and higher | n/a | 2.2 and higher | yes
Events with synonymous event tests | "Events can be created that have more than one of the same event test (e.g. a ""Tablespace Full"" test for ""SYSTEM"", another ""Tablespace Full"" event test for ""USER"") | "2.2 and higher | all supported by EM 2.2 | 2.2. and higher | yes
Job and Events Notification filters |  |  |  |  |
Filters that apply to both paging & email | Allows you to filter pages & emails based on job and event status | 2.1 | all agents supported by EM 2.1 | 2.1 | yes
Different filters for paging & email | Allows separate filters for pages & emails based on job and event status | 2.2 and higher | all agents supported by EM 2.2 | 2.2 and higher | yes
Customization for paging & email messages | Allows you to customize the messages for email and pages | Diagnostics Pack 2.2. and higher | all agents supported by EM 2.2 | 2.2 and higher | yes
Advanced O/S event tests | New event tests that monitor operating system specific metrics | 2.2 and higher | 8.1.7 and higher | 2.2 and higher | yes
Concurrent Manager Events | Events to monitor error conditions against the Oracle Applications Concurrent Processing Server | 2.0.4 and higher | 8.1.5 and higher | 2.0.4 and higher | yes
Forms Server Events | Events to monitor error conditions against the Oracle Developer Forms Server | 2.0.4 and higher (2.0.4 console needs Forms Extensions. | 8.0.6 and higher (requires Forms Agent Extensions. | 2.0.4 and higher (2.0.4 console needs Forms Extensions. | yes
Program Filtering in Concurrent Manager Events | Allows you to monitor particular Oracle Applications Concurrent Programs. Also allows you to exclude particular Concurrent Programs from being monitored. | 2.2 and higher | 8.1.7 and higher | 2.2 and higher | yes

| Copyright © 1996-2000, Oracle Corporation. All Rights Reserved. |
Library | Service | Contents | Index