12 SNMP Monitoring

12.1 Prerequisites

You need to have a network management station (NMS) installed, and you should be familiar with how to upload and display network statistics gathered from a Management Information Base (MIB). This chapter describes the Oracle Access Manager MIB objects and the Object Identifiers (OIDs) for these objects. However, this chapter does not provide information on how to use these OIDs in your NMS to collect statistics. For such information, refer to the documentation for your NMS.

12.2 About Oracle Access Manager SNMP Monitoring and Agents

The Simple Network Management Protocol (SNMP) enables you to monitor component activity on the network that hosts your Oracle Access Manager system by collecting and displaying server-related SNMP data on a network management station. SNMP statistics commonly include data such as:

• The hosts, routers, and servers on your network

• The number of requests being processed on a particular device

• Whether or not a particular device is running

• Whether requests were processed successfully

SNMP data is displayed on a network management station (NMS). The NMS is a workstation running a network management application such as HP OpenView. You configure the NMS to display network statistics in a useful way, for instance, as a graph to show simple network statistics or to show whether the number of requests a device is processing falls within a set of defined limits.

You can capture SNMP statistics for the Identity Server and the Access Server running on any supported platform. Oracle Access Manager supports SNMP polling and trapping. Polling collects information such as:

• The version number of a component

• Configuration status

• Connection status

• Statistics on actions the component has processed

Event traps include information such as:

• Component failure

• Event failure

• Connection status

• Failure to complete actions

12.3 About the Oracle Access Manager MIB and Objects

The Management Information Base (MIB) is a specification file that contains variables relevant to the status of different Oracle Access Manager components. The SNMP Agent collects values for fields in the MIB.

Figure 12–1 illustrates the Oracle Access Manager MIB hierarchy.

Figure 12-1 The MIB hierarchy

The Oracle Access Manager MIB can be expressed as a concatenation of branch and object identifiers (OIDs). The label from the MIB root to the top node of the MIB is as follows:

iso.org.dod.internet.private.enterprises.oblix.snmp

MIB files are located in SNMP_install_dir/oblix/mibs. These files conform to SNMP Version 2.

The following discussions describe the MIB objects that are provided with the Oracle Access Manager SNMP component.

1.3.6.1.4.1.3831.10.1.1.2.1.1.localhost.6023

1.3.6.1.4.1.3831.10.1.1.2.1.1.localhost.6023

1.3.6.1.4.1.3831.10.1.1.2.1.1.9.108.111.99.97.108.104.111.115.116.6023

iso.org.dod.internet.private.enterprises.oblix.snmp.coreid. versionone

iso.org.dod.internet.private.enterprises.oblix.snmp.coreid.versionone. pppActionsTable

iso.org.dod.internet.private.enterprises.oblix.snmp.coreid.versionone.coreidDirectoryServerTable

iso.org.dod.internet.private.enterprises.oblix.snmp.coreid. versionone

iso.org.dod.internet.private.enterprises.oblix.snmp.coreid.versionone.oblixCoreidServerDown

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa.versionone

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa.versionone.aaaDirectoryServerTable

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa. versionone.aaaauthenticationPluginsTable

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa. versionone.authorizationsPluginsTable

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa. versionone.requestQueueInfoTable

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa.versionone

iso.org.dod.internet.private.enterprises.oblix.snmp.aaa.versionone. oblixAAAServerDown

12.4 Enabling and Disabling SNMP Monitoring

You use the Identity and Access Servers configuration pages to enable SNMP and to indicate the TCP/IP port where contact will be established with the SNMP Agent.

The following procedure describes how to start and stop the Oracle Access Manager SNMP Agent, and how to start the Agent on another port.

To configure collection of SNMP statistics

1. From the Identity (or Access) System Console, select System Configuration, Identity Server (or Access Server.)

2. Click a link for a particular server.

3. Select the Modify button to display the page where you can turn SNMP monitoring on or off, as follows:

   • Turn On: Select the SNMP State On button at the bottom of the page.

   • Turn Off: Select the SNMP State Off button at the bottom of the page.

4. In the SNMP Agent Registration Port field, enter the port number to define or change the port on which the SNMP Agent listens.

5. Restart the Identity Server (or Access Server).

12.5 Setting Up SNMP Agent and Trap Destinations

You use the following command to setup an SNMP Agent against an SNMP Manager:

setup_agent -i

The -i option is required.

Following procedures describe and illustrate how to configure the Oracle Access Manager SNMP Agent and trap destinations.

To configure the SNMP Agent and trap destinations

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

   where SNMPDIR is the directory where you have installed the SNMP Agent.

2. Use the setup_agent command with the following options:

   -i <install_dir>-g Configure General Parameters-u <Agent SNMP UDP Port>-c <Agent Community String>-p <Agent TCP Port>-S <Run in silent mode>--help Prints help message

To add a trap destination in silent mode

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

2. Use the setup_agent command with the following options:

   -a-m <Manager Station>-t <Trap port>

To delete a trap destination in silent mode

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

2. Use the setup_agent command with the following options:

   -d-m <Manager Station>-t <Trap port>

To configure general parameters first

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

2. Use the following setup_agent command:

   > ./setup_agent -i $SNMPDIR -g -u <UDP Port> -c public -p <TCP Port>
   
   

   This goes to the Manager Station Trap Configuration menu.

To add an SNMP Manager directly after general parameters

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

2. Use the following setup_agent command:

   > ./setup_agent -i $SNMPDIR -a -m <Mgr M/c> -t <Mgr Port>
   
   

To delete an SNMP Manager directly after adding one

1. Change to the directory containing the SNMP setup_agent command.

   For example:

   > cd $SNMPDIR/oblix/tools/setup
   
   

2. Use the following setup_agent command:

   > ./setup_agent -i $SNMPDIR -d -m <Mgr M/c> -t <Mgr Port>
   
   

   You can add any number of Manager Stations. The Agent then sends all the traps to the configured SNMP Managers.

12.6 Changing SNMP Configuration Settings

A configuration file named obscoreboard_params.xml contains information that defines the collection of SNMP statistics. This file is located in:

Component_install_dir/identity|access/oblix/config

where Component_install_dir is the directory where the component is installed and identity|access represents either the Identity Server or Access Server, respectively.

Identity System File: obscoreboard_params.xml

Access System File: obscoreboard_params.xml

In this file, you can configure threshold levels to determine when various MIB counters are activated.

The following parameters are specified only in the Access Server file obscoreboard_params.xml:

• NumberofAuthenticationPlugins: The maximum number of authentication plug-ins that may be loaded in the Access System. The Access Server maintains information on the number of plug-ins that are loaded. If the actual number of plug-ins loaded by the Access Server exceeds the value specified for NumberofAuthenticationPlugins, the difference is displayed as the counter aaaOverflowforAuthenticationPluginSlots.

• NumberOfAuthorizationPlugins: The maximum number of authorization plug-ins that may be loaded in the Access System. The Access Server maintains information on the number of plug-ins that are loaded. If the actual number of plug-ins loaded by the Access Server exceeds the value specified for NumberofAuthorizationPlugins, the difference is displayed as the counter aaaOverflowforAuthorizationPluginSlots.

The following parameter is specified only in the Identity Server file obscoreboard_params.xml:

• NumberOfPPPPluginActions: The number of Identity Event API plug-in actions that may be connected with this Identity Server. When the Identity Server starts, it reads this value and monitors the actual number of Identity Event API plug-ins. If the number of active plug-ins exceeds the value for NumberOfPPPPluginActions, the difference is indicated by the counter coreidOverflowForPPPActionsSlots.

The following parameters are provided in both scoreboard files:

• NumberOfServiceThreads: The value for this parameter is read by the Identity or Access Server at startup. This parameter controls how many slots to allocate (one for each service thread) to maintain SNMP information for each service thread. The server monitors the number of service threads being used. The actual number of service threads is configured through the administration console, from the command line, or as part of a configuration file. This parameter does not control the number of threads to be started by the Identity Server. If the actual number used exceeds this value, there is no SNMP data generated regarding the extra threads.

• NumberOfConfiguredDS: The number of directory servers configured for this Identity or Access Server.

• DsFailureTrapTimeSpan: The amount of time to wait before sending the next failure trap to the same directory server.

• NumOfSlotsInEventQueue: The number of slots to be used in the event queue. This parameter must be updated if traps are not detected. However, the default value of five should be adequate for most installations.

• SleepTimeInMilliSec: The interval in milliseconds that the Identity or Access Server uses to check whether the SNMP Agent is up and running.

• semaphore_filepath: Information about the semaphore created by the Access Server. Semaphores are used for synchronization between the component (the Identity or Access Server) and the SNMP Agent. This information is used to automatically clean up the semaphore if the component fails.

• semaphore_id: The Agent semaphore identifier.

Changing these settings affect the memory map file used for SNMP data collection. On Unix, the memory map file is located in

/tmp/netpoint/scoreboard/component/process-id.osb

On Windows, this file is located in

Component_install_dir/oblix/scoreboard/process-id.osb

12.7 Logging for SNMP

The SNMP Agent supports logging. Once the SNMP Agent is enabled, it is always set to a certain log level. The SNMP logs can assist with troubleshooting. You can configure what is logged and the type of logs to generate in the Agent configuration file. This file resides in

SNMP_install_dir/oblix/config/snmp_agent_config_info.xml

where SNMP_install_dir is the directory where the SNMP Agent was installed.

The log_level parameter in the Agent configuration file may have one of the following values:

• 0: Debug

• 1: Information

• 2: Warning

• 3: Error

• 4: No logging (turns logging off)

12.8 SNMP Messages

The following are SNMP-related messages.

Message:

MErrNoConfigFile {Could not find agent configuration file at location (full path to the agent configuration file)}

Description: The installation directory is not correct, or the configuration file is not present. Uninstall and reinstall the SNMP Agent.

Message:

MLogAgentStarted {Agent successfully started on port SNMP port number}

Description: Status message.

Message:

MErrAddressInUse {Agent was not able to bind to port port number, address already in use}

Description: The SNMP Agent is unable to bind to its configured TCP registration port. Reconfigure the Agent to use another TCP port, or make the port available by stopping the application using the port.

Message:

Agent was not able to bind to specified port, system lacked sufficient buffer space or queue was full.

Description: The SNMP Agent port is unavailable.

Message:

MErrTLUnsupported {Agent was not able to bind to specified port, address family not supported by protocol family}

Description: The specified port does not support SNMP. Configure a different port.

Message:

MErrRetriveIDs {Error: Unable to determine the uid/gid for which this snmp agent is installed.}

Description: The user who tried to start the SNMP Agent does not have the appropriate permissions. The user should start the SNMP Agent as root or as the user who installed the Agent.

Message:

MErrCouldNotSetIDs {Error: You don't have sufficient access rights to run this snmp agent.}

Description: You need to log in with administrative rights to be able to install the SNMP Agent. If you did not do this, the Agent is unable to run.

Message:

MLogAlreadyRunning {Agent is already running with process id (Process identifier of the agent).}

Description: The user is trying to start the Agent when it is already running.

Message:

MErrRegBindFailed {Error: Unable to bind to configured registration port (configured registration port number).}

Description: The SNMP Agent is unable to bind to the port configured on the Oracle Access Manager server configuration page. Specify a different port, as described in "Enabling and Disabling SNMP Monitoring".

Message:

MErrRegListenFailed {Error: Unable to start listening on configured registration port (configured registration port number).}

Description: This message is displayed on Windows if the port is already in use by another application.

Message:

MErrReadingMsg {Error reading message sent by component.}

Description: The SNMP Agent and the Oracle Access Manager server talk over a TCP connection. If the Agent encounters a malformed message, it logs an error.

Message:

MErrNotRegMsg {Error: Agent expects only registration messages on the registration socket.}

Description: The Agent only expects registration messages on the TCP connection from a server that connects to it. If it finds that the message is not a registration message, it logs an error.

Message:

MErrMissingMmapFilename {Error: Registration message was missing the component scoreboard file name.}

Description: The scoreboard file is where the Identity or Access server stores the statistics that are read by the Agent. This name is communicated by the server to the Agent at registration time. If the registration request is missing the file information, this message is logged.

Message:

MErrMappingScoreboard {Error: Unable to memory map the scoreboard file (full path to the scoreboard file) registered by component.}

Description: This error can occur due to file permission issues, that is, the Agent cannot read or open the scoreboard file.

Message:

MErrUnknownComponent {Error: Unknown component type specified in scoreboard file.}

Description: The component type is specified in the registration request. The Agent processes information for the Identity Server and Access Server. If the component type is not either of these, this message is logged.

Message:

MErrIndexExists {Error: A component has already registered in table (OID for the table for that component) with index (index that is already in use by some other component).}

Description: The same instance of a component tried to register again. Each instance of a component is uniquely identified by a key or index by the same SNMP Agent. If another component instance tries to register using the same key or index, this message is logged.

Message:

MErrCreatingAgentSemaphore {Error: Unable to create named semaphore (full path to the agent semaphore file) for agent-component event dispatching.}

Description: The Agent and the component create one semaphore that is cleaned up at shutdown. In case of unclean shutdown, the semaphores are deleted on the next server/Agent startup. Probable causes are that the system has run out of semaphores or there are permission issues while creating the semaphore.

Message:

MErrOnSelect {Error: Select() call returned error code (error code returned for the select() call).}

Description: This is an error code returned directly from the function. This message is used for troubleshooting purposes.

Message:

MErrOnPoll {Error: Poll() call returned error code (error code returned for the poll() call).}

Description: This is an error code returned directly from the function. This message is used for troubleshooting purposes.

Message:

MErrNotDeregMsg {Error: Agent expected a de-registration message on the socket, instead got a message with code (message code for the message received).}

Description: The Agent only expects a de-registration message from a component once the component has registered.

Message:

MErrRemovingComponent {Error: Component with table oid (OID for the table for that component) and index (index which identifies the component in that table) could not be removed.}

Description: The component has already de-registered, and there has been another request to remove it.

Message:

MErrMissingEvent {Error: Unable to retrieve event from component with table oid (OID for the table for that component) and index (index which identifies the component in that table).}

Description: The component sends an event to the Agent, and the Agent converts this to an appropriate trap. The component also signals the Agent that it has dispatched an event. If the Agent is signaled but it does not find an event, this message is logged.

Message:

MErrMissingTrapData {Error: Missing trap meta-data for component from table oid (OID for the table) and index (index that identifies the component in that table) with event (event identifier supplied by the component).}

Description: The component did not deliver the complete data for an event.

Message:

MLogMappedScoreboard {Mapped scoreboard file (full path to the scoreboard file) for a component.}

Description: This is a status message.

Message:

MLogComponentRegistered {Component registered with table oid (OID for the table) and index (index that identifies the component).}

Description: This is a status message.

Message:

MLogComponentDeregistered {Component with table oid (OID for the table) and index (index that identifies the component) de-registered.}

Description: This is a status message.

Message:

MLogComponentFailed {Component with table oid (OID for the table) and index (index that identifies the component) failed.}

Description: This is a status message indicating that the Oracle Access Manager component did not deregister properly. This action is treated as a component failure by the SNMP Agent.

Message:

MLogSentTrap {Sent trap with trap oid (OID for the trap sent) for component with table oid (OID for the component table) and index (index that identifies the component in the table).}

Description: This is a status message.

Message:

MLogSemCleanup {Found left-over semaphore from previous run with key (key for the stale left-over semaphore) and file path (file path for the stale left-over semaphore), successfully cleaned up the semaphore.}

Description: Status message. The Agent and the component create one semaphore that is cleaned up at shutdown. In case of unclean shutdown, the semaphores are deleted on the next server/Agent startup.

Message:

MErrSemCleanup {Found left-over semaphore with key (key for the stale left-over semaphore) and file path (file path for the stale left-over semaphore). Encountered errors while removing it.}

Description: The Agent and the component create one semaphore that is cleaned up at shutdown. In case of unclean shutdown, the semaphores are deleted on the next server/Agent startup. This message would be logged if the Agent encountered errors while cleaning up the semaphores from a previous run. There may be permission issues.

Message:

MSBCreateFailed {Access Server: Could not create scoreboard file (full path for the file) with size file size.}

Description: The probable cause for this message is the system could not create the file due to insufficient space.

Message:

MCreateSemFailed {Access Server: Could not create event queue semaphore with path full path.}

Description: The Agent and the component create one semaphore that is cleaned up at shutdown. In case of unclean shutdown, the semaphores are deleted on the next server/Agent startup. This message is generated when the system has run out of semaphores or there are permission issues when creating the semaphore. Try increasing the semaphore limit on the machine.

Message:

MSBDirCreateFailed {Access Server: Could not create scoreboard file file name.}

Description: The system could not create the required directory for the scoreboard file, probably due to insufficient permissions.

12.9 Discrepancies Between Netstat and SNMP Values

When using the netstat command, the value returned for this command may not always match the information collected for the MIB variables:

aaaDirectoryServerNoOfLiveConnections 
coreidDirectoryServerNoOfLiveConnections

Table 12-11 explains the reason for this discrepancy and the chain of events that takes place

12.10 Configuring the Shutdown Interval

To ensure that an Identity or Access component can perform a clean shutdown, enough time must be allocated to ensure that all cleanup activities can be completed. For the Identity Server, the Access Server, and the SNMP Agent, the shutdown_time parameter specifies the time allocated for the server to attempt a clean shutdown. This parameter is located in globalparams.xml. The default shutdown time is five seconds.

The globalparams file location is as follows:

For Access Server:

AccessServer_install_dir/access/oblix/apps/common/bin/globalparams.xml

For Identity Server:

Identity_install_dir/identity/oblix/apps/common/bin/globalparams.xml

The default shutdown time appears as follows in these files:

shutdown_time:5

You can change the value to any time, specified in seconds.