# Configuration Guide

# Logging, Auditing, and Monitoring

This section describes how to monitor, log, and audit WLOC services and resources.

- Logging
- Auditing WLOC Actions
- Monitoring
- Viewing Events

## Logging

The WLOC Controller and each Agent generate log messages that provide information about events such as service deployments, action failures, and other events. These messages are saved in log files that, by default, are located in the `log` sub-directory where the Agent or Controller was installed.

In addition, the Controller and Agents output messages to standard out. You can filter this output by message severity. For example, you can configure that only messages of `WARNING` severity or higher are output to standard out.

### Configuring Logging

You can configure the following aspects of the logging feature:

Table 8-1 Logging Configuration
Field | Description
--- | ---
Severity | The severity of messages to write to the log file. See Message Severity.
Log File Name | Path name of the log file. The path can be absolute or relative to the Controller or Agent installation directory. To include a time and date stamp in the file name when the log file is rotated, add `java.text.SimpleDateFormat` variables to the file name. Surround each variable with percentage (%) characters. For example, if the file name is defined to be `myserver_%yyyy%_%MM%_%dd%_%hh%_%mm%.log`, the log file will be named `myserver_yyyy_mm_dd_hh_mm.log`. When the log file is rotated, the rotated file name contains the date stamp. For example, if the log file is rotated on 2 April, 2008 at 10:05 AM, the log file that contains the old messages will be named `myserver_2008_04_02_10_05.log`. If you do not include a time and date stamp, the rotated log files are numbered in order of creation. For example, `myserver.log00007`.
Rotation Type | Rotation style to use for rotating log files. Valid options include By Size and By Time. For more information about log file rotation, see Rotating Log Files.
Rotation Size | Maximum log file size (in KBs) before the current log file is rotated and a new log file is created. This value is valid only if the Rotation type was set to By Size. When the file size is reached, the file is renamed by incrementing the file extension (e.g., controller.log0001, controller.log0002, etc.) and saved in the rotation directory. The server then writes messages to a new file named filename.log. WLOC sets a threshold size limit of 500 MB before it forces a hard rotation to prevent excessive log file growth.
Rotation Directory | Directory in which log files are stored.
Number of Files Limited | Flag that specifies whether to limit the number of stored log files. (Requires that you specify a file rotation type of SIZE or TIME.) After WLOC reaches this limit, it deletes the oldest log file and creates a new log file with the latest suffix. If you do not enable this option, the server creates new files indefinitely and you must clean up these files as necessary.
Rotation File Count | Maximum number of log files to create before overwriting them. When the maximum number of log files is reached, the oldest file is overwritten. This value is valid only if the Number of files limited flag is enabled. This number does not include the file that the server uses to store current messages.
Rotation on Startup | Flag that specifies whether log files should be rotated automatically at startup.
Rotation Time | Time of day that log files are rotated initially. Specify using the format: `hh:mm`. This value is used only if the Rotation Type is set to `By Time`. If the time you specify has already elapsed, WLOC will perform the day’s rotation immediately.
Rotation Time Span | Frequency at which log files are rotated with Rotation Time Span Factor. Specify a number using the format: `hh:mm`. The default is 00:00 and equates to midnight. The actual rotation frequency is calculated as follows: `(Rotation Time Span) * (Rotation Time Span Factor)`. This value defaults to 24 and is valid only if the Rotation type is set to `By Time`.
Rotation Time Span Factor | Frequency at which log files are rotated with Rotation Time Span. The rotation frequency is calculated as follows: `(Rotation Time Span) * (Rotation Time Span Factor)`. This value is specified in milliseconds and defaults to 3600000. This value is valid only if the Rotation type is set to `By Time`.
Standard Out Severity | The severity of log file messages to write to standard out. See Output to Standard Out and Standard Error and Message Severity.

### Viewing Log Messages

Log messages can be viewed as follows:

- All current Controller log files can be viewed in the Administration Console by selecting the Controller’s Logs tab. All rotated log files can be examined only by opening the log files individually.

The default location of Controller logs is the `logs` subdirectory where the Controller was installed.

- All Agent log files must be examined by accessing the log files directly on the file system. Currently, they cannot be viewed in the WLOC Administration Console.

The default location of Agent logs is the `logs` subdirectory where the Agent was installed.

Note: | Do not modify log files by editing them manually. Modifying a file changes the timestamp and can confuse log file rotation. In addition, editing a file might lock it and prevent updates.

### Log Message Format

When a Controller or Agent writes a message to its log file, the first line of each message begins with `####` followed by the message attributes. Each attribute is contained between angle brackets.

The following is an example of a message in the Controller’s log file:

          ####<Oct 25, 2007 5:32:09 PM EDT> <Info> <LOCExecuteEngine> <> <>
          <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default
          (self-tuning)'> <> <> <> <1193347929031> <BEA-2010502> <Action
          Succeeded:EmailNotificationAction([WLS-Cluster,ID=7111863992976504417],
          WLS-Cluster,ID=-4438336769125385457) for event       com.bea.adaptive.execute.internal.SyntheticServiceEvent@63bb71.>

In this example, the message attributes are: Locale-formatted Timestamp, Severity, Subsystem, Machine Name, Server Name, Thread ID, User ID, Transaction ID, Diagnostic Context ID, Raw Time Value, Message ID, and Message Text. For information about how these attributes are used, see Log Message Attributes.

If the message includes a stack trace, the stack trace is included in the message text.

WLOC uses the host computer’s default character encoding for the messages it writes.

### Output to Standard Out and Standard Error

In addition to writing messages to log files, a Controller or Agent can print a subset of its messages to standard out. Usually, standard out is the shell (command prompt) in which you are running the Controller or Agent. However, some operating systems enable you to redirect standard out to some other location.

You can filter this output by message severity. For example, you can configure that only messages of `WARNING` severity or higher are output to standard out. You can also configure whether WLOC prints stack traces to standard out.

When a Controller or Agent writes a message to standard out, the output does not include the `####` prefix.

### Log Message Attributes

The log messages contain a consistent set of attributes as described in Table 8-2. In addition, if your application uses WebLogic logging services to generate messages, its messages will contain these attributes.

Table 8-2 WLOC Log Message Attributes
Attribute | Description
--- | ---
Locale-formatted Timestamp | Time and date when the message originated, in a format that is specific to the locale. The JVM that runs the Controller or each Agent refers to the host computer operating system for information about the local time zone and format.
Severity | Indicates the degree of impact or seriousness of the event reported by the message. See Message Severity.
Subsystem | Indicates the subsystem of WLOC that was the source of the message.
Thread ID | Identifies the origins of the message. The `Thread ID` is the ID that the JVM assigns to the thread in which the message originated.
User ID | The user ID under which the associated event was executed. To execute some pieces of internal code, WLOC authenticates the ID of the user who initiates the execution and then runs the code under a special Kernel Identity user ID.
Raw Time Value | The timestamp in milliseconds.
Message ID | A unique seven-digit identifier. All message IDs that WLOC generates start with `BEA-` and fall within a numerical range of 2010000 to 2019999.
Message Text | A description of the event or condition.

### Message Severity

The severity attribute of a WLOC log message indicates the potential impact of the event or condition that the message reports. Table 8-3 lists the severity levels of log messages by severity, from lowest to highest.

Table 8-3 Message Severity
Severity | Meaning
--- | ---
`TRACE` | A trace level event used for user-level debugging.
`DEBUG` | A debug message used for internal debugging.
`INFO` | A low-level informational message; used for reporting normal operations.
`NOTICE` | An informational message with a higher level of importance.
`WARNING` | A suspicious operation or configuration has occurred but it might not affect normal operation.
`ERROR` | A user error has occurred. The system or application can handle the error with no interruption and limited degradation of service.
`CRITICAL` | A system error has occurred. The system can recover but there might be a momentary loss or permanent degradation of service.
`ALERT` | A particular system is in an unusable state while other parts of the system continue to function. Automatic recovery is not possible; the immediate attention of the administrator is needed to resolve the problem.
`EMERGENCY` | The Controller or Agent is in an unusable state. This severity indicates a severe system failure or panic.

### Rotating Log Files

By default, the Controller and each Agent renames (rotates) its log file when the file grows to a size of 500 kilobytes. Each time the log file reaches this size, the WLOC renames the log file and creates a new `file-name``.log` to store new messages. By default, the rotated log files are numbered in order of creation `file-namennnnn`. You can configure WLOC to include a time and date stamp in the file name of rotated log files; for example, `file-name-%yyyy%-%mm%-%dd%-%hh%-%mm%`.`log`.

You can rotation file size, interval, and other properties based on the information in Table 8-1 and Table 8-4.

Some file systems place a lock on files that are open for reading. On such file systems, if your application is tailing the log file, or if you are using a command such as the DOS `tail -f` command in a command prompt, the tail operation stops after the server has rotated the log file. The `tail -f` command prints messages to standard out as lines are added to a file. For more information, enter `help tail` in a DOS prompt.

### Debug Log Messages

To help you and Oracle Customer Support diagnose problems with a WLOC environment, WLOC can output debug log messages that provide a detailed description of events in the runtime environment.

You can configure WLOC to output debug messages from all WLOC components or from specific components or scopes. WLOC writes debug messages to its log files and you can configure WLOC to print them to standard out.

## Auditing WLOC Actions

By default, the WLOC Audit Service is enabled and writes audit events to an audit log file. The default name of the file is `audit.log` located in the `logs` directory where the Controller or Agent was installed. The current Controller audit log is exposed in the Administration Console. The Controller’s rotated audit logs and all Agent logs must be examined by directly accessing the file.

You can configure the Audit Service in the Administration Console. For the Controller, this is performed on the Controller’s Audit tab. For an Agent, it is performed on the Audit tab of the individual Agent. You specify name and rotation settings for Audit logs using the same settings as those used for Controller logs. For a description of these fields, see Table 8-1. In addition, you can specify the audit types and whether or not to enable auditing as described in Table 8-4.

Table 8-4 Audit Service Configuration
Field | Description
--- | ---
Enabled | Flag that specifies whether the audit log service is enabled.
Audit Types | The category of events to audit. For an Agent, select ALL or AGENT_ACTION. For the Controller, select ALL or one or more of the following: CONTROLLER_CONFIGURATION SERVICE_CONFIGURATION RULES CONTROLLER_ACTION ADJUDICATION AGENT_CONFIGURATION For more information, see Audit Event Types.

### Audit Event Types

The WLOC Audit Service generates and records audit events as described in Table 8-5.

Table 8-5 Audit Event Types
Field | Description
--- | ---
ALL | Include all audit types.
CONTROLLER_CONFIGURATION | (Controller Only) Any changes to the configuration of the Controller.
AGENT_CONFIGURATION | (Controller Only) Any changes to the configuration of an Agent.
SERVICE_CONFIGURATION | (Controller Only) Any changes to the configuration of a service.
RULES | (Controller Only) Each time a rule evaluates to true, WLOC generates a Rule Audit Event containing the relevant context of the Rule execution.
CONTROLLER_ACTION | (Controller Only) An action initiated by a WLOC policy.
ADJUDICATION | (Controller Only) An adjudication decision (approves/deny).
AGENT_ACTION | (Agent Only) Any JVM lifecycle actions (such as stage/start/stop/destroy) initiated by the Agent.

### Audit Format

The WLOC Audit Service uses message identifiers numbered 2014100 through 2014199 for all audit entries. Audit records begin with the`####` marker and each field uses the `<` prefix and `>` suffix. Table 8-6 describes audit record fields.

Table 8-6 Audit Record Fields
Field | Description
--- | ---
Date | Message time and date in a format that is specific to the locale. The JVM that runs the Controller or each Agent refers to the host computer operating system for information about the local time zone and format. For example, Aug 20, 2007 2:11:24 PM EDT
Severity | The severity levels are INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY, DEBUG.
Subsystem | LOCAudit
Server Name | The Controller or Agent’s machine name.
Thread ID | ID that the JVM assigns to the thread in which the message originated.
Timestamp | A millisecond timestamp.
Audit Type | Always `<Audit>`. Also see Audit Event Types.
Message ID | A unique message identifier to enable easy retrieval and sorting for Audit events of a given type. The message ID range for WLOC is 2014100 through 2014199.
Message Body | Message text.

Listing 8-1 Example Audit Record

    ####<Aug 20, 2007 2:11:24 PM EDT> <Info> <LOCAudit> <seacoast1> <Thread-19> <1187633484747> <ControllerAction> <BEA-2013411> <Configured Pipeline configName with Pipeline Identifier PipeID successfully completed execution at time Aug 20, 2007 2:11:24 PM EDT. Execution elapsed time was 10,000 milliseconds.>

## Monitoring

For all active resources, the WLOC Administration Console provides flexible tools for designing and displaying charts and graphs. For active services, resource pools, JVMs, and MBean servers, you can specify a chart that displays:

- The amount of resources the service is using from a resource pool relative to the amount of resources available.
- Runtime statistics from each JVM within the service. The amount of information that is available depends on what the JVM provides.

Note: | WLOC does not support monitoring Open MBeans.

### Monitoring Service Performance

The Administration Console can display CPU and memory charts that indicate the performance of a service. For services managed by Plain Agents, you need to start up the JVM’s management server when you start the service’s processes. For information about how to do this, see JVM Arguments for Processes.

Note: | The service must be using a JRockit JVM.

For more information, see “ Monitor Resources” in the WLOC Administration Console Online Help.

## Viewing Events

By default, events are displayed at the bottom of the Administration Console page in the Task and Events viewer. This viewer displays up to the last 20 events that have occurred and is active no matter which tab is currently selected in the console. This viewer provides the same ability to filter messages that appear in it as provided on the Events page. You can also create custom viewing tabs with unique filters.

You can also can view events using the Events tab in the Console.

In addition, you may define Administrative policies that will cause the Console to generate console messages based on some constraint. For additional information, see Administrative Policies.