Skip Navigation Links
Exit Print View
| Oracle Solaris Cluster Error Messages Guide Oracle Solaris Cluster 4.1

Search Scope:

- This Document
- Entire Library

## Message IDs 600000–699999

This section contains message IDs 600000–699999.

`600170 scnas fatal error - scnas_validate_host() error %d`

Description: The program responsible for retrieving NAS configuration information from the CCR has suffered an internal error. Continued errors of this type may lead to a compromise in data integrity.

Solution: Contact your authorized Oracle service provider as soon as possible to determine whether a workaround or patch is available.

`600398 validate - file %s does not exist`

Description: The parameter file of option -N of the start, stop or probe command does not exist.

Solution: correct the filename and reregister the dataservice.

`600552 validate: User is not set but it is required`

Description: -2 The user is not set in the parameter file, or in the start stop or probe command.

Solution: -2 Set he user in the appropriate place. If it is from the master server use the parameter file, otherwise use the start, stop or probe command.

`600874 CCR: Can't access table %s (cluster %s) while updating it on node %s, errno = %d.`

Description: The indicated error occurred while updating the the indicated table on the indicated node. The errno value indicates the nature of the problem. errno values are defined in the file /usr/include/sys/errno.h. An errno value of 28 (ENOSPC) indicates that the root file system on the node is full. Other values of errno can be returned when the root disk has failed (EIO) or some of the CCR tables have been deleted outside the control of the cluster software (ENOENT).

Solution: There may be other related messages on the node where the failure occurred. These may help diagnose the problem. If the root file system is full on the node, then free up some space by removing unnecessary files. If the indicated table was accidently deleted, then boot the offending node in -x mode to restore the indicated table from other nodes in the cluster. The CCR tables are located at /etc/cluster/ccr/. If the root disk on the afflicted node has failed, then it needs to be replaced.

`600967 Could not allocate buffer for DBMS log messages: %m`

Description: Fault monitor could not allocate memory for reading RDBMS log file. As a result of this error, fault monitor will not scan errors from log file. However it will continue fault monitoring.

Solution: Check if system is low on memory. If problem persists, please stop and start the fault monitor.

`601609 Nameserver failed to register with cluster HA framework`

Description: The Nameserver repl server failed to register with the cluster HA framework.

Solution: This is an unrecoverable error, and the cluster needs to be rebooted. Also contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`601624 CMM: Quorum device %d (%s) with votecount = %d removed.`

Description: The specified quorum device with the specified votecount has been removed from the cluster. A quorum device being placed in maintenance state is equivalent to it being removed from the quorum subsystem's perspective, so this message will be logged when a quorum device is put in maintenance state as well as when it is actually removed.

Solution: This is an informational message, no user action is needed.

`601901 Failed to retrieve the resource property %s for %s: %s.`

Description: The query for a property failed. The reason for the failure is given in the message.

Solution: Save a copy of the /var/adm/messages files on all nodes. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`601905 Door arguments invalid, request ignored.`

Description: An event listener client sent bad door call arguments to the proxy event listener daemon.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`602480 dismount_unconnected_asm_diskgroup - Oracle ASM instance (%s) has dismounted diskgroup (%s)`

Description: The Oracle ASM diskgroup has been dismounted.

Solution: None required. Informational message.

`602793 oep_validate - /opt/ORCLscoep/.%s_passwd does not exist`

Description: /opt/ORCLscoep/.${RESOURCE}_passwd does not exist.

Solution: Ensure that /opt/ORCLscoep/.${RESOURCE}_passwd exists. Refer to the Oracle Solaris Cluster for Oracle External Proxy Data Service document for more information. Otherwise, contact Oracle support services for further assistance.

`603092 An unknown value for the RG_mode property was found for resource group %s.`

Description: The RG_mode of the specified resource group is neither Failover or Scalable.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`603096 resource %s disabled.`

Description: This is a notification from the rgmd that the operator has disabled a resource. This message can be used by system monitoring tools.

Solution: This is an informational message; no user action is needed.

`603149 Error reading file %s`

Description: Specified file could not be read or opened.

Solution: Examine other syslog messages occurring around the same time on the same node, to see if the source of the problem can be identified.

`603192 The resource type version of resource %s is NULL.`

Description: The HAStoragePlus resource type version of the specified resource was found to be NULL.

Solution: Check the cluster configuration. If the problem persists, contact your authorized Oracle service provider.

`603289 Could not bring the affinity source resource group %s offline on node %s`

Description: Failed to bring the inter-cluster affinity source resource group offline on the given node. Possible cause could be that the remote zone cluster is down.

Solution: No action required.

`603913 last probe failed, Tomcat considered as unavailable`

Description: the last sanity check was unsuccessful, may be out of sessions

Solution: none required, it is recommended to observe Tomcats number of configured sessions

`604016 Failed to open contract ctl file %s: %s`

Description: Cannot open the contract control file.

Solution: Check the contract(4) man page to learn more about the error. Also make sure the basic contract functionalities are working correctly. Contact your Oracle service provider for more information.

`604153 clcomm: Path %s errors during initiation`

Description: Communication could not be established over the path. The interconnect might have failed or the remote node might be down.

Solution: Any interconnect failure should be resolved, and/or the failed node rebooted.

`604333 The FilesystemCheckCommand extension property is set to %s.`

Description: Self explanatory.

Solution: This is an informational message, no user action is needed.

`604431 Wrong number of input arguments = %d`

Description: Incorrect arguments are passed to fedwrapper, which should never happen.

Solution: Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing the problem.

`604841 %s - The user %s does not belongs to project %s`

Description: The specified user does not belong to the project defined by Resource_project_name or Rg_project_name.

Solution: Add the user to the defined project in /etc/project.

`604889 PNM proxy : Unable to get a reference to the version manager`

Description: PNM proxy got a nil reference to the Version manager and hence failed to retrieve the version.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`605136 Command request "%s" came in on the public network (%s), rather than the private (%s)`

Description: The rpc.scrcmd daemon received an illegal request to over the public network to execute a command. Command requests of this type are expected over the private network only.

Solution: Report this problem to you Oracle service representative. It could be that an internal error in the Oracle Solaris Cluster code has neglected to send the command request over the correct network.

`605200 Received a connect request from a node not configured in the cluster. Nodeid %u ipaddr %s`

Description: CCR tables are temporarily out of sync.

Solution: Check the error message and try again. Contact your authorized Oracle service provider for assistance.

`605502 Validate - DHCP is not enabled (DAEMON_ENABLED)`

Description: The DHCP resource requires that that the /etc/inet/dhcpsvc.conf file has DAEMON_ENABLED=TRUE.

Solution: Ensure that /etc/inet/dhcpsvc.conf has DAEMON_ENABLED=TRUE by configuring DHCP appropriately, i.e. as defined within the Oracle Solaris Cluster Data Service for DHCP.

`605879 Validate - CON_LIMIT=%s is incorrect, default CON_LIMIT=50 is being used`

Description: The value specified for CON_LIMIT is invalid.

Solution: Either accept the default value of CON_LIMIT=50 or change the value of CON_LIMIT to be less than or equal to 100 when registering the resource. CON_LIMIT=50 will now be used.

`606203 Couldn't get the root vnode: error (%d)`

Description: The file system is corrupt or was not mounted correctly.

Solution: Run fsck, and mount the affected file system again.

`606346 uname failed with error %d`

Description: The PNM daemon encountered an error with the uname function. The message contains the error code returned by the uname function.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`606362 The stop command <%s> failed to stop the application. Will now use SIGKILL to stop the application.`

Description: The stop command was unable to stop the application. The STOP method will now stop the application by sending it SIGKILL.

Solution: Look at application and system logs for the cause of the failure.

`606429 start_ohs - The start command (%s) completed successfully.`

Description: The Oracle HTTP Server started successfully.

Solution: None.

`606649 Failed to retrieve the cluster '%s' information. Retrying...`

Description: HAStoragePlus was not able to get the cluster configuration, but it will retry automatically.

Solution: This is an informational message, no user action is needed.

`606758 SCSLM <%s> pool_value_alloc error <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`607452 cl_execd: memory allocation failed size = %d`

Description: Could not allocate memory. Node is too low on memory.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`607475 validate_common - The directory %s for the configured Psft_Domain %s either does not exist or is not a directory`

Description: The directory for the configured Psft_Domain either does not exist or is not a directory.

Solution: Ensure that Psft_Domain contains the correct domain name and ensure that it got configured.

`607541 Generic_Affinity and UDP_Affinity are mutually exclusive features.`

Description: Generic_Affinity and UDP_Affinity are mutually exclusive features.

Solution: Enable either Generic_Affinity or UDP_Affinity for the resource.

`607678 clconf: No valid quorum_resv_key field for node %u`

Description: Found the quorum_resv_key field being incorrect while converting the quorum configuration information into quorum table.

Solution: Check the quorum configuration information.

`607726 sysevent_subscribe_event(): %s`

Description: The cl_apid or cl_eventd was unable to create the channel by which it receives sysevent messages. It will exit.

Solution: Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`608113 syncsa_serverd Unable to retrieve version from version manager: %s`

Description: The syncsa daemon got a nil reference to the Version manager and hence failed to retrieve the version.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`608202 scha_control: resource group <%s> was frozen on Global_resources_used within the past %d seconds; exiting`

Description: A scha_control call has failed with a SCHA_ERR_CHECKS error because the resource group has a non-null Global_resources_used property, and a global device group was failing over within the indicated recent time interval. The resource fault probe is presumed to have failed because of the temporary unavailability of the device group. A properly-written resource monitor, upon getting the SCHA_ERR_CHECKS error code from a scha_control call, should sleep for awhile and restart its probes.

Solution: No user action is required. Either the resource should become healthy again after the device group is back online, or a subsequent scha_control call should succeed in failing over the resource group to a new master.

`608270 validate_instance - %s You must set the clustered SAP instance Instance_number per-node extension_property for the node that was just added before you bring online the resource group on the newly added node`

Description: The SAP instance Instance_number per-node extension property needs to be defined.

Solution: When adding a node to the resource group, the SAP instance Instance_number per-node extension property is set to the default value, which might nott be valid for this deployment. Instead, if you required a different value from the default value, set the SAP instance Instance_number per-node extension property for the newly added node. To set the SAP instance Instance_number extension property, use the following command replacing <added-node>, <sid> and <resource> as required. clrs set -p 'Instance_number{<added-node>}'=<instance_number> <resource>

`608414 The %s value should be greater than %d: Hence ignoring this value from %s file at line %d.`

Description: The zone handler detected a bad value for the specified tunable in the specified configuration file.

Solution: Fix the specified tunable value so that it meets the restrictions.

`608749 Failed to retrieve the resource handle of resource %s: %s.`

Description: There was a failure while retrieving to retrieve a handle of the named resource. The reason that the error occurred is also provided in the message.

Solution: The operation will be retried after some time. But there might be a delay in failing over some HA-NFS resources. If this message is logged repeatedly, save a copy of the syslog and contact your authorized Oracle service provider.

`608876 PCRUN: %s`

Description: The rpc.pmfd server was not able to monitor a process, and the system error is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`609118 Error creating deleted directory: error (%d)`

Description: While mounting this file system, PXFS was unable to create some directories that it reserves for internal use.

Solution: If the error is 28(ENOSPC), then mount this FS non-globally, make some space, and then mount it globally. If there is some other error, and you are unable to correct it, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`609401 The %s extension property is not set.`

Description: While validating the extension properties, The HADBM_PASSWORDFILE extension property is found not to be set.

Solution: This is an informational message, the user has to provide the complete path to the hadbm admin password file if the domain is created with a password.

`609697 Function: validate: The Host property must specify an existing host`

Description: The Host property must be set to an existing host.

Solution: Specify the Host property to an existing host.

`609730 clcomm: failed to create thread for monitor`

Description: The system could not create the needed thread, because there is inadequate memory.

Solution: There are two possible solutions. Install more memory. Alternatively, reduce memory usage. Since this happens during system startup, application memory usage is normally not a factor.

`609889 UNRECOVERABLE ERROR: %s file not present.`

Description: /usr/cluster/lib/scadmin/ql/upgrade_partition file not present.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`610006 No entry for file-system %s at mountpoint %s found in /etc/vfstab.`

Description: The entry for the specified shared QFS file system is missing from the /etc/vfstab file.

Solution: Add a valid entry for this file system to /etc/vfstab and repeat the operation.

`610178 get_property - failed to retrieve system property %s.`

Description: Internal error - Failed to retrieve a system property for a resource.

Solution: No user action is possible.

`610273 IPMP group %s has failed, so scalable resource %s in resource group %s may not be able to respond to client requests. A request will be issued to relocate resource %s off of this node.`

Description: The named IPMP group has failed, so the node may not be able to respond to client requests. It would be desirable to move the resource to another node that has functioning IPMP groups. A request will be issued on behalf of this resource to relocate the resource to another node.

Solution: Check the status of the IPMP group on the node. Try to fix the adapters in the IPMP group.

`610633 %s: problem waiting for the daemon, read errno %d`

Description: Internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`610960 Failed to get the list of disks from CCR, line %d`

Description: Internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`611676 ct_pr_tmpl_set_param: %s`

Description: The libclcontract code was unable to configure a process template in which to launch the requested service. Consequently, the process that is using libclcontract was unable to launch the process under its control.

Solution: If the message is isolated, it was probably a transient error. Retry the requested action. If there are many instances of the message, there is a problem with the contract file system. Search for other syslog error messages on the same node. Save a copy of the /var/adm/messages files on all nodes, and report the problem to your authorized Oracle service provider.

`611732 statvfs failed on the mount point %s for zone cluster %s node : %s`

Description: The file system specified on the mount point is not available for zone cluster.

Solution: Check the statvfs(2) man page for errors and try to resolve the problem. Otherwise contact your authorized Oracle service provider.

`611913 Updating configuration information.`

Description: The HADB START method is updating it's configuration information.

Solution: This is an informational message, no user action is needed.

`612124 Volume configuration daemon not running.`

Description: Volume manager is not running.

Solution: Bring up the volume manager.

`612135 validate_asm - Deploying Oracle ASM 11gR2 or later requires Oracle Grid Infrastructure for Clusters and resource type SUNW.scalable_asm_instance_proxy`

Description: Oracle 11.2.0.1.0 or later requires Oracle Grid Infrastructure for Cluster and resource type SUNW.scalable_asm_instance_proxy.

Solution: Ensure that Oracle Grid Infrastructure for Cluster has been successfully installed and that SUNW.scalable_asm_instance_proxy is used.

`612436 Method <%s> on resource <%s>: Execution failed: SLM error`

Description: Resource group is under SC SLM control and an SLM error occurred. Some errors might be configuration errors. Check fed SLM errors for more details.

Solution: Move RG_SLM_type to manual and restart the resource group.

`612562 Error on line %ld`

Description: Indicates the line number on which the error was detected.

Solution: Please ensure that all entries in the custom monitor action file are valid and follow the correct syntax. After the file is corrected, validate it again to verify the syntax.

`612689 Failed to register configuration callbacks for cznetd daemon`

Description: The cznetd daemon was unable to register for configuration callbacks.

Solution: These callbacks are used only for enabling or disabling the private IP communication for zone clusters. So, this feature will be unavailable. To recover, it might be necessary to reboot this node or the entire cluster. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`612819 reservation warning(%s) - MHIOCGRP_PREEMPTANDABORT error(%d) will retry in %d seconds`

Description: The device fencing program has encountered errors while trying to access a device. The failed operation will be retried

Solution: This is an informational message, no user action is needed.

`612931 Unable to get device major number for %s driver : %s.`

Description: System was unable to translate the given driver name into device major number.

Solution: Check whether the /etc/name_to_major file is corrupted. Reboot the node if problem persists.

`613200 Function: check_ndbd: The MySQL Cluster data node id=%s is starting`

Description: The connection to the MySQL Cluster management server was possible, but the requested data node is started.

Solution: none

`613522 clexecd: Error %d from poll. Exiting.`

Description: clexecd program has encountered a failed poll(2) system call. The error message indicates the error number for the failure.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`613814 SCSLM <%s> zone <%s> no users`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`613966 Error: attempted to run this script in global zone`

Description: The svc:/system/cluster/sc_ng_zones:default SMF service should run only in non-global zones.

Solution: Disable the svc:/system/cluster/sc_ng_zones:default service in the global zone.

`613984 scha_control: request failed because the given resource group <%s> does not contain the given resource <%s>`

Description: A resource monitor (or some other program) is attempting to initiate a restart or failover on the indicated resource and group by calling scha_control(1ha),(3ha). However, the indicated resource group does not contain the indicated resource, so the request is rejected. This represents a bug in the calling program.

Solution: The resource group may be restarted manually on the same node or switched to another node by using clresourcegroup or the equivalent GUI command. Contact the author of the data service (or of whatever program is attempting to call scha_control) and report the error.

`614203 Perform_takeover - No takeover was performed, because the real MySQL resource %s is not online on this node`

Description: No takeover performed, because the MySQL resource is not online on this node

Solution: None

`614266 SVM is not properly installed, %s not found.`

Description: Could not find the program that is required to support Solaris Volume Manager with Oracle Solaris Cluster.

Solution: Verify the installation of Solaris Volume Manager and the Solaris version. Refer to the Oracle Solaris Cluster documentation for installation procedures. If the problem persists, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`614310 Function: validate - SLrunlevel not set in %s`

Description: The SLrunlevel variable is not set in the parameter file sczbt_<resource name>.

Solution: Set the SLrunlevel variable in the parameter file sczbt_<resource name>.

`614373 Membership: Unable to create CCR callback thread for membership subsystem. Zone cluster membership support would not be available fully`

Description: A thread that supports dynamic addition/removal of zone clusters to the membership subsystem could not be created. Membership for dynamically added or removed zone clusters would not work properly. This might be due to lack of memory.

Solution: Lack of memory might lead to other problems on the node. You must free up memory on the node.

`614556 sema_post child: %s`

Description: The rpc.pmfd server was not able to act on a semaphore. The message contains the system error. The server does not perform the action requested by the client, and an error message is output to syslog.

Solution: Determine if the machine is running out of memory. If this is not the case, save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`614872 Create_configuration - MySQL replication resource group %s contains a MySQL database resource`

Description: The MySQL replication resource group contains a MySQL database resource already

Solution: Specify a replication resource group without MySQL database resources

`615120 fatal: unknown scheduling class '%s'`

Description: An internal error has occurred. The daemon indicated in the message tag (rgmd or ucmmd) will produce a core file and will force the node to halt or reboot to avoid the possibility of data corruption.

Solution: Save a copy of the /var/adm/messages files on all nodes, and of the core file generated by the daemon. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`615207 %s: ct_pr_event_get_exitstatus: %s`

Description: The pmmd was unable to determine the exit status of a process under its control that exited. It will assume failure.

Solution: Search for other syslog error messages on the same node. Save a copy of the /var/adm/messages files on all nodes, and report the problem to your authorized Oracle service provider.

`616976 stop_ndbd_shutdown: The restart -n of all data nodes was unsuccesful, because the mgm server is unreachable, retry it in 5 seconds`

Description: The restart -n of the datanode was unsuccessful, the managemnt server is unreachable.

Solution: None

`617169 Error while unbinding '%s' in the name server. Exiting.`

Description: The cl_execd program could not unbind the corba object from the global name server.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`617385 readdir_r: %s`

Description: An error occurred when rpc.pmfd tried to use the readdir_r function. The reason for the failure is also given.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`617602 The SUNW.qfs resource '%s' is not online on this node. Skipping this resource ...`

Description: The specified SUNW.qfs resource is not online on this node. That means the MDS server(s) of shared qfs file systems of this resource will not be running on this node. Hence skipping the resource.

Solution: This is an informational message; no user action is needed.

`617716 SCSLM zone <%s> pool configuration error, pool <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`617876 clcomm: getzonenamebyid failed: %s`

Description: The getzonenamebyid(3C) operation failed. The man page for getzonenamebyid describes possible error codes.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`617917 Initialization failed. Invalid command line %s %s`

Description: Unable to process parameters passed to the call back method. This is an internal error.

Solution: Please report this problem.

`618032 start_asmproxy - Startup of asm_proxy_daemon failed for Oracle ASM instance (%s)`

Description: The monitor for type SUNW.scalable_asm_proxy failed to start.

Solution: Contact your Oracle support representative for further assistance.

`618075 sig_handler sigfillset: %s`

Description: Internal error.

Solution: Save the syslog messages file. Contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`618107 Path %s initiation encountered errors, errno = %d. Remote node may be down or unreachable through this path.`

Description: Communication with another node could not be established over the path.

Solution: Any interconnect failure should be resolved, and/or the failed node rebooted.

`618236 cvm_probe_once: Could not get device group info for (%s)`

Description: An error occurred during the retrieval of information about the disk set or disk group.

Solution: Check the status of the disk set or disk group. Contact your authorized Oracle service provider for further assistance in diagnosing the problem.

`618585 clexecd: getmsg returned %d. Exiting.`

Description: clexecd program has encountered a failed getmsg(2) system call. The error message indicates the error number for the failure.

Solution: The clexecd program will exit and the node will be halted or rebooted to prevent data corruption. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`618709 Failed to generate callback for add adapter event`

Description: Cluster cannot generate a callback for the RDS module. Operation of RDS sockets might be affected.

Solution: If Reliable Datagram Sockets (RDS) is used by applications running on this cluster, contact your Oracle service representative to determine if a workaround or patch is available.

`618764 fe_set_env_vars() failed for Resource <%s>, resource group <%s>, method <%s>`

Description: The rgmd was unable to set up environment variables for a method execution, causing the method invocation to fail. Depending on which method was being invoked and the Failover_mode setting on the resource, this might cause the resource group to fail over or move to an error state.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`619059 sigaction(2) failed for pmmd_adm while querying disposition for signal SIG%s: error %s`

Description: pmmd_adm was unable to query the signal disposition for a signal number because the sigaction(2) function failed. The message contains the system error. pmmd_adm cannot do its requested action.

Solution: Save a copy of the /var/adm/messages files on this node, and report the problem to your authorized Oracle service provider.

`619178 The hostname in the uri %s does not match any of the network resource dependencies.`

Description: The hostname in the URI of Monitor_uri_list property does not match any of the dependent network resource hostnames.

Solution: Make sure the URI of Monitor_uri_list property contains a valid hostname.

`619213 t_alloc (recv_request) failed with error %d`

Description: Call to t_alloc() failed. The "t_alloc" man page describes possible error codes. ucmmd will exit and the node will abort.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`619485 failed to get handle for zone %s: %s.`

Description: System has failed to get handle for zone.

Solution: For specific error info, check the syslog message.

`619986 Invalid zone cluster id %d passed for zone %s`

Description: Zone cluster id information that has been passed does not match the client credentials.

Solution: Someone has tampered with the Oracle Solaris Cluster code. This is a security violation. Shut down the identified zone.

`620007 UCMM is not in a stable state; can not continue IO probing !`

Description: Because the UCMM is in an unstable state, the probe cannot continue.

Solution: Check the state of the UCMM and, if required, restart the UCMM and repeat the operation.

`620193 Validation failed.`

Description: Validation of the ScalMountPoint resource failed.

Solution: Look for specific errors from the function 'validate_mountpoint_rs()' that caused validation to fail. Correct the error and repeat the operation.

`620333 asm_proxy_daemon_read_pipe - the named pipe (%s) does not exist`

Description: The named pipe does not exist.

Solution: None required. The named pipe will be recreated.

`620599 The metadata server of configured SUNW.qfs resources either moved to other node or this node is acting as metadata server being last client of shared qfs file systems. Zone halt operation completed successfully.`

Description: The zone halt handler completed its operation successfully.

Solution: This is an informational message; no user action is needed.

`621212 [PID %d] sigprocmask(2) failed : error %s`

Description: sigprocmask(2) failed in the specified process.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`621426 %s is not an ordinary file.`

Description: The path listed in the message is not an ordinary file.

Solution: Specify an ordinary file path for the extension property.

`621667 Method <%s> on resource <%s> is unkillable and the failover_mode is set to soft, will not abort the node <%s>.`

Description: The specified callback method for the specified resource is stuck in the kernel, and could not be killed with a SIGKILL. If the failover_mode for the resource was set to HARD, rgmd would have rebooted the node. But the failover_mode for the resource is not set to HARD. So, the node is not being rebooted.

Solution: None. This is normal behavior of the RGM. Other syslog messages that occured just before this might indicate the cause of the method failure.

`621686 CCR: Invalid checksum length %d in table %s, expected %d.`

Description: The checksum of the indicated table has a wrong size. This causes the consistency check of the indicated table to fail.

Solution: Boot the offending node in -x mode to restore the indicated table from backup or other nodes in the cluster. The CCR tables are located at /etc/cluster/ccr/.

`622011 SCSLM exec <%s> strlen is zero`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`622352 Could not kill the derby data service monitor`

Description: The monitor program of the derby data service could not be stopped. This should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`622570 validate_zcboot_resource - Oracle Grid Infrastructure resource %s is not registered.`

Description: The Oracle Grid Infrastructure resource sun.<resource> is not registered.

Solution: You must create the Oracle Grid Infrastructure resource sun.<resource> of type sun.zcboot_proxy.type, where <resource> represents the Oracle Solaris Cluster SUNW.wait_zc_boot resource name. Refer the Oracle Solaris Cluster documentation for information on how to do this.

`622945 fork_event: ct_pr_event_get_pid: %s`

Description: The rpc.pmfd was unable to retrieve the pid from a contract event. The rpc.pmfd will continue to monitor the process, but the rpc.pmfd may have missed an event of interest.

Solution: Search for other syslog error messages on the same node. Save a copy of the /var/adm/messages files on all nodes, and report the problem to your authorized Oracle service provider.

`622960 Failed to lookup command log for zone cluster %s`

Description: The CCR raised an exception while looking up the command log table.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`623159 %s: Hostname not set.`

Description: The hostname of the cluster node has not been set.

Solution: Check if the hostname has been set for the node. Contact your authorized Oracle service provider for further assistance.

`623528 clcomm: Unregister of adapter state proxy failed`

Description: The system failed to unregister an adapter state proxy.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`623635 Warning: Failed to configure client affinity for group %s: %s`

Description: A call to the underlying scalable networking code failed.

Solution: Save a copy of the /var/adm/messages files on all nodes. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`623669 Internal Error. Failed to check status of SUNW.HAStoragePlus resource.`

Description: This is an internal error during SUNW.HAStoragePlus resource dependencies status validation.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`623785 Warning: Failed to update the /etc/inet/hosts file.`

Description: This service has failed to update the /etc/inet/hosts file on zone cluster.

Solution: Examine if the IP of the zone is the loopback or localhost setting in /etc/inet/hosts file. If so, manually remove the zone from such setting. You can also add an entry with the correct IP for the zone.

`624447 fatal: sigaction: %s (UNIX errno %d)`

Description: The rgmd has failed to initialize signal handlers by a call to sigaction(2). The error message indicates the reason for the failure. The rgmd will produce a core file and will force the node to halt or reboot to avoid the possibility of data corruption.

Solution: Save a copy of the /var/adm/messages files on all nodes, and of the rgmd core file. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`624484 metastat output is empty for %s/%s.`

Description: Output from metastat command is empty. Either the metaset is empty or the node is not the owner of the metaset.

Solution: Determine why the node is not hosting the disk set or disk group. Examine the output from metaset. Determine whether the SUNW.vucmm_framework resource and either the SUNW.vucmm_svm resource or the SUNW.vucmm_cvm resource are online. Or in legacy mode determine whether the SUNW.rac_framework resource and either the SUNW.rac_svm resource or the SUNW.rac_cvm resource are online

`624617 Tag %s: could not get retry count`

Description: The rpc.pmfd server was not able to allocate memory for the history of the tag shown. Monitoring of the process associated with the tag is stopped and pmfadm returns an error. This problem can occur if the machine has low memory.

Solution: Determine if the machine is running out of memory. If this is not the case, save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`624622 Notifying cluster that this node is panicking`

Description: The node is notifying the other remaining cluster nodes that the node is going down due to a system panic.

Solution: No action required.

`625493 Error: ${PROCESS} is not running.`

Description: The initfed init script found that the rpc.pmfd is not running. The rpc.fed will not be started, which will prevent the rgmd from starting, and which will prevent this node from participating as a full member of the cluster.

Solution: Examine other syslog messages occurring at about the same time to determine why the rpc.pmfd is not running. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`625862 ct_event_read error'd out: %d`

Description: Not able to read the event from the file descriptor obtained from the pbundle.

Solution: Check the contract(4) man page to learn more about the error. Also make sure the basic contract functionalities are working correctly. Contact your Oracle service provider for more information.

`626478 stat of file %s failed: <%s>.`

Description: There was a failure to stat the specified file.

Solution: Make sure that the file exists.

`626904 cl_execd: Attempt to set scheduling class to Fair share returned %d. Exiting.`

Description: cl_execd program failed to set the priority and the scheduling class to Fair share for the task in execution.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`627169 Function: val_parfile - Directory %s does not exist or is not a directory`

Description: The parameter file directory you specified in the config file does not exist.

Solution: Fix the appropriate config file, either sczbt_config or sczsh_config with an existing directory.

`627479 validate_diskgroup - Oracle ASM diskgroup (%s) is already defined by %s`

Description: The Oracle ASM diskgroup specified with ASM_DISKGROUPS is already specified by another Oracle Solaris Cluster resource.

Solution: Ensure the correct Oracle ASM disk group was specified within the Oracle Solaris Cluster ASM_DISKGROUPS extension property and that the Oracle ASM disk group is not already defined by another Oracle Solaris Cluster resource.

`627610 clconf: Invalid clconf_obj type`

Description: An invalid clconf_obj type has been encountered while converting an clconf_obj type to group name. Valid objtypes are "CL_CLUSTER", "CL_NODE", "CL_ADAPTER", "CL_PORT", "CL_BLACKBOX", "CL_CABLE", "CL_QUORUM_DEVICE".

Solution: This is an unrecoverable error, and the cluster needs to be rebooted. Also contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`627736 Error while unpacking data: %s`

Description: An internal error has occurred in the inter-process communication between Oracle Solaris Cluster processes. Related error messages might be found near this one in the syslog output.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`627890 Unhandled error from probe command %s: %d`

Description: The data service detected an internal error.

Solution: Informational message. No user action is needed.

`628075 Start of HADB node %d completed successfully.`

Description: The HADB resource successfully started the specified node.

Solution: This is an informational message, no user action is needed.

`628155 DB_name, %s, does not match last component of Confdir_list, %s.`

Description: The db_name extenstion property does not match the database name in the confdir_list path.

Solution: Check to see if the db_name and confdir_list extension properties are correct.

`628473 Verify SVM installation. metaclust binary %s not found`

Description: Could not find the program that is required to support Solaris Volume Manager with Oracle Solaris Cluster.

Solution: Verify installation of Solaris Volume Manager and updates. Refer to the Solaris OS installation documentation. If the problem persists, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`628545 %s:error in method %s: %d`

Description: Problem in the execution of the smf method.

Solution: The waitpid, which was waiting on the completion of the process, exited because of a problem. The error will describe the reason. Contact your Oracle service provider for more information.

`628771 CCR: Can't read CCR metadata.`

Description: Reading the CCR metadata failed on this node during the CCR data server initialization.

Solution: There may be other related messages on this node, which may help diagnose the problem. For example: If the root disk on the afflicted node has failed, then it needs to be replaced. If the cluster repository is corrupted, then boot this node in -x mode to restore the cluster repository from backup or other nodes in the cluster. The cluster repository is located at /etc/cluster/ccr/. Contact your authorized Oracle service provider for assistance.

`628803 Error: The existing RAC framework resource version "%s" is less than the minimum acceptible version "%s".`

Description: The existing RAC framework resource version indicates that it is not VUCMM aware.

Solution: Update the existing RAC framework resource version.

`629085 Validation failed. Resource group property RG_AFFINITIES should specify a SCALABLE resource group containing the RAC framework resource.`

Description: The resource being created or modified must belong to a resource group that has a strong positive affinity with the scalable resource group containing the RAC framework resource.

Solution: If not already created, create the scalable resource group containing the RAC framework resource and its associated resources. Specify a strong positive (++) affinity with the scalable resource group containing the RAC framework resource by using RG_AFFINITIES property of the resource group.

`629355 None of the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Skipping the checks for the existence and permissions of the start/stop/probe commands.`

Description: Some checks are only performed when the SUNW.HAStoragePlus resource is online on the local node.

Solution: This message is informational; no user action is needed.

`629584 pthread_mutex_init: %s`

Description: The cl_apid was unable to initialize a synchronization object, so it was not able to start-up. The error message is specified.

Solution: Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`630096 Check_switchover - the check for switchover failed because of replication issues on the MySQL resource %s`

Description: The check for the replication status failed

Solution: Fix the replication issues

`630106 Exporting zpool " %s " has succeeded after %d seconds`

Description: This is an informational message that HAStoragePlus has succeeded to export the specified ZFS pool.

Solution: No user action is needed.

`630179 Not attempting to start resource group <%s> on node <%s>, because the resource groups <%s> for which it has strong positive affinities are not online.`

Description: The rgmd is enforcing the strong positive affinities of the resource groups. This behavior is normal and expected.

Solution: No action required. If desired, use clresourcegroup to change the resource group affinities.

`631373 Fatal error; aborting the rpc.fed daemon.`

Description: The rpc.fed server experienced an unrecoverable error, and is aborting the node.

Solution: Save the syslog messages file. Examine other syslog messages occurring around the same time on the same node, to see if the cause of the problem can be identified. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`631408 PCSET: %s`

Description: The rpc.pmfd server was not able to monitor a process, and the system error is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`631636 Oracle Solaris Cluster is waiting for lock on devicei %s. Lock is currently held by %s for fsck/mount.`

Description: Oracle Solaris Cluster boot is waiting for the mentioned node to complete fsck/mount of a global filesystem and release the lock on a device.

Solution: Check the console of the specified cluster node to see if any of the nodes are waiting for a manual fsck to be done. If this is so, exiting the shell after performing the fsck will allow the boot of the other nodes to continue.

`631648 Retrying to retrieve the resource group information.`

Description: An update to cluster configuration occured while resource group properties were being retrieved

Solution: Ignore the message.

`631859 IO to file-system %s, through mountpoint %s failed.`

Description: The file system is in a bad state and requires maintenance.

Solution: Check the condition of the file system and restart the ScalMountPoint resource.

`631884 Ifconfig proxy: Could not initialize the ORB. Exiting.`

Description: The ifconfig proxy program was unable to initialize its interface to the low-level cluster machinery.

Solution: Make sure the nodes are booted in cluster mode. If so, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`632027 Not stopping HADB node %d which is in state %s`

Description: The specified HADB node is already in a non running state and does not need to be stopped.

Solution: This is an informational message, no user action is needed.

`632156 start_diskgroup - Oracle ASM instance (%s) failed to mount diskgroup (%s)`

Description: The Oracle ASM instance failed to mount the diskgroup.

Solution: Check the syslog for further messages. Oracle Solaris Cluster Will attempt to restart the resource which will attempt to mount the disgkgroup again on this zone or another zone.

`632337 cl_execd: dup2 of stdin returned with errno %d while executing (%s). Exiting.`

Description: cl_execd program has encountered a failed dup2(2) system call. The error message indicates the error number for the failure.

Solution: The cl_execd program will exit and the node will be halted to prevent data corruption. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`632391 zc_member(%d): Could not get zone name. Exiting.`

Description: Failed to get the current zonename. Support for the current zone cluster might not work properly.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`632435 Error: attempting to copy larger list to smaller`

Description: The cl_apid experienced an internal error.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`633010 sv_get_zone_iptype: Could not get zone handle for zone %s,error = %d`

Description: During initialization, there was a failure to get information about a specific zone in the system.

Solution: This is an informational message. No user action is required.

`633024 Invalid service FMRI or manifest file provided in the file provided for the proxied_service_instances property`

Description: The file that is provided for the "proxied_service_instances" property has invalid entries.

Solution: Please check the entries for the service FMRI and/or path to the manifest file in the file that is provided for the "proxied_service_instances" property.

`633745 pthread_kill: %s`

Description: The rpc.fed server encountered an error with the pthread_kill function. The message contains the system error.

Solution: Save the syslog messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`634138 validate_common - %s/bin/sqlplus either does not exist or is not executable`

Description: ${ORACLE_HOME}/bin/sqlplus either does not exist or is not executable.

Solution: Ensure that ${ORACLE_HOME} represents the directory where the Oracle files are installed.

`634250 nfsd daemon is dead. It will not be restarted as Failover_mode is set to LOG_ONLY.`

Description: HA-NFS probe has determined that one of the daemon is dead. However, since the Failover_mode property is set to LOG_ONLY, the daemon will NOT be restarted.

Solution: This is an informational message, no action is needed.

`634333 Service group '%s:%s' deleted`

Description: The service group by that name is no longer known by the scalable services framework.

Solution: This is an informational message, no user action is needed.

`634437 Function: validate - Global zones mountpoint %s does not exist`

Description: The directory used as a mountpoint in the global zone is missing.

Solution: Create the mountpoint in the global zone.

`634576 SCSLM open <%s> error <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`634693 Waiting for the PMF to stop`

Description: The PMF service is being stopped.

Solution: Please wait. No action is required. This is an informational message.

`634957 thr_keycreate failed in init_signal_handlers`

Description: The ucmmd failed in a call to thr_keycreate(3T). ucmmd will produce a core file and will force the node to halt or reboot to avoid the possibility of data corruption.

Solution: Save a copy of the /var/adm/messages files on all nodes and of the ucmmd core. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`635163 scdidadm: ftw(/dev/rmt) failed attempting to discover new paths: %s, ignoring it`

Description: Error in accessing /dev/rmt directory when trying to discover new paths under that directory. Because of the error, we may not discover any new tape devices added to this system.

Solution: If the error is "No such file or directory" or "Not a directory" and there are tape devices attached to this host intended for access, create /dev/rmt directory by hand and run scgdevs command. If the error is "Permission denied" and there are tape devices attached to this host intended for access, check and fix the permissions in /dev/rmt and run scgdevs command. If there are no tape devices attached to this host, no action is required. For complete list of errors, refer to the ftw(3C) man page.

`635342 QUORUM_GENERIC: quorum_reserve error: Read check found node's key is absent on quorum device %s.`

Description: An error was encountered when placing the node's reservation key on the specified quorum device. The write operation of the key succeeded, but the read operation to confirm the presence of the key failed. This error occurs because of unexpected behavior from the device driver, the multipathing driver, or the device's firmware.

Solution: Contact your authorized Oracle service provider to determine if a workaround or patch is available.

`635546 Unable to access the executable %s associated with the FilesystemCheckCommand extension property: %s.`

Description: Self explanatory.

Solution: Check and correct the rights of the specified filename by using the chown/chmod commands.

`635655 All the sQFS file systems configured to this node through SUNW.qfs resources might not be available. Zone boot operation is not successful.`

Description: The zone boot handler failed to make the shared qfs file systems configured for this node available.

Solution: Investigate the zone boot log messages and rectify the problem. If problem persists after rebooting the zone, contact your authorized Oracle service provider with copy of /var/adm/messages files on all nodes.

`635839 first probe was unsuccessful, try again in 5 seconds`

Description: the first sanity check was unsuccessful, may be out of sessions

Solution: none required, it is recommended to observe Tomcats number of configured sessions

`635928 Role change %s detected.`

Description: The role of the Oracle instance does not match the role that is indicated by the combination of the extension properties dataguard_role and standby_mode.

Solution: Update the extension properties dataguard_role and standby_mode to reflect the actual role of the Oracle instance. If the problem persists, contact your Sun support representative for further assistance.

`636339 Deletion of FMRI %s failed`

Description: Attempt of deletion of SMF service is failed.

Solution: User need to manually bring SMF service online. message.

`637220 Function: validate_options - %s Option %s not set`

Description: The option is not specified in the start, stop or probe command.

Solution: Make sure that the appropriate config file is correct and reregister the resource.

`637372 invalid IP address in hosts list: %s`

Description: The allow_hosts or deny_hosts for the CRNP service contained an invalid IP address. This error may cause the validate method to fail or prevent the cl_apid from starting up.

Solution: Remove the offending IP address from the allow_hosts or deny_hosts property.

`637466 Function: start_sczbt - Running command %s %s failed. Unable to start zone.`

Description: Unable to perform platform specific setup for the solaris8 or solaris9 zone.

Solution: Disable the resource and manually run '/usr/lib/brand/solaris8/s8_p2v <zonename>' for solaris8 or '/usr/lib/brand/solaris9/s9_p2v <zonename>' for solaris9 on the node where the start failed. Correct any errors reported, then restart the resource.

`637476 Failed to add NO-OP flag for %s to ccr.`

Description: Failed to update the XMl configuration to the CCR.

Solution: Check the syslog for further messages. Determine why the ccr update failed.

`637886 Error: Unable to start ${SERVER}.`

Description: Unable to start Quorum Device monitoring daemon

Solution: Contact your authorized Oracle service provider to determine whether a workaround or a patch is available.

`637975 Stop of HADB node %d completed successfully.`

Description: The resource was able to successfully stop the specified HADB node.

Solution: This is an informational message, no user action is needed.

`638544 clcomm: Adapter %s has been disabled`

Description: A network adapter has been disabled.

Solution: No action required.

`638660 %s: terminated by signal (%d)`

Description: The vxprint command was terminated by a signal.

Solution: Contact your authorized Oracle service provider for further assistance.

`638675 Directory %s is not owned by user (UID) %d`

Description: The directory is not owned by the uid which is listed in the message.

Solution: Set the permissions on the directory so that it is owned by the uid which is listed in the message.

`638833 scha_resourcegroup_open() failed for RG %s: %s`

Description: A call to scha_resourcegroup_open() failed.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`639042 Function: validate: The %s_script %s must be an absolute path`

Description: The script must be an absolute path.

Solution: Specify an absolute path for the script name.

`639057 Loadlimit %s property removed.`

Description: This is a notification from the rgmd that a loadlimit property has been removed. This message can be used by system monitoring tools.

Solution: This is an informational message; no user action is needed.

`639176 Resource <%s> Update method suppressed because a resource is restarting in resource group <%s>`

Description: Following the update of a resource group or resource property, rgmd is unable to execute an Update method on the indicated resource because the resource group is currently busy on this node executing a resource restart. This should not occur, and might indicate a logic error in rgmd. The rgmd suppresses execution of the Update method on this resource. Depending on what property was updated, this might prevent this resource or its monitor from functioning properly on this node.

Solution: To recover from this error, execute another update of a resource group or resource property. You may re-execute the same update that produced the error. Save a copy of the /var/adm/messages files on all nodes, and contact your Oracle service provider to determine whether a workaround or patch is available.

`639390 Failed to create a temporary filename.`

Description: HAStoragePlus was not able to create a temporary filename in /var/cluster/run.

Solution: Check that permissions are right for directory /var/cluster/run. If the problem persists, contact your authorized Oracle service provider.

`639531 Fork failed.`

Description: The proxy event listener daemon could not fork. The daemon would exit now.

Solution: It is possible that there is not enough memory available. Lack of memory might lead to other problems. You must free up memory on the node or add more memory to the system. If that does not resolve the problem, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`639766 Error in finding the network specification for this node.`

Description: Cannot obtain network specification for this node.

Solution: Make sure you have setup the IP, netmask, NIS domain etc on the name servers and DHCP server.

`639808 Validation failed. The vucmmd daemon will not be started on this node.`

Description: At least one of the modules of the VUCMM framework returned error during validation. The vucmmd daemon will not be started on this node and this node will not be able to run SVM multiowner disksets or CVM shared volumes.

Solution: This message can be ignored if this node is not configured to run the VUCMM framework. Examine other syslog messages logged at about the same time to determine the configuration errors. Examine the VUCMM reconfiguration log file /var/cluster/vucmm/vucmm_reconf.log. Correct the problem and reboot the node. If problem persists, save a copy of the log files on this node and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`639855 IPMP group %s has status %s. Assuming this node cannot respond to client requests.`

Description: The state of the IPMP group named is degraded.

Solution: Make sure all adapters and cables are working. Look in the /var/adm/messages file for message from the network monitoring daemon (cl_pnmd).

`640029 PENDING_ONLINE: bad resource state <%s> (%d) for resource <%s>`

Description: The rgmd state machine has discovered a resource in an unexpected state on the local node. This should not occur and may indicate an internal logic error in the rgmd.

Solution: Look for other syslog error messages on the same node. Save a copy of the /var/adm/messages files on all nodes, and report the problem to your authorized Oracle service provider.

`640232 dismount_unconnected_asm_diskgroup - Oracle ASM instance (%s) diskgroup (%s) is connected and remains mounted`

Description: The Oracle ASM diskgroup has not been dismounted.

Solution: None required. Informational message.

`640366 Entry for file system mount point %s is absent from global zone %s.`

Description: HAStoragePlus looked for the specified mount point in the specified file (usually /etc/vfstab) but didn't find it.

Solution: Usually, this means that a typo has been made when filling the FilesystemMountPoints property -or- that the entry for the file system and mount point does not exist in the specified file.

`640408 INTERNAL ERROR: Command %s Failed.`

Description: An internal error has occurred.

Solution: Save a copy of the /var/adm/messages files on all nodes. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`640484 clconf: No valid votecount field for quorum device %d`

Description: Found the votecount field for the quorum device being incorrect while converting the quorum configuration information into quorum table.

Solution: Check the quorum configuration information.

`640521 UNRECOVERABLE ERROR: /usr/cluster/lib/scadmin/ql/data_change file not present.`

Description: /usr/cluster/lib/scadmin/ql/data_change file not present.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`640542 %s - the sql command %s failed`

Description: The mysql command failed.

Solution: Consult the mysql error logs to determine the root cause

`640673 SCSLM <%s> pool_associate error <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`640718 cl_execd: Could not resolve '%s' in the name server. Exiting.`

Description: The cl_execd program coudn't resolve the cl_exec server object from the global name server.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`640799 pmf_alloc_thread: ENOMEM`

Description: The rpc.pmfd server was not able to allocate a new monitor thread. As a consequence, the rpc.pmfd server was not able to monitor a process. This problem can occur if the machine has low memory. An error message is output to syslog.

Solution: Determine if the machine is running out of memory. If this is not the case, save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`640902 Membership: To achieve fully connected membership, attempt to halt local zone cluster node '%s' (incn %d, zone ID %d, zone uniq ID %d).`

Description: The specified zone cluster has disconnects in its private network. The membership subsystem will attempt to halt the specified zone cluster node on the local node, to achieve a fully connected membership.

Solution: This message is informational. No action is needed.

`641293 fatal: could not start infrastructure table callback handler thread`

Description: Could not start the rgmd thread that will service infrastructure table callbacks from CCR. This might be due to inadequate memory on the system.

Solution: Add more memory to the system. If that does not resolve the problem, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`641686 sigaction: %s`

Description: The rpc.pmfd server was not able to set its signal handler. The message contains the system error. This happens while the server is starting up, at boot time. The server does not come up, and an error message is output to syslog.

Solution: Save the syslog messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`641790 Failed to delete domain XML %s from ccr.`

Description: Failed to delete domain configuration from CCR.

Solution: Check the syslog for further messages. Determine why the ccr delete failed.

`641934 Error: Unable to determine Oracle CRS version.`

Description: The Oracle CRS version cannot be determined.

Solution: Check that Oracle CRS has been installed properly.

`642075 reservation notice(%s) - MHIOCGRP_RESERVE success during retry attempt: %d`

Description: Informational message from reserve on ioctl success during retry.

Solution: No user action required.

`642220 There is already an instance of this daemon running`

Description: The server did not start because there is already another identical server running

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`642286 Stop of HADB database did not complete.`

Description: The resource was unable to successfully run the hadbm stop command either because it was unable to execute the program, or the hadbm command received a signal.

Solution: This might be the result of a lack of system resources. Check whether the system is low in memory and take appropriate action.

`642451 This resource doesn't depend on any SUNW.HAStoragePlus resources. Proceeding with the normal checks.`

Description: The data service normally checks for the status of its dependent SUNW.HAStoragePlus resources. The check is not done because there is not dependency on an HAStoragePlus resource in your configuration.

Solution: This message is informational; no user action is needed.

`643802 Resource group is online on more than one node.`

Description: An internal error has occurred. Resource group should be online on only one node.

Solution: Save a copy of the /var/adm/messages files on all nodes. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`644313 Unable to stop HADB nodes on %s`

Description: The resource was unable to successfully run the hadbm stopnode command either because it was unable to execute the program, or the hadbm command received a signal.

Solution: This might be the result of a lack of system resources. Check whether the system is low in memory and take appropriate action.

`644514 HAIP <%s> popen error <%s>`

Description: popen() error.

Solution: Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`644715 Diskgroup (%s) not multi-owner!`

Description: The disk set that is specified is not a multiowner disk set, or the disk group that is specified is not a shared-disk group.

Solution: Specify a multiowner disk set or a shared-disk group. Contact your authorized Oracle service provider for further assistance in diagnosing the problem.

`644850 File %s is not readable: %s.`

Description: Unable to open the file in read only mode.

Solution: Make sure the specified file exists and have correct permissions. For the file name and details, check the syslog messages.

`644941 Probe failed, HTTP GET Response Code for %s is %d.`

Description: The status code of the response to a HTTP GET probe that indicates the HTTP server has failed. It will be restarted or failed over.

Solution: This message is informational; no user action is needed.

`644992 Invalid user probe file %s.`

Description: Incorrect user probe file specified.

Solution: Ensure that a valid user probe file is specified.

`645309 Thread already running for node %d.`

Description: The cl_eventd does not need to create a thread because it found one that it could use.

Solution: This message is informational only, and does not require user action.

`645431 Oracle Grid Infrastructure resource %s has ACTION_SCRIPT=%s which is incorrect.`

Description: The Oracle Grid Infrastructure resource ACTION_SCRIPT attribute value is incorrect.

Solution: You must set the ACTION_SCRIPT attribute for the Oracle Grid Infrastructure resource equal to <basedir>/SUNWscor/dsconfig/bin/scproxy_crs_action. Note that <basedir> is typically /opt. Refer to the Oracle Solaris Cluster documentation for information on how to do this.

`645463 More than one matching zpool for '%s': zpools configured to HAStoragePlus must be unique.`

Description: Self explanatory.

Solution: Ensure there is a unique zpool for the specified zpool name by renaming the duplicate zpools.

`645501 %s initialization failure`

Description: Failed to initialize the hafoip or hascip callback method.

Solution: Retry the operation. If the error persists, contact your Oracle service representative.

`645583 Outgoing connections feature is not supported in base Solaris.`

Description: An attempt was made to enable the outgoing connections feature. The base Solaris version on the cluster nodes does not support this feature.

Solution: Upgrade to a compatible version of Solaris to enable this feature.

`646037 Probe timed out.`

Description: The data service fault monitor probe could not complete all actions in Probe_timeout. This may be due to an overloaded system or other problems. Repeated timeouts will cause a restart or failover of the data service.

Solution: If this problem is due to an overloaded system, you may consider increasing the Probe_timeout property.

`646392 sigfillset returned %d. Exiting.`

Description: The rtreg server program has encountered a failed sigfillset(3C) call. The error message contains the error number for the failure.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`646622 ucmmd got stale upgrade callback from version manager: upgrade version is %d.%d while current version is %d.%d`

Description: Version manager delivered upgrade callback to ucmmd about a new version that is not higher than the current version of ucmmd.

Solution: No user action is required. This message is for informational purposes only.

`646664 Online check Error %s: %ld: %s`

Description: Error detected when checking ONLINE status of RDBMS. Error number is indicated in message.This can be because of RDBMS server problems or configuration problems.

Solution: Check RDBMS server using vendor provided tools. If server is running properly, this can be fault monitor set-up error.

`646815 PCUNSET: %s`

Description: The rpc.pmfd server was not able to monitor a process, and the system error is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`646865 Monitor probe time of %0.2f seconds is %0.2f percent of Probe timeout.`

Description: This message indicates that the probe time of the monitor exceeded the probe timeout threshold(50%) for the resource.

Solution: This message suggests that the Probe_timeout extension property for this resource might be set too low. If the indicated probe execution time is typical for your level of system load, consider increasing the value of Probe_timeout to avoid extraneous restarts or failovers that would be triggered by exceeding the configured probe timeout.

`646950 clcomm: Path %s being cleaned up`

Description: A communication link is being removed with another node. The interconnect might have failed or the remote node might be down.

Solution: Any interconnect failure should be resolved, and/or the failed node rebooted.

`647187 Function: start_sczsh - %s does not exist or is not executable in zone %s - early End`

Description: The command specified for variable ServiceStartCommand within the /opt/SUNWsczone/sczsh/util/sczsh_config configuration file is not executable or does not exist in the specified zone.

Solution: Make sure that the command specified for variable ServiceStartCommand within the /opt/SUNWsczone/sczsh/util/sczsh_config configuration file is existing and executable for user root in the specified zone. If you do not want to re-register the resource, make sure the variable ServiceStartCommand is properly set within the ${PARAMETERDIR}/sczsh_${RS} parameterfile.

`647478 Cannot access dir: %s (%s)`

Description: Unable to access the directory because of the indicated reason.

Solution: Check that the directory exists and has the correct permissions.

`648057 Successful auto recovery of HADB database.`

Description: The database was successfully reinitialized.

Solution: This is an informational message, no user action is needed.

`648109 check_psadmin_sstatus_error - psadmin -c sstatus -d %s reported an error message`

Description: An error messages has been reported while performing psadmin -c sstatus -d ${Psft_Domain}.

Solution: Check the log files for the configured PeopleSoft Enterprise application server domain configuration for any errors reported.

`648118 Migration of domain %s is cancelled, the domain state is now in active state.`

Description: The domain migration operation was cancelled.

Solution: None required. Informational message.

`648926 The original file system with mount point %s specified in the /etc/vfstab of a global zone for the lofs file system with mount point %s configured for the zone cluster %s is a local file system. The AffinityOn value cannot be FALSE for local file systems.`

Description: HAStoragePlus detected that the specified mount point in /etc/vfstab is a local mount point, hence extension property AffinityOn must be set to True.

Solution: Set the AffinityOn extension property of the resource to True.

`649149 stop_opmn - The stop command (%s) timed-out.`

Description: The Oracle Process Management and Notification Server failed to stop within the STOP_TIMEOUT period.

Solution: Increase the value of the STOP_TIMEOUT property.

`649584 Modification of resource group <%s> failed because none of the nodes on which VALIDATE would have run for resource <%s> are currently up`

Description: Before it will permit the properties of a resource group to be edited, the rgmd runs the VALIDATE method on each resource in the group for which a VALIDATE method is registered. For each such resource, the rgmd must be able to run VALIDATE on at least one node. However, all of the candidate nodes are down. "Candidate nodes" are either members of the resource group's Nodelist or members of the resource type's Installed_nodes list, depending on the setting of the resource's Init_nodes property.

Solution: Boot one of the resource group's potential masters and retry the resource creation operation.

`649648 svc_probe used entire timeout of %d seconds during read operation and exceeded the timeout by %d seconds. Attempting disconnect with timeout %d`

Description: The probe timed out while reading from the application.

Solution: If the problem persists investigate why the application is responding slowly or if the Probe_timeout property needs to be increased.

`649799 SCSLM strdup error <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`650231 The quorum configuration task succeeded on node ${nodename}`

Description: Quorum disk was configured and the cluster was initialized on the last of the 2 nodes to join the cluster.

Solution: None.

`650825 Method <%s> on resource <%s> terminated due to receipt of signal <%d>`

Description: A resource method was terminated by a signal, most likely resulting from an operator-issued kill(1). The method is considered to have failed.

Solution: No action is required. The operator can choose to issue a clresourcegroup command to bring resource groups onto desired primaries, or re-try the administrative action that was interrupted by the method failure.

`650889 Failed to update the OPERATION value within /var/cluster/%s_OPERATION, [%s]`

Description: Failed to update the OPERATION value within /var/cluster

Solution: Determine why the updated failed. If the problem persists, contact your Oracle support representative for further assistance.

`651024 %s can't plumb interface %s: out of instance numbers on %s.`

Description: The adapter already has the maximum number of logical interfaces that can be configured.

Solution: Try one of the following actions: - Add a new adapter to the IPMP group - Remove from the interface all logical interfaces that are not needed - Specify a new IPMP group for this resource

`651091 INTERNAL ERROR: Invalid upgrade-from tunablity flag <%d>; aborting node`

Description: A fatal internal error has occurred in the RGM.

Solution: Since this problem might indicate an internal logic error in the rgmd, save a copy of the /var/adm/messages files on all nodes, and the output of clresourcetype show -v, clresourcegroup show -v +, and clresourcegroup status +. Report the problem to your authorized Oracle service provider.

`651093 reservation message(%s) - Fencing node %d from disk %s`

Description: The device fencing program is taking access to the specified device away from a non-cluster node.

Solution: This is an informational message, no user action is needed.

`651327 Failed to delete scalable service group %s: %s.`

Description: A call to the underlying scalable networking code failed.

Solution: Save a copy of the /var/adm/messages files on all nodes. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`651848 Failed to halt the nodes of the partition`

Description: The upgrade process halts the nodes of the second partition. It then waits till all the nodes of the partition have halted. The upgrade process was unable to ping all nodes of the partition.

Solution: Cluster upgrade has failed. Reboot all the nodes out of cluster mode and recover from upgrade. Finish the cluster upgrade by using the standard upgrade method.

`652019 Executing in zones context with id %d that does not match id %d in the door header`

Description: Zone information that has been passed does not match the client credentials.

Solution: Someone has tampered with the Oracle Solaris Cluster code. This is a security violation. Shut down the identified zone.

`652060 Error when disabling Oracle CRS autostart using: "%s", error(%s), "%s".`

Description: Disabling Oracle CRS autostart failed.

Solution: Check errors on the system and correct them before restarting the CRS framework resource. If you cannot resolve this problem, contact your Oracle service representative for assistance.

`652399 Ignoring the SCHA_ERR_SEQID while retrieving %s`

Description: An update to the cluster configuration tables occured while trying to retrieve certain cluster related information. However, the update does not affect the property that is being retrieved.

Solution: Ignore the message

`652517 Unable to open device %s. %s`

Description: The device could not be opened.

Solution: Run the command metaset to determine if the metaset is online on the node. Run metastat to determine the state of the volume.

`652662 libsecurity: program %s (%lu) rpc_createerror: %s`

Description: A client of the specified server was not able to initiate an rpc connection. The error message generated with a call to clnt_spcreateerror(3NSL) is appended.

Solution: Save the /var/adm/messages file. Check the messages file for earlier errors related to the rpc.pmfd, rpc.fed, or rgmd server.

`652764 libsecurity, door_call: %s; will retry`

Description: A door_call failed with the specified reason.

Solution: No action necessary.

`653062 Syntax error on line %s in dfstab file.`

Description: The specified share command is incorrect.

Solution: Correct the share command using the dfstab(4) man pages.

`653151 The specified zone cluster name '%s' does not exist.`

Description: The specified zone cluster name does not exist.

Solution: Check the zone cluster support on the current running version of cluster and then check the existence of the specified zone cluster. If zone cluster name exists, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`653910 [PID %d] Door call failed : error %s`

Description: The specified process failed to send a request to the proxy event listener daemon.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`654006 Error occured during instance shutdown (%s).`

Description: An error occured during an attempt by the agent to shutdown an Oracle instance.

Solution: Check the /var/adm/messages file for details of the error. If this does not provide sufficient information, refer to the alert log file of the Oracle instance for details on the error. Contact your Oracle support representative for further assistance.

`654276 realloc failed with error code: %d`

Description: The call to realloc(3C) in the cl_apid failed with the specified error code.

Solution: Increase swap space, install more memory, or reduce peak memory consumption.

`654520 INTERNAL ERROR: rgm_run_state: bad state <%d> for resource group <%s>`

Description: The rgmd state machine on this node has discovered that the indicated resource group's state information is corrupted. The state machine will not launch any methods on resources in this resource group. This may indicate an internal logic error in the rgmd.

Solution: Other syslog messages occurring before or after this one might provide further evidence of the source of the problem. If not, save a copy of the /var/adm/messages files on all nodes, and (if the rgmd crashes) a copy of the rgmd core file, and contact your authorized Oracle service provider for assistance.

`654659 SCSLM <%s> zone <%s> weak while strong running`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`654749 Ifconfig proxy daemonize() : fork1 failed, errno = %d. Exiting.`

Description: The ifconfig proxy program has encountered failure of fork1(2) system call. The error message indicates the error number for the failure.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`654823 Function: validate: The replicated enqueue server resource property is empty`

Description: The replicated enqueue server resource is empty.

Solution: Specify a list of valid replicated enqueue server resources in the REPENQRES property.

`655203 Invalid upgrade callback from version manager: unrecognized ucc_name %s`

Description: The rgmd received an invalid upgrade callback from the version manager. The rgmd will ignore this callback.

Solution: No user action required. If you were trying to commit an upgrade, and it does not appear to have committed correctly, save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing the problem.

`655410 getsockopt: %s`

Description: The cl_apid received the following error while trying to deliver an event to a CRNP client. This error probably represents a CRNP client error or temporary network congestion.

Solution: No action required, unless the problem persists (ie. there are many messages of this form). In that case, examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`655416 setsockopt: %s`

Description: The cl_apid experienced an error while configuring a socket. This error may prohibit event delivery to CRNP clients.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`655523 Write to server failed: server %s port %d: %s.`

Description: The agent could not send data to the server at the specified server and port.

Solution: This is an informational message, no user action is needed. If the problem persists the fault monitor will restart or failover the resource group the server is part of.

`655966 Error: pmfadm could not stop fault monitor, tag "%s" error="%s"`

Description: The CRS framework monitor failed to stop or is not running.

Solution: Check for other errors on the system and correct them before restarting the CRS framework resource. If you cannot resolve this problem, contact your Oracle service representative for assistance.

`656764 fatal: rgmx_update_configured_nodes() : failed`

Description: The RGM couldn't update membership information for farm nodes and will abort to avoid undesired results.

Solution: Contact your authorized Oracle representatives for resolution of the problem.

`656795 CMM: Unable to bind <%s> to nameserver.`

Description: An instance of the userland CMM encountered an internal initialization error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`657145 Function: check_repenqpre: The giveover of the replicated enqueue server resource %s is impossible`

Description: The giveover of the replicated enqueue server resource is impossible.

Solution: None.

`657447 %s: stopped by signal (%d)`

Description: The vxprint command was stopped by a signal.

Solution: Contact your authorized Oracle service provider for further assistance.

`657495 Tag %s: error number %d in throttle wait; process will not be requeued.`

Description: An internal error has occured in the rpc.pmfd server while waiting before restarting the specified tag. rpc.pmfd will delete this tag from its tag list and discontinue retry attempts.

Solution: If desired, restart the tag under pmf using the 'pmfadm -c' command.

`657750 File-system type "%s" is not allowed.`

Description: The value of the property FileSystemType is not "nas" or "s-qfs".

Solution: Specify the correct value for the property and repeat the operation.

`657759 Error: An instance of ${SERVER} is already running.`

Description: Quorum Device monitoring daemon is already running.

Solution: No user action is required.

`657885 sigwait: %s`

Description: The daemon was unable to configure its signal handling functionality, so it is unable to run.

Solution: Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`657922 stop_ohs - The stop command (%s) completed successfully.`

Description: The Oracle HTTP Server stopped successfully.

Solution: None.

`658049 dl_bind: DL_ERROR_ACK protocol error`

Description: Could not bind to the physical device. We are trying to open a fast path to the private transport adapters.

Solution: Reboot of the node might fix the problem.

`658329 CMM: Waiting for initial handshake to complete.`

Description: The userland CMM has not been able to complete its initial handshake protocol with its counterparts on the other cluster nodes, and will only be able to join the cluster after this is completed.

Solution: This is an informational message, no user action is needed.

`658555 Retrying to retrieve the resource information.`

Description: An update to cluster configuration occured while resource properties were being retrieved

Solution: Ignore the message.

`658972 Error: The existing RAC CVM resource version "%s" is less than the minimum acceptible version "%s".`

Description: The existing RAC CVM resource version indicates that it is not VUCMM aware.

Solution: Update the existing RAC CVM resource version.

`659109 UNRECOVERABLE ERROR: Oracle Solaris Cluster boot: Could not load module $m`

Description: A kernel module is missing or is corrrupted.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`659369 ct_status_read: %s`

Description: The libclcontract code was unable to retrieve the status of the newly created process template. The libclcontract code will kill the newly forked process because the process using the libclcontract code is unable to monitor the forked process.

Solution: If the message is isolated, it was probably a transient error. Retry the requested action. If there are many instances of the message, there is a problem with the contract file system. Search for other syslog error messages on the same node. Save a copy of the /var/adm/messages files on all nodes, and report the problem to your authorized Oracle service provider.

`659665 kill -KILL: %s`

Description: The rpc.fed server is not able to stop a tag that timed out, and the error message is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Examine other syslog messages occurring around the same time on the same node, to see if the cause of the problem can be identified.

`659683 check_dhcp - Active interface has changed from %s to %s`

Description: The DHCP resource's fault monitor has detected that the active interface has changed.

Solution: No user action is needed. The fault monitor will restart the DHCP server.

`659757 The file system associated with mount point %s will be locally mounted in zone cluster %s. The AffinityOn value cannot be FALSE.`

Description: HAStoragePlus detected that the specified mount point in the /etc/vfstab file is a local mount point. Therefore, the extension property AffinityOn must be set to True.

Solution: Set the AffinityOn extension property of the resource to True.

`659827 CCR: Can't access CCR metadata on node %s errno = %d.`

Description: The indicated error occurred when CCR is trying to access the CCR metadata on the indicated node. The errno value indicates the nature of the problem. errno values are defined in the file /usr/include/sys/errno.h. An errno value of 28(ENOSPC) indicates that the root files system on the node is full. Other values of errno can be returned when the root disk has failed(EIO).

Solution: There may be other related messages on the node where the failure occurred. These may help diagnose the problem. If the root file system is full on the node, then free up some space by removing unnecessary files. If the root disk on the afflicted node has failed, then it needs to be replaced. If the cluster repository is corrupted, boot the indicated node in -x mode to restore it from backup. The cluster repository is located at /etc/cluster/ccr/.

`660000 DID Reloader is complete.`

Description: The DID reloader daemon completed successfully.

Solution: This is a debug message, no user action is necessary.

`660332 launch_validate: fe_set_env_vars() failed for resource <%s>, resource group <%s>, method <%s>`

Description: The rgmd was unable to set up environment variables for method execution, causing a VALIDATE method invocation to fail. This in turn will cause the failure of a creation or update operation on a resource or resource group.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Re-try the creation or update operation. If the problem recurs, save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance.

`660368 CCR: CCR service not available, service is %s.`

Description: The CCR service is not available due to the indicated failure.

Solution: Reboot the cluster. Also contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`660752 Validation failed in the start method.`

Description: The resource did not pass the validation checks.

Solution: Determine whether the properties of the resource are valid. Verify the status of the disk set or disk group. Contact your authorized Oracle service provider for further assistance in diagnosing the problem.

`660974 file specified in USER_ENV %s does not exist`

Description: 'User_env' property was set when configuring the resource. File specified in 'User_env' property does not exist or is not readable. File should be specified with fully qualified path.

Solution: Specify existing file with fully qualified file name when creating resource. If resource is already created, please update resource property 'User_env'.

`661077 dl_info: DL_INFO_ACK protocol error`

Description: Could not get a DLPI info_ack message from the physical device. We are trying to open a fast path to the private transport adapters.

Solution: Reboot of the node might fix the problem.

`661560 All the SUNW.HAStoragePlus resources that this resource depends on are online on the local node. Proceeding with the checks for the existence and permissions of the start/stop/probe commands.`

Description: The HAStoragePlus resource that this resource depends on is local to this node. Proceeding with the rest of the validation checks.

Solution: This message is informational; no user action is needed.

`661778 clcomm: memory low: freemem 0x%x`

Description: The system is reporting that the system has a very low level of free memory.

Solution: If the system fails soon after this message, then there is a significantly greater chance that the system ran out of memory. In which case either install more memory or reduce system load. When the system continues to function, this means that the system recovered and no user action is required.

`662410 validate_zcboot_resource - Oracle Grid Infrastructure resource %s has ACTION_SCRIPT=%s which either does not exist or is not executable.`

Description: The Oracle Grid Infrastructure resource sun.<resource> ACTION_SCRIPT attribute value is incorrect.

Solution: You must set the ACTION_SCRIPT attribute of the Oracle Grid Infrastructure resource, sun.<resource>, equal to <basedir>/SUNWscor/dsconfig/bin/scproxy_crs_action. Note that <resource> represents the Oracle Solaris Cluster SUNW.wait_zc_boot resource name and <basedir> is typically /opt. Refer to the Oracle Solaris Cluster documentation for information on how to do this.

`662507 Could not run %s. User program did not execute cleanly.`

Description: There were problems making an upcall to run a user-level program.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`662516 SIOCGLIFNUM: %s`

Description: The ioctl command with this option failed in the cl_apid. This error may prevent the cl_apid from starting up.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`662761 <%s>:<%s>:Process core dump at <%s> for method <%s> may fail.`

Description: The rpc.fed couldn't ensure a writable directory in which to dump process cores. The exact message indicates the nature of the error.

Solution: Low memory might be a transient error. If it persists, rebooting might help. If that fails to help, swap space might need to be increased. If it is a different error, it might be due to insufficient privileges for ensuring a writable directory to store the core. Check and correct the permissions for path indicated in the message.

`663089 clexecd: %s: sigwait returned %d. Exiting.`

Description: clexecd program has encountered a failed sigwait(3C) system call. The error message indicates the error number for the failure.

Solution: The clexecd program will exit and the node will be halted or rebooted to prevent data corruption. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`663293 reservation error(%s) - do_status() error for disk %s`

Description: The device fencing program has encountered errors while trying to access a device. All retry attempts have failed.

Solution: The action which failed is a scsi-2 ioctl. These can fail if there are scsi-3 keys on the disk. To remove invalid scsi-3 keys from a device, use 'scdidadm -R' to repair the disk (see scdidadm man page for details). If there were no scsi-3 keys present on the device, then this error is indicative of a hardware problem, which should be resolved as soon as possible. Once the problem has been resolved, the following actions may be necessary: If the message specifies the 'node_join' transition, then this node may be unable to access the specified device. If the failure occurred during the 'release_shared_scsi2' transition, then a node which was joining the cluster may be unable to access the device. In either case, access can be reacquired by executing '/usr/cluster/lib/sc/run_reserve -c node_join' on all cluster nodes. If the failure occurred during the 'make_primary' transition, then a device group might have failed to start on this node. If the device group was started on another node, move it to this node by using the cldevicegroup command. If the device group was not started, Start it by using the cldevicegroup command. If the failure occurred during the 'primary_to_secondary' transition, then the shutdown or switchover of a device group might have failed. If so, you can retry the desired action.

`663352 sysevent_subscribe_event() for %s failed : %s`

Description: The proxy event listener daemon could not subscribe with the Solaris sysevents framework for listening to cluster events, because sysevent_subscribe_event(3SYSEVENT) failed. Hence, the daemon will not receive any cluster event notifications.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`663458 Failed to add qfs name '%s' to qfs list for resource '%s'.`

Description: The zone boot handler failed to generate the list of shared qfs file systems for a resource.

Solution: Investigate using zone boot log messages and try to rectify the problem. If problem persists after rebooting the zone, contact your authorized Oracle service provider with copy of /var/adm/messages files on all nodes.

`663692 failback attempt failed on resource group <%s> with error <%s>`

Description: The rgmd daemon was unable to failback the specified resource group to a more preferred node. The additional error information in the message explains why.

Solution: Examine other syslog messages occurring around the same time on the same node. These messages may indicate further action.

`663851 Failover %s data services must have exactly one value for extension property %s.`

Description: Failover data services must have one and only one value for Confdir_list.

Solution: Create a failover resource group for each configuration file.

`663897 clcomm: Endpoint %p: %d is not an endpoint state`

Description: The system maintains information about the state of an Endpoint. The Endpoint state is invalid.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`663943 CMM: Unable to reset node information on quorum disk.`

Description: This node was unable to reset some information on the quorum device. This will lead the node to believe that its partition has been preempted. This is an internal error. If a cluster gets divided into two or more disjoint subclusters, exactly one of these must survive as the operational cluster. The surviving cluster forces the other subclusters to abort by grabbing enough votes to grant it majority quorum. This is referred to as preemption of the losing subclusters.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`664065 DID reconfiguration discovered an invalid disk path. You must remove this path before you can add a new path. Run 'cldevice clear' then rerun 'cldevice refresh'.`

Description: Invalid disk path found during DID reconfiguration.

Solution: Remove the invalid disk path and run 'cldevice clear' and rerun 'cldevice refresh'.

`664089 All global device services successfully switched over to this node.`

Description: Self explanatory.

Solution: This is an informational message, no user action is needed.

`664371 %s: Not in cluster mode. Exiting...`

Description: Must be in cluster mode to execute this command.

Solution: Reboot into cluster mode and retry the command.

`664508 Domain %s does not exist.`

Description: The domain does not exist.

Solution: You must ensure that the domain exists.

`664658 Waiting for the RGM to stop`

Description: The RGM service is being stopped.

Solution: Please wait. No action is required. This is an informational message.

`664741 Failed to execute zoneadm reboot for zone %s`

Description: The rgmd failed in an attempt to execute a zoneadm reboot call, due to a failure to communicate with the rpc.fed daemon. The zone will stay up, but the rgmd will not attempt to start services on the zone.

Solution: Reboot the zone. Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing the problem.

`665015 Scalable service instance [%s,%s,%d] registered on node %s.`

Description: The specified scalable service has been registered on the specified node. Now, the gif node can redirect packets for the specified service to this node.

Solution: This is an informational message, no user action is needed.

`665090 libsecurity: program %s (%lu); getnetconfigent error: %s`

Description: A client of the specified server was not able to initiate an rpc connection, because it could not get the network information. The pmfadm or scha command exits with error. The rpc error is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`665318 Not attempting to start resource group <%s> on node <%s> because a hard load limit <%s> would be exceeded.`

Description: The rgmd is unable to bring the specified resource group online on the specified node because the specified hard load limit would be exceeded, even after all possible resource group preemptions.

Solution: This message indicates normal enforcement of resource group load limits. No user action is required. You can adjust the loadlimits node property, or the loadfactors or priority resource group property, to allow additional load on that node. You can also check if a resource group in the ERROR_STOP_FAILED state exists on the indicated node. Even if it has low priority, this resource group cannot be switched off of the node until the error condition has been cleared.

`665368 Failed to wake up probing thread (ID %d) : %s`

Description: The thread that is used for the I/O probe could not be sent the wake-up signal.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`665564 Failed to commit the %s to the CCR table %s.`

Description: An unknown error has occurred while updating the CCR with the current zone cluster name.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`665931 Initialization error. CONNECT_STRING is NULL`

Description: Error occured in monitor initialization. Monitor is unable to get resource property 'Connect_string'.

Solution: Check syslog messages for errors logged from other system modules. Check the resource configuration and value of 'Connect_string' property. Check syslog messages for errors logged from other system modules. Stop and start fault monitor. If error persists then disable fault monitor and report the problem.

`666368 SCSLM unknown method 0x%x`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`666391 clcomm: invalid invocation result status %d`

Description: An invocation completed with an invalid result status.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`666514 Node is not booted with xVM.`

Description: Solaris is not booted with xVM.

Solution: Ensure that the default boot grub menu is set to boot Solaris xVM.

`666603 clexecd: Error %d in fcntl(F_GETFD). Exiting.`

Description: clexecd program has encountered a failed fcntl(2) system call. The error message indicates the error number for the failure.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`666617 Invalid global device path %s detected for SAMFS mount point %s.`

Description: The specified global device path used by the SAM-FS file system is not valid.

Solution: Check the SAM-FS file system configuration.

`666889 Received non interrupt heartbeat on %s - path timeouts are likely`

Description: A heartbeat message was unexpectedly delivered in the non-interrupt context. Oracle Solaris Cluster requires that network drivers must deliver heartbeat messages in the interrupt context.

Solution: Check for the right version of the driver.

`667067 sv_get_zone_iptype: Could not init zone handle`

Description: During initialization, there was a failure to get information about configured zones in the system.

Solution: There might be other related error messages which might be helpful. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`667180 set_scproxy_resource - Oracle Grid Infrastructure type sun.storage_proxy.type is not registered.`

Description: The Oracle Grid Infrastructure type sun.storage_proxy.type is not registered.

Solution: You must create the Oracle Grid Infrastructure type sun.storage_proxy.type. Refer to the Oracle Solaris Cluster documentation for information on how to do this.

`667314 CMM: Enabling failfast on quorum device %s failed.`

Description: An attempt to enable failfast on the specified quorum device failed.

Solution: Check whether the specified quorum disk has failed. This message might also be logged when a node is booting up and has been preempted from the cluster, in which case no user action is necessary.

`667451 clq_scsi3: Ioctl MHIOCGRP_RESERVE returned error(%d) also thereservation key was not found on quorum device %s`

Description: An error was encountered while trying to place node's reservation key on specified quorum device. The cluster software read the reservation key and found another node's key present. This error occurs because of unexpected behavior from the device driver or multipathing driver or device's firmware.

Solution: Look for updates in device driver, multipathing driver or device firmware. If no updates are available contact the Oracle device driver support team.

`667548 CCR: Could not create split brain CCR change flag.`

Description: The CCR data server failed to create the file to indicate that there has been a CCR update during split brain under the weak-membership model.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`667613 start_mysql - myisamchk found errors in some index in %s, perform manual repairs`

Description: mysiamchk found errors in MyISAM based tables.

Solution: Consult MySQL documentation when repairing MyISAM tables.

`667638 Cluster initialization encountered a problem on node ${nodename}, manual initialization by using clsetup(1CL) might be necessary`

Description: The attempt to automatically reset cluster installmode or quorum votes failed on one or more nodes. The last node to join the cluster is supposed to perform these tasks. However, other nodes also perform registration tasks when they join the cluster. This message indicates that one or more of these attempts failed, which prevented the last node from initializing the cluster.

Solution: Run clsetup after all nodes have joined the cluster, to complete post-installation setup.

`667957 %s: pipe returned %d. Exiting.`

Description: Internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`668074 Successfully failed-over resource group.`

Description: This message indicates that the rgm successfully failed over a resource group, as expected.

Solution: This is an informational message.

`668209 Function: lofsmount - Non-global zones mount from %s to %s with options %s failed`

Description: The mount of the directory into the root path of the non-global zone failed.

Solution: Watch the verbose error message in the zones start messages and fix the issue with appropriate methods.

`668255 [PID %d] Could not allocate memory`

Description: There is not enough memory available.

Solution: Lack of memory might lead to other problems. You must free up memory on the node or add more memory to the system. If that does not resolve the problem, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`668763 Failed to retrieve information on %s: %s.`

Description: Self explanatory.

Solution: Check the cluster configuration. If the problem persists, contact your authorized Oracle service provider.

`668866 Successful shutdown; terminating daemon`

Description: The cl_apid daemon is shutting down normally due to a SIGTERM.

Solution: No action required.

`668883 Entry in the global zone %s for file system mount point %s is incorrect: %s.`

Description: The format of the entry for the specified mount point in the specified file is invalid.

Solution: Edit the file (usually /etc/vfstab) and check that entries conform to its format.

`669026 fcntl(F_SETFD) failed in close_on_exec`

Description: A fcntl operation failed. The "fcntl" man page describes possible error codes.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`669265 Failed to retrive information '%s' for node '%s' in cluster '%s' :%s.`

Description: Self explanatory.

Solution: Check the cluster configuration. If the problem persists, contact your authorized Oracle service provider.

`669317 fencing fatal error(%s) - host name not specified.`

Description: The zone cluster fencing program has suffered an internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available. Copies of /var/adm/messages from all nodes should be provided for diagnosis.

`669632 Cleanup after upgrade has failed on the nodes that have the joined the cluster`

Description: Cluster upgrade has completed on all the nodes that have joined the cluster. However cleanup of the upgrade state has failed.

Solution: Retry cleanup of the upgrade state by running /usr/cluster/lib/scadmin/ql/ql_cleanup on any node of the cluster.

`669680 fatal: rpc_control() failed to set the maximum number of threads`

Description: The rgmd failed in a call to rpc_control(3N). This error should never occur. If it did, it prevent to dynamically adjust the maximum number of threads that the RPC library can create, and as a result could prevent a maximum number of some scha_call to be handled at the same time.

Solution: Examine other syslog messages occurring at about the same time to see if the source of the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing the problem. Reboot the node to restart the clustering daemons.

`669693 The Oracle instance was found to be disabled. Instance startup will not be attempted.`

Description: The Oracle RAC instance has been disabled.

Solution: If the instance needs to be brought online, run the Oracle Solaris Cluster enable command on the resource. This will internally enable the RAC instance and bring it online.

`669742 Successfully obtained the list of SUNW.qfs resources that are configured for this zone cluster.`

Description: This is a notification message about successful retrieval of configured SUNW.qfs resources for this zone cluster.

Solution: This is an informational message; no user action is needed.

`670107 Error in getting the contract id from repository for svc:%s`

Description: Cannot read the contract id from SMF repository

Solution: The previous messages should give more details for the reason. Make sure the basic SMF functionalities are working correctly. Contact your Oracle service provider for more information.

`670283 Issuing a resource restart request because the application exited.`

Description: This message indicates that the fault monitor is about to request a resource restart because the application exited. If the request fails, refer to the syslog messages that appear after this message.

Solution: This is an informational message.

`670303 The zpool '%s' is not available from this node.`

Description: The device that you added to the zpool is not accessible from this node.

Solution: Check the zpools that can be accessed from this node by doing "zpool list" and "zpool import". If the zpool that is listed in the error message is among them, contact your authorized Oracle service provider. Otherwise, remove the zpool from the resource group or remove the specified node from the resource group.

`670753 reservation fatal error(%s) - unable to determine node id for node %s`

Description: The device fencing program has suffered an internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available. Copies of /var/adm/messages from all nodes should be provided for diagnosis. It may be possible to retry the failed operation, depending on the nature of the error. If the message specifies the 'node_join' transition, then this node may be unable to access shared devices. If the failure occurred during the 'release_shared_scsi2' transition, then a node which was joining the cluster may be unable to access shared devices. In either case, it may be possible to reacquire access to shared devices by executing '/usr/cluster/lib/sc/run_reserve -c node_join' on all cluster nodes. If the failure occurred during the 'make_primary' transition, then a device group has failed to start on this node. If another node was available to host the device group, then it should have been started on that node. If desired, it might be possible to switch the device group to this node by using the cldevicegroup command. If no other node was available, then the device group will not have been started. You can use the cldevicegroup command to retry the attempt to start the device group. If the failure occurred during the 'primary_to_secondary' transition, then the shutdown or switchover of a device group has failed. The desired action may be retried.

`670799 CMM: Registering reservation key on quorum device %s failed with error %d.`

Description: An error was encountered while trying to place the local node's reservation key on the specified quorum device. This node will ignore this quorum device.

Solution: There might be other related messages on this and other nodes that are connected to this quorum device that might indicate the cause of this problem. Refer to the quorum disk repair section of the administration guide for resolving this problem.

`670814 Blocking in RGM`

Description: The RGM is blocked until all cluster file systems are available.

Solution: This is an informational message, no user action is needed.

`671376 cl_apid internal error: unable to update client registrations.`

Description: The cl_apid experienced in internal error that prevented it from modifying the client registrations as requested.

Solution: Examine other syslog messages occurring at about the same time to see if the problem can be identified. Save a copy of the /var/adm/messages files on all nodes and contact your authorized Oracle service provider for assistance in diagnosing and correcting the problem.

`671394 start_oep_proxy - Incorrect PLUGIN_NAME [%s] specified`

Description: The PLUGIN_NAME is not known.

Solution: Enter a valid PLUGIN_NAME.

`671553 Unsupported task ${task} in ${POSTCONFIGFILE}`

Description: A task was found on the postconfiguration CCR file that was not understood. This probably indicated that the CCR file was corrupt.

Solution: Manually configure quorum disks and initialize the cluster or set the resource_security property by using clsetup after all nodes have joined the cluster.

`671561 start_ohs - Start command (%s) timed-out.`

Description: The Oracle HTTP Server failed to start within the START_TIMEOUT period.

Solution: Increase the value of the START_TIMEOUT property.

`671602 pid %u <%s> project <%s> user <%s> setproject() success after %d retry`

Description: A warning message.

Solution: None.

`671793 Could not stop telemetry data service monitor (resource: %s)`

Description: The monitor program of the telemetry data service could not stop. This should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`671954 waitpid: %s`

Description: The rpc.pmfd or rpc.fed server was not able to wait for a process. The message contains the system error. The server does not perform the action requested by the client, and an error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`672016 stop_asm - Oracle ASM instance (%s) shutdown abort successful`

Description: The Oracle ASM instance was successfully shutdown abort

Solution: None required. Informational message.

`672264 Ifconfig proxy server Killed(sigterm) Exiting.`

Description: The ifconfig program got a SIGTERM signal and is exiting.

Solution: This is an informational message. No user action is required.

`672525 Resource <%s> of Resource Group <%s> failed monitor check on node <%s>`

Description: Message logged for failed scha_control monitor check methods on specific node.

Solution: No user action required.

`673574 Error in trying to retrieve the configured network resources : %s.`

Description: Error trying to retrieve network address associated with a resource.

Solution: Create a network address resource referenced by the resource_dependencies property.

`673965 INTERNAL ERROR: starting_iter_deps_list: invalid dependency type <%d>`

Description: A non-fatal internal error has occurred in the rgmd state machine.

Solution: Since this problem might indicate an internal logic error in the rgmd, save a copy of the /var/adm/messages files on all nodes, and the output of clresourcetype show -v, clresourcegroup show -v +, and clresourcegroup status +. Report the problem to your authorized Oracle service provider.

`674163 Failfast: timeout - unit "%s"%s.`

Description: A failfast client has encountered a timeout and is going to panic the node.

Solution: There may be other related messages on this node which may help diagnose the problem. Resolve the problem and reboot the node if node panic is unexpected.

`674214 rebalance: no primary node is currently found for resource group <%s>.`

Description: The rgmd is currently unable to bring the resource group online because all of its potential masters are down or are ruled out because of strong resource group affinities, repeated start method failures, or hard loadlimit violations.

Solution: Examine the syslog output on all nodes to find out why the resource group failed to come online. As a cluster reconfiguration is completed, additional nodes become eligible and the resource group may go online without the need for any user action. If the resource group remains offline, repair and reboot broken nodes so they can rejoin the cluster; or use clresourcegroup to edit the Nodelist or RG_affinities property of the resource group, so that the Nodelist contains nodes that are cluster members and on which the resource group's strong affinities can be satisfied. If repeated Start or Prenet_start method failures are occurring, check for correct configuration of the data service. If the resource group is excluded by load limits, you might enable the resource group to come online if you decrease the resource group's loadfactors, increase its priority, increase the loadlimits of the nodes, or by taking other services offline.

`674359 load balancer deleted`

Description: This message indicates that the service group has been deleted.

Solution: This is an informational message, no user action is needed.

`674415 svc_restore_priority: %s`

Description: The rpc.pmfd or rpc.fed server was not able to run the application in the correct scheduling mode, and the system error is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`674848 fatal: Failed to read CCR`

Description: The rgmd is unable to read the cluster configuration repository. This is a fatal error. The rgmd will produce a core file and will force the node to halt or reboot to avoid the possibility of data corruption.

Solution: Save a copy of the /var/adm/messages files on all nodes, and of the rgmd core file. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`675353 Stop Proxy monitor using pmfadm failed. tag %s error=%s`

Description: The agent's attempt to stop the Monitor whose internal id is %s of a resource of type SUNW.scalable_rac_server_proxy failed with error %s.

Solution: Contact your Oracle support representative for further assistance.

`675360 clexecd: %s: sigaddset returned %d while adding signal %d. Exiting.`

Description: clexecd program has encountered a failed sigaddset(3C) system call. The error message indicates the error number for the failure while adding SIGHUP.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`675432 pmf_monitor_suspend: poll: %s`

Description: The rpc.pmfd server was not able to monitor a process. and the system error is shown. This error occurred for a process whose monitoring had been suspended. The monitoring of this process has been aborted and can not be resumed.

Solution: Save the syslog messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`675480 Could not create zone cluster %s due to CCR exception %s`

Description: libzccfg was not able to create the specified zone cluster due to an internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`675508 The SUNW.qfs resource '%s' that is configured for zone cluster '%s' does not have dependency on SUNW.wait_zc_boot resource. This might affect the functionality of SUNW.qfs resource.`

Description: The zone boot handler detected an invalid configuration, where a SUNW.qfs resource that is configured for a zone cluster is not having dependency on SUNW.wait_zc_boot resource. This will affect the functionality of SUNW.qfs resource.

Solution: Correct the configuration by adding required depedency. dependency.

`675748 clq_scsi3: Ioctl to register node's key returned success but read check found that key is not present on quorum device %s.`

Description: Ioctl to register node's key on specified quorum device returned success but read check found that key is not present. This error occurs because of unexpected behavior from the device driver or multipathing driver or device's firmware.

Solution: Please look for updates in device driver, multipathing driver or device firmware. If no updates are available contact Oracle device driver support team.

`675776 Stopped the fault monitor.`

Description: The fault monitor for this data service was stopped successfully.

Solution: No action needed.

`676370 asmdg_proxy_daemon - restarting to include the new Proxy_probe_timeout (%s)`

Description: Restarting as the Proxy_probe_timeout has changed.

Solution: None required. The ASM proxy daemon will be restarted to ensure the new Proxy_probe_timeout value is used.

`676558 WARNING: Global_resources_used property of resource group <%s> is set to non-null string, assuming wildcard`

Description: The Global_resources_used property of the resource group was set to a specific non-null string. The only supported settings of this property in the current release are null ("") or wildcard ("*").

Solution: No user action is required; the rgmd will interpret this value as wildcard. This means that method timeouts for this resource group will be suspended while any device group temporarily goes offline during a switchover or failover. This is usually the desired setting, except when a resource group has no dependency on any global device service or pxfs file system.

`676632 Cannot get the property %s of resource %s.`

Description: The scha_resource_get call failed.

Solution: Check the syslog for further messages.

`676787 SCSLM munmap <%s> error <%s>`

Description: Should never occur.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`677273 cl_execd: Unable to create thread. Exiting.`

Description: The cl_execd program encountered a failure while executing thr_create(3C).

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`677278 No network address resource in resource group.`

Description: A resource has no associated network address.

Solution: For a failover data service, add a network address resource to the resource group. For a scalable data service, add a network resource to the resource group referenced by the RG_dependencies property.

`677428 %s can't UP %s.`

Description: This means that the Logical IP address could not be set to UP.

Solution: There could be other related error messages which might be helpful. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`677759 Unknown status code %d.`

Description: Checking for HAStoragePlus resources returned an unknown status code.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`677788 Cannot determine the node name`

Description: Internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`678073 IPMP group %s is incapable of hosting %s addresses`

Description: The cluster is undergoing a reconfiguration.

Solution: None. This is an informational message only.

`678116 The lofs file system with mount point %s configured to zone cluster %s has the 'cluster-control' property value '%s'. The lofs file systems with 'cluster-control' property value '%s' will not be managed by the cluster.`

Description: Self explanatory.

Solution: You can either set the "cluster-control" property value to 'true' or remove the file system mount point from the resource configuration and repeat the operation.

`678290 The output of the command \'%s\' was not as expected.`

Description: The output of 'metastat' was not as expected.

Solution: Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`678648 reservation notice(%s) - MHIOCTKOWN success during retry attempt: %d`

Description: Informational message from reserve on ioctl success during retry.

Solution: No user action required.

`678985 cl_execd: dup2 of stdout returned with errno %d while exec'ing (%s). Exiting.`

Description: cl_execd program has encountered a failed dup2(2) system call. The error message indicates the error number for the failure.

Solution: The cl_execd program will exit and the node will be halted to prevent data corruption. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`679060 Internal error, trying to add a string to an invalid string array.`

Description: An internal programming error has been detected.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`679264 Starting auto recovery of HADB database.`

Description: All the Oracle Solaris Cluster nodes able to run the HADB resource are running the resource, but the database is unable to be started. The database will be reinitialized by running hadbm clear and then the command, if any, specified by the auto_recovery_command extension property.

Solution: This is an informational message, no user action is needed.

`679595 oep_validate - ONS_NODES=%s is invalid, ensure ONS_NODES=<node>:<port>[,<node>:<port>]`

Description: ONS_NODES is invalid

Solution: Ensure that ${ONS_NODES} has a node:port syntax.

`679874 Unable to execvp %s: %s`

Description: The libclcontract code was not able to exec the specified process, possibly due to bad arguments. The message contains the system error. The server process (which is using libclcontract) does not perform the action requested by the client, and an error message is output to syslog.

Solution: Verify that the file path to be executed exists. If all looks correct, save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`680088 The entry in mnttab for mount point '%s' has too few fields.`

Description: The entry in mnttab for the specified mount point has fewer fields than are defined for a single mnttab entry.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`680960 Unable to write data: %s.`

Description: Failed to write the data to the socket. The reason might be expiration of timeout, hung application or heavy load.

Solution: Check if the application is hung. If this is the case, restart the appilcation.

`681520 dl_attach: DL_OK_ACK protocol error`

Description: Could not attach to the private interconnect interface.

Solution: Reboot of the node might fix the problem.

`681723 Failed to create non-existent mount point %s: %s.`

Description: HAStoragePlus detected that the mount point was missing and failed to create the mount point.

Solution: Check the error message. Rectify the problem and retry the operation. If the problem persists, contact your authorized Oracle service provider.

`681919 validate_sapstartsrv - %s You must set the clustered sapstartsrv Instance_name per-node extension_property for the node that was just added before you bring online the resource group on the newly added node`

Description: The sapstartsrv Instance_name per-node extension property needs to be defined.

Solution: When adding a node to the resource group, the sapstartsrv Instance_name per-node extension property is set to the default value, which might not be valid for this deployment. Instead, if you required a different value from the default value, set the sapstartsrv Instance_name per-node extension property for the newly added node. To set the sapstartsrv Instance_name extension property, use the following command replacing <added-node>, <sid> and <resource> as required. clrs set -p 'Instance_name{<added-node>}'=<instance_name> <resource>

`682214 fencing fatal error(%s) - excecution not allowed in global cluster.`

Description: The zone cluster fencing program has suffered an internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available. Copies of /var/adm/messages from all nodes should be provided for diagnosis.

`683231 Could not start server.`

Description: The Oracle instance could not be started by the agent.

Solution: Check the /var/adm/messages file for details of the error. If this does not provide sufficient information, refer to the alert log file of the Oracle instance for details on the error. Contact your Oracle support representative for further assistance.

`683248 sysevent_subscribe_event() for %s failed.`

Description: Failed to subscribe to the specified event.

Solution: Save a copy of the /var/adm/messages file of this node and contact your authorized Oracle service provider for assistance in diagnosing the problem.

`683391 skip path = %s: I/O probe already running`

Description: Self explanatory.

Solution: No action required.

`683997 Failed to retrieve the resource group property %s: %s`

Description: Unable to retrieve the resource group property.

Solution: For the property name and the reason for failure, check the syslog message. For more details about the api failure, check the syslog messages from the RGM .

`684056 Failed to build path to status message file.`

Description: An error occurred when the path to the status message file was being built.

Solution: Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`684625 clq_scsi3: There are spurious keys on the device %s. Use a different device`

Description: There are spurious keys on the device. This is unexpected behavior from the device driver, multipathing driver or device firmware.

Solution: Use a different device as a quorum device.

`684753 store_binding: <%s> bad bind type <%d>`

Description: During a name server binding store an unknown binding type was encountered.

Solution: No action required. This is informational message.

`684810 svc_restore_priority: Could not set required scheduling parameters: %s. Diagnostic information (%s).`

Description: The server was not able to restore the original scheduling mode. The system error message is shown. An error message is output to syslog.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`685155 default port:<%d> will be used to probe ma`

Description: While parsing the mgt.cfg configuration file ,the ma server port field was not found or an error occured in parsing the port field within the file.

Solution: This is an informational message, no user action is needed. The ma server will start up with default port.

`685247 zc_incn(%d) in zone '%s' could not initialize clconf. Exiting.`

Description: The zc_incn running in a zone cluster failed to initialize clconf. Support for the zone cluster might not work properly.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`685270 Failed to retrieve the resource handle for (%s in cluster %s): %s.`

Description: Unable to get a handle to this resource's configuration data.

Solution: Internal error. Contact user support for assistance.

`685816 scds_get_rs_resource_dependencies() failed.`

Description: A call to scds_get_rs_resource_dependencies() failed.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`685886 Failed to communicate: %s.`

Description: While determining the health of the data service, fault monitor is failed to communicate with the process monitor facility.

Solution: This is internal error. Save /var/adm/messages file and contact your authorized Oracle service provider. For more details about error, check the syslog messges.

`686167 Incorrect resource group properties for VUCMM framework resource group %s. Value of Maximum_primaries(%s) is not equal to number of nodes in the Nodelist property(%s).`

Description: The VUCMM framework resource group is a scalable resorce group. The value of the Maximum_primaries and Desired_primaries properties should be set to the number of nodes specified in the Nodelist property. VUCMM framework will not function correctly without these values.

Solution: Specify the correct values of the properties and reissue the command. Refer to the Oracle Solaris Cluster documentation for procedures to specify the properties of a scalable resource group.

`686419 %s: Unblock parent, write errno %d`

Description: Internal error.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`686433 validate_instance - %s You must set the clustered SAP instance Instance_name per-node extension_property for the node that was just added before you bring online the resource group on the newly added node`

Description: The SAP instance Instance_name per-node extension property needs to be defined.

Solution: When adding a node to the resource group, the SAP instance Instance_name per-node extension property is set to the default value, which might nott be valid for this deployment. Instead, if you required a different value from the default value, set the SAP instance Instance_name per-node extension property for the newly added node. To set the SAP instance Instance_name extension property, use the following command replacing <added-node>, <sid> and <resource> as required. clrs set -p 'Instance_name{<added-node>}'=<instance_name> <resource>

`687198 Validate_sbp - MySQL resource group is not specified in option %s`

Description: The required real mysql resource group is not specified.

Solution: Make sure that the required resource group is specified.

`687457 Attempting to kill pid %d name %s resulted in error: %s.`

Description: HA-NFS callback method attempted to stop the specified NFS process with SIGKILL but was unable to do so because of the specified error.

Solution: The failure of the method would be handled by Oracle Solaris Cluster. If the failure happened during starting of a HA-NFS resource, the resource would be failed over to some other node. If this happened during stopping, the node would be rebooted and HA-NFS service would continue on some other node. If this error persists, please contact your local Oracle service provider for assistance.

`687496 fatal: cannot send mappings to remote nodes`

Description: This node encountered an unexpected error while communicating with other cluster nodes during a cluster reconfiguration. The rgmd will produce a core file and will cause the node to halt or reboot.

Solution: Save a copy of the /var/adm/messages files on all nodes, and of the rgmd core file. Contact your authorized Oracle service provider for assistance in diagnosing the problem.

`687543 shutdown abort did not succeed.`

Description: HA-Oracle failed to shutdown Oracle server using 'shutdown abort'.

Solution: Examine log files and syslog messages to determine the cause of failure.

`687580 Unable to get zonename for zone id %d, errno = %d. Exiting\n.`

Description: The pnm proxy program encountered a failure when trying to get the zone name.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`687654 validate_common - Psft_Home %s either does not exist or is not a directory`

Description: ${Psft_Home} either does not exist or is not a directory.

Solution: Ensure that ${Psft_Home} represents the directory where the PeopleSoft Enterprise files are installed.

`687672 Cannot import '%s' : pool may be in use from other system, it was last accessed by '%s'.`

Description: Self explanatory.

Solution: Check on the specified cluster node that has imported the specified pool. Try to export the zpool by issuing "zpool export -f <poolname>" and re-try the operation. If the problem persists, contact your authorized Oracle service provider.

`687929 daemon %s did not respond to null rpc call: %s.`

Description: HA-NFS fault monitor failed to ping an nfs daemon.

Solution: No action required. The fault monitor will restart the daemon if necessary.

`688163 clexecd: pipe returned %d. Exiting.`

Description: clexecd program has encountered a failed pipe(2) system call. The error message indicates the error number for the failure.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`688262 Failover of the derby data service failed.`

Description: The failover attempt of the derby data service was rejected or encountered an error.

Solution: For more detailed error message, check the syslog messages. Check whether the Pingpong_interval has appropriate value. If not, adjust it by using clresourcegroup show. Otherwise, use clresourcegroup switch to switch the resource group to a healthy node.

`688990 Giveover attempt failed for %s: resource group is unable to bring online on a potential primary node. retries exhausted.`

Description: An error occurred while attempting to switch over the QFS meta data resource group. This error persisted until all retries were exhausted.

Solution: Verify that the cluster is correctly configured. If it is, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`689232 validate_common - The configured Psft_User %s has uid 0 and would run as root user. You must run the application as a non-root user.`

Description: The configured Psft_User has uid 0 and would run as the user root. You must run the application as a non-root user.

Solution: Ensure that the configured Psft_User does not have uid 0.

`689538 Listener %s did not stop.(%s)`

Description: Failed to stop Oracle listener using 'lsnrctl' command. HA-Oracle will attempt to kill listener process.

Solution: None

`689677 can't start cl_pnmd due to lock.`

Description: An attempt was made to start multiple instances of the PNM daemon cl_pnmd(1M), or cl_pnmd(1M) has problem acquiring a lock on the file (/var/cluster/run/pnm_lock).

Solution: Check if another instance of cl_pnmd is already running. If not, remove the lock file (/var/cluster/run/pnm_lock) and start cl_pnmd by sending KILL (9) signal to cl_pnmd. PMF will restart cl_pnmd automatically.

`690209 Membership: State machine for '%s' related to zone cluster '%s' entered an invalid state %s; previous state was %s, previous event was %s.`

Description: The membership state machine for this membership related to a zone cluster has entered an invalid state.

Solution: Shut down the zone cluster and then boot it back. This series of actions will clear the immediate problem. Also contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`690417 Protocol is missing in system defined property %s.`

Description: The specified system property does not have a valid format. The value of the property must include a protocol.

Solution: Use clresource to specify the property value with protocol. For example: TCP.

`690463 Cannot bring server online on this node.`

Description: Oracle server is running but it cannot be brought online on this node. START method for the resource has failed.

Solution: Check if Oracle server can be started manually. Examine the log files and setup. Clear START_FAILED flag on the resource and bring the resource online.

`690846 Function: validate: ERROR_ON_SHOW is not numeric or 0`

Description: In the parameter file, there is a non numeric character in the value for the ERROR_ON_SHOW variable.

Solution: Fix the ERROR_ON_SHOW variable in the parameter file.

`691005 retval = %d)`

Description: The task execution by the cl_execd interface failed.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`691323 WARNING: Oracle Solaris Cluster boot: Could not load module $m, modload returns $res`

Description: A kernel module for RDS/SDP failed to load.

Solution: If RDS/SDP is required by the application running on the cluster, contact your Oracle service representative to determine if a workaround or patch is available.

`691489 Fault with resource detected. Disabling resource %s.`

Description: The fault monitor has detected an unrecoverable fault with the specified resource. As a result, the fault monitor issues a request to the RGM to disable the resource.

Solution: No user action required.

`691493 One or more of the SUNW.HAStoragePlus resources that this resource depends on is in a different resource group. Failing validate method configuration checks.`

Description: The HAStoragePlus resource that this resource depends on must be configured into the same resource group.

Solution: Move the HAStoragePlus resource into this resource's resource group.

`692466 Error while unbinding 'rtreg_proxy_server' in the name server. Exiting.`

Description: The rtreg server program could not register the corba object in the name server.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`692486 Could not initialize the ORB. Exiting.`

Description: The rtreg server program was unable to initialize its interface to the low-level cluster machinery.

Solution: Make sure the nodes are booted in cluster mode. If so, contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`692939 CMM: Unable to get reference to membership engine.`

Description: An instance of the userland CMM encountered an internal initialization error. The userland CMM could not obtain a reference to the membership engine object.

Solution: Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`693579 stop_mysql - Failed to flush MySql logfiles for %s`

Description: mysqladmin command failed to flush MySQL logfiles.

Solution: Either was MySQL already down or the faultmonitor user does not have the right permission to flush logfiles. The defined faultmonitor should have Process, Select, Reload, and Shutdown privileges and for MySQL 4.0.x also Super privileges.

`694012 Error getting the zonepath for the zone name %s: %s.`

Description: There was an error getting the zonepath for the zone.

Solution: For specific error info, check the syslog message.

`694561 An error occurred while reading the /etc/vfstab file of global zone for file system mount point %s in zone cluster %s. This file system could be original file system for the lofs file system with mount point %s configured to zone cluster %s.`

Description: An error occurred while the vfstab entry for the specified mount point was being read.

Solution: Verify that the vfstab entry for the mount point is correct and repeat the operation.

`695340 scf_handle_bind failed: %s`

Description: An API call failed.

Solution: Examine log files and syslog messages to determine the cause of the failure. Take corrective action based on any related messages. If the problem persists, report it to your Oracle support representative for further assistance.

`695728 Skipping checks dependant on HAStoragePlus resources on this node.`

Description: This resource will not perform some filesystem specific checks (during VALIDATE or MONITOR_CHECK) on this node because atleast one SUNW.HAStoragePlus resource that it depends on is online on some other node.

Solution: None.

`695798 start_diskgroup - Oracle ASM instance (%s) has successfully mounted diskgroup (%s)`

Description: The Oracle ASM instance has successfully mounted the diskgroup.

Solution: None required. Informational message.

`696051 Not attempting to start resource group <%s> on node <%s>, because the resource groups <%s> for which it has strong negative affinities are online.`

Description: The rgmd is enforcing the strong negative affinities of the resource groups. This behavior is normal and expected.

Solution: No action required. If desired, use clresourcegroup to change the resource group affinities.

`696463 rgm_clear_util called on resource <%s> with incorrect flag <%d>`

Description: An internal rgmd error has occurred while attempting to carry out an operator request to clear an error flag on a resource. The attempted clear action will fail.

Solution: Since this problem might indicate an internal logic error in the rgmd, save a copy of the /var/adm/messages files on all nodes, and the output of clresourcetype show -v, clresourcegroup show -v +, and clresourcegroup status +. Report the problem to your authorized Oracle service provider.

`696789 Error: Oracle Solaris Cluster software does not support transitioning from run level 3 to levels S (single-user), 1, or 2; halting`

Description: A transition which is not allowed by Oracle Solaris Cluster software has been requested. Oracle Solaris Cluster software does not permit transition from run level 3 to run level 1, 2 or S. The node is halted.

Solution: Reboot the node.

`696903 Failed to configure the private IP address for a zone`

Description: The private IP address assigned to a zone was not configured correctly.

Solution: The private IP address assigned to the zone is used for private IP communication for the zone. So, this feature may not work as expected as a result of this error. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`697026 did instance %d created.`

Description: Informational message from scdidadm.

Solution: No user action required.

`697108 t_sndudata in send_reply failed.`

Description: Call to t_sndudata() failed. The "t_sndudata" man page describes possible error codes. ucmmd will exit and the node will abort.

Solution: Save the /var/adm/messages file. Contact your authorized Oracle service provider to determine whether a workaround or patch is available.

`697334 Memory allocation failure.`

Description: Unable to allocate memory.

Solution: No user action required.

`697340 Giveover attempt failed: no other node is currently available to host the resource group. Should be OK as no other node should be accessing the QFS file system if this is the last MDS node.`

Description: No other node found that can host this RG. This is Ok, since this is the last node sharing the QFS if this is a properly configured cluster. Just let Solaris unmount the QFS file system.

Solution: Informational message.

`697691 Resource state of %s is changed to Offline. Note that the VUCMM framework will not be stopped by STOP method.`

Description: The stop method of the resource was called by the resource-group manager. The stop method is called in the following conditions: - Disabling a resource - Changing the state of the resource group to bring a node offline - Shutting down a cluster If the VUCMM framework was running prior to calling the stop method, it will continue to run even if the resource state is changed to Offline.

Solution: If you want to stop the VUCMM framework on the node, you might need to reboot the node.

`698258 Function: validate - Failied to get serivce state for %s`

Description: The SMF service state for %s is not known.

Solution: Review the /opt/SUNWsczone/sczsmf/util/sczsmf_config configuration file and make sure you have specified the correct SMF service name.

`698272 Probing thread for mountpoint %s is hanging for timeout period %d seconds`

Description: The test for performing I/O operations is hanging. One possible reason for this could be network connectivity failure.

Solution: Check the network connectivity and restart the ScalMountPoint resource.

`698587 sig2str(3C) failed for pmmd_adm while converting signal number %d to a signal name.`

Description: pmmd_adm was unable to convert a valid signal number to its signal name, because the sig2str(3C) function failed.

Solution: Save a copy of the /var/adm/messages files on this node, and report the problem to your authorized Oracle service provider.

`699104 VALIDATE failed on resource <%s>, resource group <%s>, time used: %d%% of timeout <%d, seconds>`

Description: The resource's VALIDATE method exited with a non-zero exit code. This indicates that an attempted update of a resource or resource group is invalid.

Solution: Examine syslog messages occurring just before this one to determine the cause of validation failure. Re-try the update.

`699167 The status of the service %s is in maintenance.`

Description: The specified SMF service is in maintenance.

Solution: Determine the reason to be in maintenance by using svcs(1) and by checking /var/adm/messages. Re-try the operation after the problem is resolved.

`699413 PMF is restarting tag <%s>, cmd_path="%s"`

Description: The process with the given tag failed to stay up and exceeded the allowed number of retry attempts (given by the 'pmfadm -n' option), and the action (given by the 'pmfadm -a' option) was initiated by rpc.pmfd. The action succeeded (i.e., returned zero), and rpc.pmfd restarted the exited process. For more information, see pmfadm(1M).

Solution: This message is informational; no user action is needed.

Copyright © 2000, 2012, Oracle and/or its affiliates. All rights reserved. Legal Notices |