Note: This is an archival copy of Security Sun Alert 246846 as previously published on http://sunsolve.sun.com.|
Latest version of this security advisory is available from http://support.oracle.com as Sun Alert 1019819.1.
Solaris 10 Operating System
Date of Resolved Release
A Security Vulnerability in the OpenSSL PKCS#11 Engine May Result in Denial of Service (DoS) Due to a Corrupted Session Cache
A security vulnerability in the OpenSSL PKCS#11 engine as shipped with Solaris 10 may affect applications which make use of this engine. The exact impact will vary depending on the application. This vulnerabiltiy may allow a local or remote unprivileged user to cause certain cryptographic operations within the application to fail (namely RSA_sign and RSA_verify), which is a type of Denial of Service (DoS).
For example, if the OpenSSL PKCS#11 engine is used for SSL processing (for example, in the Apache webserver) it will result in SSL connections being dropped in the SSL handshake phase, thereby causing a Denial Of Service (DoS) due to a corrupted session cache.
2. Contributing Factors
This issue can occur in the following releases:
1. Solaris 8 and 9 and OpenSolaris are not impacted by this issue since they do not ship with the OpenSSL PKCS#11 engine.
2. This issue only impacts applications making use of the OpenSSL PKCS#11 engine. The exact method for determining whether an application is making use of this functionality will vary depending on the application.
The symptoms will vary depending on the application. For example, if the OpenSSL PKCS#11 engine is used for SSL processing, failed SSL handshakes may be observed as in the following example (from the Apache debug log file):
[Thu Sep 06 08:28:05 2008] [info] [client 192.0.2.1]
Until patches can be applied, sites may wish to disable the OpenSSL PKCS#11 engine altogether or just for RSA/DSA/DH functions (if possible) so that crypto processing will be done only via native OpenSSL routines. This may degrade the performance depending on how the engine is used and on the hardware crypto providers present in the system.
The method for doing this will vary depending on the application, and the documentation should be consulted for more details. For example, to disable the OpenSSL PKCS#11 engine in the Apache web server (see apache(1M)), comment out the following line from the SSL configuration file and restart the Webserver:
For Apache server shipped with Solaris 10, this directive is configured in '/etc/apache/httpd.conf' or '/etc/apache2/ssl.conf'.
This issue is addressed in the following releases:
Copyright 2000-2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054 U.S.A. All rights reserved.
This solution has no attachment