The issue we are seeing is that if you install GlassFish Enterprise 2.1 with HADB on Windows XP professional with Service Pack 3 or Windows Vista, it is installed using the NSS libraries. GlassFish starts up fine. However the problem arises with the fact that you can run NSS in FIPS mode or not in FIPS mode. When you install it, it gets installed in non-FIPS mode. You use a program called modutil (located in the Sun\AppServer\lib directory) to set a flag in the NSS databases to tell it to operate in FIPS mode. The command is: "modutil -dbdir c:\Sun\Appserver\domains\domain1\config -fips true". When we try to run the modutil that is installed with GF Enterprise 2.1, it gets an error that is not real useful. It looks as follows:
Using database directory c:\Sun\Appserver\domains\domain1\config...
An I/O error occurred during security authorization.
ERROR: Unable to switch FIPS modes.
Other functions on that modutil work fine, just this one will not work. If we take the binaries from NSS and use modutil from there it successfully sets the database to FIPS, but when we try to start up GlassFish we get a different error.
We have looked at the source code for modutil, and it is not readily apparent what the error is. One option is that it may be tied to the specific version of windows. We noticed that the documentation for GL Enterprise 2.1 refers to Windows NT 4.0. But the download does not require us to specify which version of Windows we are using. Is this windows specific and does that mean it is not supported on Windows XP or Vista?
Thanks in advance for any advice on this...
[Message sent by forum member 'leswestberg' (les.westberg_at_agilex.com)]
http://forums.java.net/jive/thread.jspa?messageID=367117