Much appreciated....
Based on this, it seems that you don't have complete JDK installed on
this system since installer is running with following java.home:
C:\Program Files\Java\jre1.5.0_13
This is JRE and not full JDK. Since JRE does not contain keytool utility
required for initial domain configuration, this installation step fails.
So, in order to install and use Glassfish v3, you will first need to
download and install either JDK 5 or JDK 6 which you can get here:
http://java.sun.com/javase/downloads/index.jsp
Once you installed JDK, you can simply delete your existing installation
directory and rerun the installation.