It appears that the shell might be eating your %. I tried the following
in bash on Linux, and it works fine:
$ asadmin create-auth-realm --classname
com.sun.enterprise.security.auth.realm.ldap.LDAPRealm --property
"directory=ldap\://localhost\:389:base-dn=dc.example:jaas-context=ldapRealm:search-filter=cn\=%s"
myrealm
Are you using Windows? On Windows, the % sign is used for variable
substitution in the shell.
You might want to try multimode to enter the command, i.e., enter:
asadmin multimode
This will then prompt for a command to enter, i.e., create-auth-realm ...
Tom