users@glassfish.java.net

Re: glassfish 3.1.2.2 (build 5) - default install - problem with keeping too many connections in the queue, while system usage is low

From: Mladen Adamovic <mladen.adamovic_at_gmail.com>
Date: Fri, 20 Jul 2012 11:53:40 +0200

Hi Oleksiy,

unfortunatelly, I haven't used netstat to test that, I assumed that by
looking what is happening and by using glassfish monitoring. Perhaps I
was wrong, but I needed to describe what is going on, and I was tried
my best.

Later during the day I was curious what is different in config at that
webserver comparing to another webserver, and changed
httpthreadpool Max Thread Pool Size: 400
default value is 5.
this value of 400 threads, was choosen so that webserver hopefully
doesn't run out of 1024 network descriptors it has most.

I'd like to bring you to your attention the fishy value
Network Listeners -> http-listener-1 of 250 max connections.
The text near max connections states: Maximum number of connections in
keep-alive mode
but it is actually according to Oracle.com documentation at the
website http://docs.oracle.com/cd/E19879-01/820-4343/abefr/index.html
"Max Connections controls the number of requests that a particular
client can make over a keep-alive connection. "

So glassfish console text "Maximum number of connections in keep-alive
mode" seems incorrect, it shall be written "Maximum number of requests
per connection in keep-alive mode", which is different. Perhaps
someone can check in the source code what does that value actually do
and change the description in admin console and online doc so it's not
ambiguous?

If it's actually the number of connections in keep-alive mode,
keep-alive connections would take 250 threads and only 5 is provided
by default config.

With max threads set to 400,

after half day monitoring
Jul 19, 2012 6:08:48 PM Jul 20, 2012 9:42:38 AM

By looking into monitoring now, I can see that many connections looks queued
CountQueued15MinutesAverage 6218 count
CountQueued1MinuteAverage 294 count
CountOpenConnections 62 count
CountTotalConnections 153786 count
CurrentThreadCount 66 count

but still the load at the server is very low:
# uptime
 11:43:23 up 16:06, 1 user, load average: 0.02, 0.06, 0.00

Could you explain me why so many connections have been queued in last
15 minutes and load at the server has been 0.06? (Virtuzzo VPS with
2x1.5 GHz guaranteed).
Does make sense unless OpenConnections were MaxConnections in the past
15 minutes? Or perhaps due to the fact I use only 1 acceptor thread?

Processing time is very low ProcessingTime 3 millisecond
but MaxTime is 3942 millisecond

Again, still, why so many connections hav been queued in last 15
minutes and load at the server was 0.06? Looks fishy!
Perhaps Google uses keep alive connections and max number of keep
alive connections is indeed that (not requests per connection), so
some Googlebot's simultanously keep-alive connection end up in the
queue.
CountHits 294527 count for a half day.

Regards


On Fri, Jul 20, 2012 at 10:59 AM, Oleksiy Stashok
<oleksiy.stashok_at_oracle.com> wrote:
> Hi Mladen,
>
> by "connections wait in connection queue" you mean statistics you get using
> netstat with many connections in WAIT state?
> If you run GF 2.1 on the same environment you don't see this?
>
> Thanks.
>
> WBR,
> Alexey.
>
>
> On 07/19/2012 08:34 PM, Mladen Adamovic wrote:
>>
>> Hi guys,
>>
>> I migrated some of my websites from glassfish 2.1 to glassfish 3.1.2.2
>> and I'm having problems.
>>
>> The problem I'm encountering is that after a while, many connections
>> wait in connection queue and it even starts to refuse connections, but
>> system usage is low.
>> #uptime
>> 20:28:58 up 51 min, 1 user, load average: 0.09, 0.14, 0.24
>>
>> If I test the website with #ab -n 1000 -c 5
>> http://www.numbeo.com/cost-of-living/
>> it gets around 980 failures out of 1000.
>> I'm playing with config files, but couldn't figure out where is the
>> problem.
>> ulimit -n
>> is 1024,cannot rise it, since running in Virtuozzo enviroment.
>>
>> These are parts of my domain.xml:
>>
>> ...
>> <config name="server-config">
>> <http-service>
>> <access-log></access-log>
>> <virtual-server id="numbeo.com" hosts="numbeo.com"
>> default-web-module="NonWwwNumbeo" sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="www.numbeo.com" hosts="www.numbeo.com,
>> ww2.numbeo.com" default-web-module="LivingCost"
>> sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> </http-service>
>>
>> <protocol name="http-listener-1">
>> <http xpowered-by="false" default-virtual-server="server"
>> max-connections="250">
>> <file-cache></file-cache>
>> </http>
>> </protocol>
>>
>> <network-listener port="8080" protocol="http-listener-1"
>> transport="tcp" name="http-listener-1"
>> thread-pool="http-thread-pool"></network-listener>
>>
>> <thread-pool max-thread-pool-size="400"
>> name="http-thread-pool"></thread-pool>
>>
>>
>> <java-config debug-options="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009"
>> system-classpath="" classpath-suffix="">
>> <jvm-options>-XX:MaxPermSize=192m</jvm-options>
>> <jvm-options>-XX:PermSize=64m</jvm-options>
>> <jvm-options>-Djava.awt.headless=true</jvm-options>
>>
>> <jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder</jvm-options>
>> <jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
>>
>> <jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/modules/endorsed${path.separator}${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
>>
>> <jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
>>
>> <jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
>>
>> <jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext</jvm-options>
>>
>> <jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
>> <jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.port=6666</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.maxconn=1</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.ip=127.0.0.1</jvm-options>
>> <jvm-options>-Dgosh.args=--nointeractive</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.dir=${com.sun.aas.installRoot}/modules/autostart/</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.poll=5000</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.log.level=2</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.new.start=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.startTransient=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.disableConfigSave=false</jvm-options>
>> <jvm-options>-XX:NewRatio=2</jvm-options>
>> <jvm-options>-server</jvm-options>
>> <jvm-options>-Xmx2048m</jvm-options>
>> <jvm-options>-Xms1024m</jvm-options>
>> <jvm-options>-Dproduct.name=</jvm-options>
>> </java-config>
>>
>> Anyone knows what might be the problem?
>>
>>
>> -------------------------------------
>> And here is the full domain.xml (in the case I missed something):
>> <domain log-root="${com.sun.aas.instanceRoot}/logs"
>> application-root="${com.sun.aas.instanceRoot}/applications"
>> version="5">
>> <system-applications>
>> <application context-root=""
>>
>> location="${com.sun.aas.installRootURI}/lib/install/applications/__admingui"
>> name="__admingui" directory-deployed="true"
>> object-type="system-admin">
>> <module name="__admingui">
>> <engine sniffer="web"></engine>
>> <engine sniffer="security"></engine>
>> </module>
>> </application>
>> </system-applications>
>> <applications>
>> <application context-root="/OpenRewrite"
>> location="${com.sun.aas.instanceRootURI}/applications/OpenRewrite/"
>> name="OpenRewrite" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/OpenRewrite/OpenRewrite.war"></property>
>> <property name="preserveAppScopedResources"
>> value="false"></property>
>> <property name="defaultAppName" value="OpenRewrite"></property>
>> <module name="OpenRewrite">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> <application context-root="/DecisionCrowd"
>> location="${com.sun.aas.instanceRootURI}/applications/DecisionCrowd/"
>> name="DecisionCrowd" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/DecisionCrowd/DecisionCrowd.war"></property>
>> <property name="preserveAppScopedResources"
>> value="false"></property>
>> <property name="defaultAppName" value="DecisionCrowd"></property>
>> <property name="java-web-start-enabled" value="false"></property>
>> <module name="DecisionCrowd">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> <application context-root="/NonWwwNumbeo"
>> location="${com.sun.aas.instanceRootURI}/applications/NonWwwNumbeo/"
>> name="NonWwwNumbeo" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/NonWwwNumbeo/NonWwwNumbeo.war"></property>
>> <property name="preserveAppScopedResources"
>> value="false"></property>
>> <property name="defaultAppName" value="NonWwwNumbeo"></property>
>> <module name="NonWwwNumbeo">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> <application context-root="/NonWwwOpenRewrite"
>> location="${com.sun.aas.instanceRootURI}/applications/NonWwwOpenRewrite/"
>> name="NonWwwOpenRewrite" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/NonWwwOpenRewrite/NonWwwOpenRewrite.war"></property>
>> <property name="preserveAppScopedResources"
>> value="false"></property>
>> <property name="defaultAppName"
>> value="NonWwwOpenRewrite"></property>
>> <module name="NonWwwOpenRewrite">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> <application context-root="/NonWwwDecisionCrowd"
>>
>> location="${com.sun.aas.instanceRootURI}/applications/NonWwwDecisionCrowd/"
>> name="NonWwwDecisionCrowd" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/NonWwwDecisionCrowd/NonWwwDecisionCrowd.war"></property>
>> <property name="preserveAppScopedResources"
>> value="false"></property>
>> <property name="defaultAppName"
>> value="NonWwwDecisionCrowd"></property>
>> <property name="java-web-start-enabled" value="false"></property>
>> <module name="NonWwwDecisionCrowd">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> <application context-root="/LivingCost"
>> location="${com.sun.aas.instanceRootURI}/applications/LivingCost/"
>> name="LivingCost" object-type="user">
>> <property name="appLocation"
>>
>> value="${com.sun.aas.instanceRootURI}/applications/__internal/LivingCost/LivingCost.war"></property>
>> <property name="defaultAppName" value="LivingCost"></property>
>> <module name="LivingCost">
>> <engine sniffer="security"></engine>
>> <engine sniffer="web"></engine>
>> </module>
>> </application>
>> </applications>
>> <resources>
>> <jdbc-resource pool-name="__TimerPool"
>> jndi-name="jdbc/__TimerPool"
>> object-type="system-admin"></jdbc-resource>
>> <jdbc-resource pool-name="DerbyPool"
>> jndi-name="jdbc/__default"></jdbc-resource>
>> <jdbc-connection-pool
>> datasource-classname="org.apache.derby.jdbc.EmbeddedXADataSource"
>> res-type="javax.sql.XADataSource" name="__TimerPool">
>> <property name="databaseName"
>> value="${com.sun.aas.instanceRoot}/lib/databases/ejbtimer"></property>
>> <property name="connectionAttributes"
>> value=";create=true"></property>
>> </jdbc-connection-pool>
>> <jdbc-connection-pool is-isolation-level-guaranteed="false"
>> datasource-classname="org.apache.derby.jdbc.ClientDataSource"
>> res-type="javax.sql.DataSource" name="DerbyPool">
>> <property name="PortNumber" value="1527"></property>
>> <property name="Password" value="APP"></property>
>> <property name="User" value="APP"></property>
>> <property name="serverName" value="localhost"></property>
>> <property name="DatabaseName"
>> value="sun-appserv-samples"></property>
>> <property name="connectionAttributes"
>> value=";create=true"></property>
>> </jdbc-connection-pool>
>> </resources>
>> <servers>
>> <server name="server" config-ref="server-config">
>> <application-ref ref="__admingui"
>> virtual-servers="__asadmin"></application-ref>
>> <application-ref ref="OpenRewrite"
>> virtual-servers="www.openrewrite.com"></application-ref>
>> <application-ref ref="DecisionCrowd"
>> virtual-servers="www.decisioncrowd.com"></application-ref>
>> <application-ref ref="NonWwwNumbeo"
>> virtual-servers="numbeo.com"></application-ref>
>> <application-ref ref="NonWwwOpenRewrite"
>> virtual-servers="openrewrite.com"></application-ref>
>> <application-ref ref="NonWwwDecisionCrowd"
>> virtual-servers="decisioncrowd.com"></application-ref>
>> <application-ref ref="LivingCost"
>> virtual-servers="www.numbeo.com"></application-ref>
>> <resource-ref ref="jdbc/__TimerPool"></resource-ref>
>> <resource-ref ref="jdbc/__default"></resource-ref>
>> </server>
>> </servers>
>> <nodes>
>> <node node-host="localhost" name="localhost-domain1" type="CONFIG"
>> install-dir="${com.sun.aas.productRoot}"></node>
>> </nodes>
>> <configs>
>> <config name="server-config">
>> <http-service>
>> <access-log></access-log>
>> <virtual-server id="server"
>> network-listeners="http-listener-1,http-listener-2"></virtual-server>
>> <virtual-server id="__asadmin"
>> network-listeners="admin-listener"></virtual-server>
>> <virtual-server id="numbeo.com" hosts="numbeo.com"
>> default-web-module="NonWwwNumbeo" sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="www.numbeo.com" hosts="www.numbeo.com,
>> ww2.numbeo.com" default-web-module="LivingCost"
>> sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="openrewrite.com" hosts="openrewrite.com"
>> default-web-module="NonWwwOpenRewrite" sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="www.openrewrite.com"
>> hosts="www.openrewrite.com" default-web-module="OpenRewrite"
>> sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="decisioncrowd.com"
>> hosts="decisioncrowd.com" default-web-module="NonWwwDecisionCrowd"
>> sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> <virtual-server id="www.decisioncrowd.com"
>> hosts="www.decisioncrowd.com" default-web-module="DecisionCrowd"
>> sso-cookie-http-only="false"
>> network-listeners="http-listener-1"></virtual-server>
>> </http-service>
>> <iiop-service>
>> <orb use-thread-pool-ids="thread-pool-1"></orb>
>> <iiop-listener port="3700" id="orb-listener-1"
>> address="0.0.0.0" lazy-init="true"></iiop-listener>
>> <iiop-listener port="3820" id="SSL" address="0.0.0.0"
>> security-enabled="true">
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as"></ssl>
>> </iiop-listener>
>> <iiop-listener port="3920" id="SSL_MUTUALAUTH"
>> address="0.0.0.0" security-enabled="true">
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as" client-auth-enabled="true"></ssl>
>> </iiop-listener>
>> </iiop-service>
>> <admin-service system-jmx-connector-name="system"
>> type="das-and-server">
>> <jmx-connector port="8686" address="0.0.0.0"
>> security-enabled="false" auth-realm-name="admin-realm"
>> name="system"></jmx-connector>
>> <property name="adminConsoleContextRoot"
>> value="/admin"></property>
>> <property name="adminConsoleDownloadLocation"
>>
>> value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war"></property>
>> <property name="ipsRoot"
>> value="${com.sun.aas.installRoot}/.."></property>
>> <das-config></das-config>
>> </admin-service>
>> <connector-service></connector-service>
>> <web-container>
>> <session-config>
>> <session-manager>
>> <manager-properties></manager-properties>
>> <store-properties></store-properties>
>> </session-manager>
>> <session-properties></session-properties>
>> </session-config>
>> </web-container>
>> <ejb-container
>> session-store="${com.sun.aas.instanceRoot}/session-store">
>> <ejb-timer-service></ejb-timer-service>
>> </ejb-container>
>> <mdb-container></mdb-container>
>> <jms-service default-jms-host="default_JMS_host">
>> <jms-host host="localhost" name="default_JMS_host"></jms-host>
>> </jms-service>
>> <security-service>
>> <auth-realm
>> classname="com.sun.enterprise.security.auth.realm.file.FileRealm"
>> name="admin-realm">
>> <property name="file"
>> value="${com.sun.aas.instanceRoot}/config/admin-keyfile"></property>
>> <property name="jaas-context" value="fileRealm"></property>
>> </auth-realm>
>> <auth-realm
>> classname="com.sun.enterprise.security.auth.realm.file.FileRealm"
>> name="file">
>> <property name="file"
>> value="${com.sun.aas.instanceRoot}/config/keyfile"></property>
>> <property name="jaas-context" value="fileRealm"></property>
>> </auth-realm>
>> <auth-realm
>>
>> classname="com.sun.enterprise.security.auth.realm.certificate.CertificateRealm"
>> name="certificate"></auth-realm>
>> <jacc-provider
>> policy-provider="com.sun.enterprise.security.provider.PolicyWrapper"
>> name="default"
>> policy-configuration-factory-provider="com.sun.enterprise.security.provider.PolicyConfigurationFactoryImpl">
>> <property name="repository"
>> value="${com.sun.aas.instanceRoot}/generated/policy"></property>
>> </jacc-provider>
>> <jacc-provider
>>
>> policy-provider="com.sun.enterprise.security.jacc.provider.SimplePolicyProvider"
>> name="simple"
>> policy-configuration-factory-provider="com.sun.enterprise.security.jacc.provider.SimplePolicyConfigurationFactory"></jacc-provider>
>> <audit-module classname="com.sun.enterprise.security.Audit"
>> name="default">
>> <property name="auditOn" value="false"></property>
>> </audit-module>
>> <message-security-config auth-layer="SOAP">
>> <provider-config provider-type="client"
>> provider-id="XWS_ClientProvider"
>> class-name="com.sun.xml.wss.provider.ClientSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="dynamic.username.password"
>> value="false"></property>
>> <property name="debug" value="false"></property>
>> </provider-config>
>> <provider-config provider-type="client"
>> provider-id="ClientProvider"
>> class-name="com.sun.xml.wss.provider.ClientSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="dynamic.username.password"
>> value="false"></property>
>> <property name="debug" value="false"></property>
>> <property name="security.config"
>>
>> value="${com.sun.aas.instanceRoot}/config/wss-server-config-1.0.xml"></property>
>> </provider-config>
>> <provider-config provider-type="server"
>> provider-id="XWS_ServerProvider"
>> class-name="com.sun.xml.wss.provider.ServerSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="debug" value="false"></property>
>> </provider-config>
>> <provider-config provider-type="server"
>> provider-id="ServerProvider"
>> class-name="com.sun.xml.wss.provider.ServerSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="debug" value="false"></property>
>> <property name="security.config"
>>
>> value="${com.sun.aas.instanceRoot}/config/wss-server-config-1.0.xml"></property>
>> </provider-config>
>> </message-security-config>
>> <message-security-config auth-layer="HttpServlet">
>> <provider-config provider-type="server"
>> provider-id="GFConsoleAuthModule"
>>
>> class-name="org.glassfish.admingui.common.security.AdminConsoleAuthModule">
>> <request-policy auth-source="sender"></request-policy>
>> <response-policy></response-policy>
>> <property name="restAuthURL"
>>
>> value="https://localhost:${ADMIN_LISTENER_PORT}/management/sessions"></property>
>> <property name="loginPage" value="/login.jsf"></property>
>> <property name="loginErrorPage"
>> value="/loginError.jsf"></property>
>> </provider-config>
>> </message-security-config>
>> <property name="default-digest-algorithm"
>> value="SHA-256"></property>
>> </security-service>
>> <transaction-service
>> tx-log-dir="${com.sun.aas.instanceRoot}/logs"></transaction-service>
>> <java-config debug-options="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009"
>> system-classpath="" classpath-suffix="">
>> <jvm-options>-XX:MaxPermSize=192m</jvm-options>
>> <jvm-options>-XX:PermSize=64m</jvm-options>
>> <jvm-options>-Djava.awt.headless=true</jvm-options>
>>
>> <jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder</jvm-options>
>> <jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
>>
>> <jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/modules/endorsed${path.separator}${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
>>
>> <jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
>>
>> <jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
>>
>> <jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext</jvm-options>
>>
>> <jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
>> <jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.port=6666</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.maxconn=1</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.ip=127.0.0.1</jvm-options>
>> <jvm-options>-Dgosh.args=--nointeractive</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.dir=${com.sun.aas.installRoot}/modules/autostart/</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.poll=5000</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.log.level=2</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.new.start=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.startTransient=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.disableConfigSave=false</jvm-options>
>> <jvm-options>-XX:NewRatio=2</jvm-options>
>> <jvm-options>-server</jvm-options>
>> <jvm-options>-Xmx2048m</jvm-options>
>> <jvm-options>-Xms1024m</jvm-options>
>> <jvm-options>-Dproduct.name=</jvm-options>
>> </java-config>
>> <network-config>
>> <protocols>
>> <protocol name="http-listener-1">
>> <http xpowered-by="false" default-virtual-server="server"
>> max-connections="250">
>> <file-cache></file-cache>
>> </http>
>> </protocol>
>> <protocol security-enabled="true" name="http-listener-2">
>> <http xpowered-by="false" default-virtual-server="server"
>> max-connections="250">
>> <file-cache></file-cache>
>> </http>
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as"></ssl>
>> </protocol>
>> <protocol security-enabled="true" name="admin-listener">
>> <http xpowered-by="false"
>> default-virtual-server="__asadmin" max-connections="250"
>> encoded-slash-enabled="true">
>> <file-cache></file-cache>
>> </http>
>> </protocol>
>> <protocol security-enabled="true" name="sec-admin-listener">
>> <http default-virtual-server="__asadmin"
>> encoded-slash-enabled="true">
>> <file-cache></file-cache>
>> </http>
>> <ssl client-auth="want"
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as"></ssl>
>> </protocol>
>> <protocol name="admin-http-redirect">
>> <http-redirect secure="true"></http-redirect>
>> </protocol>
>> <protocol name="pu-protocol">
>> <port-unification>
>> <protocol-finder protocol="sec-admin-listener"
>> name="http-finder"
>> classname="com.sun.grizzly.config.HttpProtocolFinder"></protocol-finder>
>> <protocol-finder protocol="admin-http-redirect"
>> name="admin-http-redirect"
>> classname="com.sun.grizzly.config.HttpProtocolFinder"></protocol-finder>
>> </port-unification>
>> </protocol>
>> </protocols>
>> <network-listeners>
>> <network-listener port="8080" protocol="http-listener-1"
>> transport="tcp" name="http-listener-1"
>> thread-pool="http-thread-pool"></network-listener>
>> <network-listener port="8181" protocol="http-listener-2"
>> transport="tcp" name="http-listener-2"
>> thread-pool="http-thread-pool"></network-listener>
>> <network-listener port="4848" protocol="pu-protocol"
>> transport="tcp" name="admin-listener"
>> thread-pool="admin-thread-pool"></network-listener>
>> </network-listeners>
>> <transports>
>> <transport name="tcp"></transport>
>> </transports>
>> </network-config>
>> <thread-pools>
>> <thread-pool name="admin-thread-pool"
>> max-thread-pool-size="50" max-queue-size="256"></thread-pool>
>> <thread-pool max-thread-pool-size="400"
>> name="http-thread-pool"></thread-pool>
>> <thread-pool name="thread-pool-1"
>> max-thread-pool-size="200"></thread-pool>
>> </thread-pools>
>> <monitoring-service>
>> <module-monitoring-levels http-service="HIGH"
>> connector-connection-pool="HIGH" connector-service="HIGH" jvm="HIGH"
>> thread-pool="HIGH" web-container="HIGH"></module-monitoring-levels>
>> </monitoring-service>
>> <group-management-service>
>> <failure-detection></failure-detection>
>> </group-management-service>
>> </config>
>> <config name="default-config">
>> <http-service>
>> <access-log></access-log>
>> <virtual-server id="server"
>> network-listeners="http-listener-1, http-listener-2">
>> <property name="default-web-xml"
>> value="${com.sun.aas.instanceRoot}/config/default-web.xml"></property>
>> </virtual-server>
>> <virtual-server id="__asadmin"
>> network-listeners="admin-listener"></virtual-server>
>> </http-service>
>> <iiop-service>
>> <orb use-thread-pool-ids="thread-pool-1"></orb>
>> <iiop-listener port="${IIOP_LISTENER_PORT}"
>> id="orb-listener-1" address="0.0.0.0"></iiop-listener>
>> <iiop-listener port="${IIOP_SSL_LISTENER_PORT}" id="SSL"
>> address="0.0.0.0" security-enabled="true">
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as"></ssl>
>> </iiop-listener>
>> <iiop-listener port="${IIOP_SSL_MUTUALAUTH_PORT}"
>> id="SSL_MUTUALAUTH" address="0.0.0.0" security-enabled="true">
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as" client-auth-enabled="true"></ssl>
>> </iiop-listener>
>> </iiop-service>
>> <admin-service system-jmx-connector-name="system">
>> <jmx-connector port="${JMX_SYSTEM_CONNECTOR_PORT}"
>> address="0.0.0.0" security-enabled="false"
>> auth-realm-name="admin-realm" name="system"></jmx-connector>
>> <property name="adminConsoleDownloadLocation"
>>
>> value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war"></property>
>> <das-config></das-config>
>> </admin-service>
>> <web-container>
>> <session-config>
>> <session-manager>
>> <manager-properties></manager-properties>
>> <store-properties></store-properties>
>> </session-manager>
>> <session-properties></session-properties>
>> </session-config>
>> </web-container>
>> <ejb-container
>> session-store="${com.sun.aas.instanceRoot}/session-store">
>> <ejb-timer-service></ejb-timer-service>
>> </ejb-container>
>> <mdb-container></mdb-container>
>> <jms-service addresslist-behavior="priority"
>> default-jms-host="default_JMS_host">
>> <jms-host port="${JMS_PROVIDER_PORT}" host="localhost"
>> name="default_JMS_host"></jms-host>
>> </jms-service>
>> <log-service log-rotation-limit-in-bytes="2000000"
>> file="${com.sun.aas.instanceRoot}/logs/server.log">
>> <module-log-levels></module-log-levels>
>> </log-service>
>> <security-service>
>> <auth-realm
>> classname="com.sun.enterprise.security.auth.realm.file.FileRealm"
>> name="admin-realm">
>> <property name="file"
>> value="${com.sun.aas.instanceRoot}/config/admin-keyfile"></property>
>> <property name="jaas-context" value="fileRealm"></property>
>> </auth-realm>
>> <auth-realm
>> classname="com.sun.enterprise.security.auth.realm.file.FileRealm"
>> name="file">
>> <property name="file"
>> value="${com.sun.aas.instanceRoot}/config/keyfile"></property>
>> <property name="jaas-context" value="fileRealm"></property>
>> </auth-realm>
>> <auth-realm
>>
>> classname="com.sun.enterprise.security.auth.realm.certificate.CertificateRealm"
>> name="certificate"></auth-realm>
>> <jacc-provider
>> policy-provider="com.sun.enterprise.security.provider.PolicyWrapper"
>> name="default"
>> policy-configuration-factory-provider="com.sun.enterprise.security.provider.PolicyConfigurationFactoryImpl">
>> <property name="repository"
>> value="${com.sun.aas.instanceRoot}/generated/policy"></property>
>> </jacc-provider>
>> <jacc-provider
>>
>> policy-provider="com.sun.enterprise.security.jacc.provider.SimplePolicyProvider"
>> name="simple"
>> policy-configuration-factory-provider="com.sun.enterprise.security.jacc.provider.SimplePolicyConfigurationFactory"></jacc-provider>
>> <audit-module classname="com.sun.enterprise.security.Audit"
>> name="default">
>> <property name="auditOn" value="false"></property>
>> </audit-module>
>> <message-security-config auth-layer="SOAP">
>> <provider-config provider-type="client"
>> provider-id="XWS_ClientProvider"
>> class-name="com.sun.xml.wss.provider.ClientSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="dynamic.username.password"
>> value="false"></property>
>> <property name="debug" value="false"></property>
>> </provider-config>
>> <provider-config provider-type="client"
>> provider-id="ClientProvider"
>> class-name="com.sun.xml.wss.provider.ClientSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="dynamic.username.password"
>> value="false"></property>
>> <property name="debug" value="false"></property>
>> <property name="security.config"
>>
>> value="${com.sun.aas.instanceRoot}/config/wss-server-config-1.0.xml"></property>
>> </provider-config>
>> <provider-config provider-type="server"
>> provider-id="XWS_ServerProvider"
>> class-name="com.sun.xml.wss.provider.ServerSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="debug" value="false"></property>
>> </provider-config>
>> <provider-config provider-type="server"
>> provider-id="ServerProvider"
>> class-name="com.sun.xml.wss.provider.ServerSecurityAuthModule">
>> <request-policy auth-source="content"></request-policy>
>> <response-policy auth-source="content"></response-policy>
>> <property name="encryption.key.alias"
>> value="s1as"></property>
>> <property name="signature.key.alias" value="s1as"></property>
>> <property name="debug" value="false"></property>
>> <property name="security.config"
>>
>> value="${com.sun.aas.instanceRoot}/config/wss-server-config-1.0.xml"></property>
>> </provider-config>
>> </message-security-config>
>> </security-service>
>> <transaction-service
>> tx-log-dir="${com.sun.aas.instanceRoot}/logs"
>> automatic-recovery="true"></transaction-service>
>> <diagnostic-service></diagnostic-service>
>> <java-config debug-options="-Xdebug
>>
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=${JAVA_DEBUGGER_PORT}"
>> system-classpath="" classpath-suffix="">
>> <jvm-options>-XX:MaxPermSize=192m</jvm-options>
>> <jvm-options>-XX:PermSize=64m</jvm-options>
>> <jvm-options>-server</jvm-options>
>> <jvm-options>-Djava.awt.headless=true</jvm-options>
>> <jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
>>
>> <jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/modules/endorsed${path.separator}${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
>>
>> <jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
>>
>> <jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
>>
>> <jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
>>
>> <jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext</jvm-options>
>>
>> <jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
>> <jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
>>
>> <jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
>> <jvm-options>-XX:NewRatio=2</jvm-options>
>> <jvm-options>-Xmx512m</jvm-options>
>>
>> <jvm-options>-Dosgi.shell.telnet.port=${OSGI_SHELL_TELNET_PORT}</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.maxconn=1</jvm-options>
>> <jvm-options>-Dosgi.shell.telnet.ip=127.0.0.1</jvm-options>
>> <jvm-options>-Dgosh.args=--noshutdown -c noop=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.dir=${com.sun.aas.installRoot}/modules/autostart/</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.poll=5000</jvm-options>
>> <jvm-options>-Dfelix.fileinstall.log.level=3</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.new.start=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.bundles.startTransient=true</jvm-options>
>>
>> <jvm-options>-Dfelix.fileinstall.disableConfigSave=false</jvm-options>
>> </java-config>
>> <availability-service>
>> <web-container-availability></web-container-availability>
>> <ejb-container-availability
>> sfsb-store-pool-name="jdbc/hastore"></ejb-container-availability>
>> <jms-availability></jms-availability>
>> </availability-service>
>> <network-config>
>> <protocols>
>> <protocol name="http-listener-1">
>> <http default-virtual-server="server">
>> <file-cache></file-cache>
>> </http>
>> </protocol>
>> <protocol security-enabled="true" name="http-listener-2">
>> <http default-virtual-server="server">
>> <file-cache></file-cache>
>> </http>
>> <ssl
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="s1as"></ssl>
>> </protocol>
>> <protocol name="admin-listener">
>> <http default-virtual-server="__asadmin"
>> max-connections="250">
>> <file-cache></file-cache>
>> </http>
>> </protocol>
>> <protocol security-enabled="true" name="sec-admin-listener">
>> <http default-virtual-server="__asadmin"
>> encoded-slash-enabled="true">
>> <file-cache></file-cache>
>> </http>
>> <ssl client-auth="want"
>> classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
>> cert-nickname="glassfish-instance"></ssl>
>> </protocol>
>> <protocol name="admin-http-redirect">
>> <http-redirect secure="true"></http-redirect>
>> </protocol>
>> <protocol name="pu-protocol">
>> <port-unification>
>> <protocol-finder protocol="sec-admin-listener"
>> name="http-finder"
>> classname="com.sun.grizzly.config.HttpProtocolFinder"></protocol-finder>
>> <protocol-finder protocol="admin-http-redirect"
>> name="admin-http-redirect"
>> classname="com.sun.grizzly.config.HttpProtocolFinder"></protocol-finder>
>> </port-unification>
>> </protocol>
>> </protocols>
>> <network-listeners>
>> <network-listener port="${HTTP_LISTENER_PORT}"
>> protocol="http-listener-1" transport="tcp" name="http-listener-1"
>> thread-pool="http-thread-pool"></network-listener>
>> <network-listener port="${HTTP_SSL_LISTENER_PORT}"
>> protocol="http-listener-2" transport="tcp" name="http-listener-2"
>> thread-pool="http-thread-pool"></network-listener>
>> <network-listener port="${ASADMIN_LISTENER_PORT}"
>> protocol="pu-protocol" transport="tcp" name="admin-listener"
>> thread-pool="http-thread-pool"></network-listener>
>> </network-listeners>
>> <transports>
>> <transport name="tcp"></transport>
>> </transports>
>> </network-config>
>> <thread-pools>
>> <thread-pool name="http-thread-pool"></thread-pool>
>> <thread-pool max-thread-pool-size="200"
>> name="thread-pool-1"></thread-pool>
>> <thread-pool name="admin-thread-pool"
>> max-thread-pool-size="50" max-queue-size="256"></thread-pool>
>> </thread-pools>
>> <group-management-service>
>> <failure-detection></failure-detection>
>> </group-management-service>
>> <management-rules></management-rules>
>> <system-property name="ASADMIN_LISTENER_PORT"
>> value="24848"></system-property>
>> <system-property name="HTTP_LISTENER_PORT"
>> value="28080"></system-property>
>> <system-property name="HTTP_SSL_LISTENER_PORT"
>> value="28181"></system-property>
>> <system-property name="JMS_PROVIDER_PORT"
>> value="27676"></system-property>
>> <system-property name="IIOP_LISTENER_PORT"
>> value="23700"></system-property>
>> <system-property name="IIOP_SSL_LISTENER_PORT"
>> value="23820"></system-property>
>> <system-property name="IIOP_SSL_MUTUALAUTH_PORT"
>> value="23920"></system-property>
>> <system-property name="JMX_SYSTEM_CONNECTOR_PORT"
>> value="28686"></system-property>
>> <system-property name="OSGI_SHELL_TELNET_PORT"
>> value="26666"></system-property>
>> <system-property name="JAVA_DEBUGGER_PORT"
>> value="29009"></system-property>
>> <monitoring-service>
>> <module-monitoring-levels></module-monitoring-levels>
>> </monitoring-service>
>> <connector-service></connector-service>
>> </config>
>> </configs>
>> <property name="administrative.domain.name" value="domain1"></property>
>> <secure-admin enabled="true"
>> special-admin-indicator="3047aff3-3214-4ac9-aa5e-a5dad78b2eea">
>> <secure-admin-principal dn="CN=localhost,OU=GlassFish,O=Oracle
>> Corporation,L=Santa
>> Clara,ST=California,C=US"></secure-admin-principal>
>> <secure-admin-principal
>> dn="CN=localhost-instance,OU=GlassFish,O=Oracle Corporation,L=Santa
>> Clara,ST=California,C=US"></secure-admin-principal>
>> </secure-admin>
>> <load-balancers></load-balancers>
>> <lb-configs></lb-configs>
>> <clusters></clusters>
>> </domain>
>>
>>
>> Anyone knows what might be the problem?
>>
>>
>>
>> --
>> Mladen Adamovic
>> Numbeo
>> Drziceva 9, 11120 Belgrade-Zvezdara, Serbia
>> (Business Registration Number 62612240)
>> Tel. +381-66-058-595
>> email: mladen.adamovic_at_gmail.com
>> web: http://www.numbeo.com
>
>



-- 
Mladen Adamovic
Numbeo
Drziceva 9, 11120 Belgrade-Zvezdara, Serbia
(Business Registration Number 62612240)
Tel. +381-66-058-595
email: mladen.adamovic_at_gmail.com
web: http://www.numbeo.com