dev@glassfish.java.net

RE: Strange phenomenon when failed in loading the ejb application

From: Jeremy Lv <lvsongping_at_cn.fujitsu.com>
Date: Fri, 24 Jan 2014 16:14:01 +0800

Hi, Hong

Cc: Jeff, Marina:



    As I haven’t attached all of the reproduced application, I have
attached them again so that you can reproduce this issue on your local
platform.



Thanks



Regards

Jeremy Lv



From: Jeremy Lv [mailto:lvsongping_at_cn.fujitsu.com]
Sent: Friday, January 24, 2014 4:08 PM
To: hong.hz.zhang_at_oracle.com
Cc: marina.vatkina_at_oracle.com; dev_at_glassfish.java.net; Jeff Tancill
Subject: RE: Strange phenomenon when failed in loading the ejb application



Hi, Hong:

Cc: Marina, Jeff:



Here’s my Comment:



It is by design that we don't generate or delete any files for load and
unload. The file generation and deletion only happens during prepare and
clean phases as part of the deployment and undeployment.
Jeremy Lv: If it is the right scenario that we shouldn’t generate or delete
any files for load and unload, the phenomenon I have describe is also
different from the right scenario. The Ejb related policy file will be
created when execute the appInfo.load(context, tracker) in the method of
ApplicationLifecycle.deploy() when start the cluster after the ejb
application has been deployed.



In the case you described, as the cluster was offline when the initial
deployment happens, we could not detect this type of error that time to roll
back. And when the error happens during the cluster starts up (which is a
load for the application), we don't do roll back. So this is kind of
expected behavior with the current design. At that time you would have to
perform an undeploy operation to clean up the application bits explicitly.

Jeremy Lv: I just feel it is strange that the ejb related policy file is
created during the cluster starts up (which is a load for the application),
why not rollback the load process when the cluster started(Delete the ejb
related policy file created during the load process)? This may cause the
some critical or blocker issues:



【Sample】

Here’s one of the similar blocker issue still opened in the JIRA

https://java.net/jira/browse/GLASSFISH-11348

Here’s the reproduced steps:

1) asadmin start-domain

2) asadmin create-cluster clu1

3) asadmin create-instance -cluster clu1 ins1

4) asadmin deploy --target clu1 compositeweb-ejb-1.0-SNAPSHOT.jar

5) asadmin deploy --target clu1 compositeweb-web-1.0-SNAPSHOT.war

6) asadmin start-cluster clu1

7) asadmin stop-cluster clu1

8) asadmin start-cluster clu1



After step(8), the following exception will be thrown out:

[2014-01-24T15:52:04.855+0800] [glassfish 4.0] [SEVERE] [NCLS-CORE-00026]
[javax.enterprise.system.core] [tid: _ThreadID=17
_ThreadName=RunLevelControllerThread-1390549911763] [timeMillis:
1390549924855] [levelValue: 1000] [[

  Exception during lifecycle processing

org.glassfish.deployment.common.DeploymentException: Error in linking
security policy for compositeweb-web-1.0-SNAPSHOT -- Inconsistent Module
State

              at
com.sun.enterprise.security.ee.SecurityUtil.linkPolicyFile(SecurityUtil.java
:336)

              at
com.sun.enterprise.security.ee.SecurityDeployer.linkPolicies(SecurityDeploye
r.java:318)

              at
com.sun.enterprise.security.ee.SecurityDeployer.access$100(SecurityDeployer.
java:87)

              at
com.sun.enterprise.security.ee.SecurityDeployer$AppDeployEventListener.event
(SecurityDeployer.java:145)

              at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:
131)

              at
org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:328)

              at
com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycl
e.java:496)

              at
com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(App
licationLoaderService.java:406)

              at
com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(Applicat
ionLoaderService.java:243)

              at
org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:298)

              at
org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:346)

              at
org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)

              at
org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncR
unLevelContext.java:225)

              at
org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java
:82)

              at
org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2358)

              at
org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:9
8)

              at
org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:8
7)

             at
org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(Cur
rentTaskFuture.java:1162)

              at
org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(Curren
tTaskFuture.java:1147)

              at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
10)

              at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
03)

              at java.lang.Thread.run(Thread.java:722)

]]



【Reason】

1). The policy file hasn’t been removed when stop the cluster or the policy
file shouldn’t be created when start the domain.



【Solution】

This is the reason why I regarded this scenario as a bug but not the
expected behavior. After all, I have the following idea to fix this bug

1) The policy file shouldn’t be created when load the application.

2) The rollback action when failed in loading or starting the application
should remove the policy file which is created during the load process.





Wish if you can give me some suggestions about this.



Thanks



Regards

Jeremy Lv





From: Hong Zhang [mailto:hong.hz.zhang_at_oracle.com]
Sent: Friday, January 24, 2014 12:20 AM
To: Jeremy Lv; marina.vatkina_at_oracle.com
Cc: dev_at_glassfish.java.net
Subject: Re: Strange phenomenon when failed in loading the ejb application



Hi, Jeremy

It is by design that we don't generate or delete any files for load and
unload. The file generation and deletion only happens during prepare and
clean phases as part of the deployment and undeployment.

In the case you described, as the cluster was offline when the initial
deployment happens, we could not detect this type of error that time to roll
back. And when the error happens during the cluster starts up (which is a
load for the application), we don't do roll back. So this is kind of
expected behavior with the current design. At that time you would have to
perform an undeploy operation to clean up the application bits explicitly.

Thanks,

- Hong

On 1/23/2014 6:29 AM, Jeremy Lv wrote:

Hi, Hong, Marina:

Cc: dev:





    I have found a strange situation recently in failing deploy the ejb
application:

Here’s my reproduced steps to reproduce this phenomenon:

1) asadmin start-domain

2) asadmin create-cluster clu1

3) asadmin create-instance -cluster clu1 ins1

4) asadmin deploy --target clu1 compositeweb-ejb.jar

5) Confirm whether there’s no file called compositeweb-ejb exist under
the directory of GF_HOME\nodes\localhost-domain1\ins1\generated\policy

6) asadmin start-cluster clu1

7) The file called compositeweb-ejb has been generated under the
directory of GF_HOME\nodes\localhost-domain1\ins1\generated\policy but
hasn’t been removed even it is failed in loading the ejb application



On the other hand, if you failed in deploying the ejb application after the
cluster has been started, the file called compositeweb-ejb under the
directory of GF_HOME\nodes\localhost-domain1\ins1\generated\policy will be
removed as expected:

1) asadmin start-domain

2) asadmin create-cluster clu1

3) asadmin create-instance -cluster clu1 ins1

4) asadmin start-cluster clu1

5) Confirm whether there’s no file called compositeweb-ejb exist under
the directory of GF_HOME\nodes\localhost-domain1\ins1\generated\policy

6) asadmin deploy --target clu1 compositeweb-ejb.jar

7) The file called compositeweb-ejb has been generated under the
directory of GF_HOME\nodes\localhost-domain1\ins1\generated\policy when
loading the application and it will be removed because of the failure of
deploying the ejb applications





【Conclusion】

After Looking into the method of EjbDeployer.clean(DeploymentContext), you
will found the following related code segment to illustrate the reason why
the scenario is different:

        //Security related cleanup is to be done for the undeploy event

◆ if( params.origin.isUndeploy()|| params.origin.isDeploy()) {



            //Removing EjbSecurityManager for undeploy case

            String appName = params.name();

            String[] contextIds =

                    ejbSecManagerFactory.getContextsForApp(appName, false);

            if (contextIds != null) {

                for (String contextId : contextIds) {

                    try {

                        //TODO:appName is not correct, we need the module
name

                        //from the descriptor.


probeProvider.policyDestructionStartedEvent(contextId);

◆ SecurityUtil.removePolicy(contextId);
// It will remove the policy file it this code has been executed.


probeProvider.policyDestructionEndedEvent(contextId);

                        probeProvider.policyDestructionEvent(contextId);

                    } catch (IASSecurityException ex) {

                        _logger.log(Level.WARNING, "Error removing the
policy file " +

                                "for application " + appName + " " + ex);

                    }



                    ArrayList<EJBSecurityManager> managers =

                            ejbSecManagerFactory.getManagers(contextId,
false);

                    if (managers != null) {

                        for (EJBSecurityManager m : managers) {

                            m.destroy();

                        }

                   }

                }

            }

            //Removing the RoleMapper

            SecurityUtil.removeRoleMapper(dc);

        }



(1). If you start the cluster after the ejb and web application had been
deployed, the value of params.origin will be “load” and it will return
false when execute “if( params.origin.isUndeploy()||
params.origin.isDeploy())”. The SecurityUtil.removePolicy(contextId) won’t
be executed, this is the reason why the policy can’t be removed when
execute the rollback action.

(2). I have also confirmed that if you deploy the web or ejb application
after the cluster is started, the value of params.origin will be
“deploy_instance” and it will return true when execute “if(
params.origin.isUndeploy()|| params.origin.isDeploy())”. Then it will
execute the SecurityUtil.removePolicy(contextId) to remove the related
policy file.





【The Key point question】

Why the scenario in rollback action is different between load and deploy the
ejb application, why remove the policy file when failed in deploying the ejb
application after cluster has been started but still keep the policy while
starting the cluster after the ejb application has been deployed.



【Conclusion】

I don’t know whether it is the right scenario or it is a bug, In my human
option, I think it is a bug,The policy file should be removed during the
rollback action both in failing loading the application and deploying the
application to consist these two different scenario.



Please also let me know if it is a right scenario.



Thanks a lot!



Best regards

Jeremy Lv

--------------------------------------------------

Lv Songping

Software Division II

    Development Department I

Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)

ADDR.: No.6 Wenzhu Road, Software Avenue,

        Nanjing, 210012, China

TEL : +86+25-86630566-9327

COINS: 7998-9327

FAX : +86+25-83317685

MAIL : lvsongping_at_cn.fujitsu.com

BLOG : https://www.java.net//author/jeremy-lv
<https://www.java.net/author/jeremy-lv>