Still closed Re: Please hold external commits to master until at least Thursday

From: Edward Burns <>
Date: Thu, 24 Sep 2015 12:26:40 -0700

>>>>> On Tue, 22 Sep 2015 11:37:51 -0700, Edward Burns <> said:

EB> Hello Mojarra committers,
EB> I have observed a spike in the number of ADF test failures in the most
EB> recent run. Please hold off on any commits to master until at least
EB> Thursday so I can get more data on the failures.

EB> I apologize for the inconvenience.

I'm sorry but I do not have good news on the ADF test front. I was
hoping to integrate JAVASERVERFACES_SPEC_PUBLIC-1 to master today but
the following ADF numbers cause me to have to pursue cleaning the
numbers up first.

Take a look at this. The NWDIF number indicates the number of failed
tests relative to a known "correct" test run. Higher numbers are
bad. We run two test jobs for ADF tests where the only thing that
differs from run to run is Mojarra. In this way we can compare how
compatible Mojarra 2.3.0-X is with Mojarra 2.2.8-X (the version going
into WLS).

The bottom line is that the NWDIF numbers on the 2.3.0-X line must be
similar to what we see on the 2.2.8-X line. We can see that as of the
20150920 run they started to diverge badly.

SECTION: 2.3.0-X adf-tests

Date tests run Mojarra timestamp NWDIF

20150902 20150709-0007 16
20150909 20150709-0007 ---
20150911 20150709-0007 154
20150913 20150709-0007 11
20150916 20150709-0007 12
20150918 20150917-1754 11
20150920 20150918-0700 147
20150923 20150921-2304 134

SECTION: 2.2.8-X adf-tests

Date tests run Mojarra timestamp NWDIF

20150912 20150911-1541 29
20150915 20150914-1206 8
20150917 20150914-1206 29
20150919 20150914-1206 28
20150922 20150914-1206 15
20150924 20150914-1206 14

I inspected the commits to 2.3 master in the range 2015-09-15 -
2015-09-22. In that range, commits came from the following individuals
(with the type of commit):

Zhijun: javadoc

Arjan: work on CDI producers

Manfred: nbproject, pom, javadoc for BCP-47

Ed: remove lambda from CdiExtension.processBean(), javadoc

We can rule out Zhijun and Manfred as the possible causes. We can rule
out Ed as a cause because the lambda change only impacted deployment on
JDK1.8u60 and the ADF tests run on an earlier JDK. Also, if the problem
fixed by Ed's commit to remove lambda had been present in the builds
that we were running the ADF tests on, the NWDIF numbers would have been
huge because nothing would deploy.

This leaves Arjan. I hate to single you out, but we may have to ask you
to roll back your commits in that range.

Let me ask Manfred if we can modify the test job to pull a specific
commit: cfea96318ea1b8de2615e7a450860fa7b017ae86. This was the last
commit before the first of two commits from Arjan in that date range.

If we get a cleaner run with commit cfea963 then I'll ask you to revert


| | office: +1 407 458 0017
| 30 Business days til JavaOne 2015
| 45 Business days til DOAG 2015