Hi,
On Thu, Sep 24, 2015 at 9:26 PM, Edward Burns <edward.burns_at_oracle.com> wrote:
>>>>>> On Tue, 22 Sep 2015 11:37:51 -0700, Edward Burns <edward.burns_at_oracle.com> said:
> I'm sorry but I do not have good news on the ADF test front. I was
> hoping to integrate JAVASERVERFACES_SPEC_PUBLIC-1 to master today but
> the following ADF numbers cause me to have to pursue cleaning the
> numbers up first.
What are the ADF tests exactly testing? Is this something I can setup
and run locally, so I can test any change on those tests too before
committing anything?
>
> Take a look at this. The NWDIF number indicates the number of failed
> tests relative to a known "correct" test run. Higher numbers are
> bad. We run two test jobs for ADF tests where the only thing that
> differs from run to run is Mojarra. In this way we can compare how
> compatible Mojarra 2.3.0-X is with Mojarra 2.2.8-X (the version going
> into WLS).
>
> The bottom line is that the NWDIF numbers on the 2.3.0-X line must be
> similar to what we see on the 2.2.8-X line. We can see that as of the
> 20150920 run they started to diverge badly.
>
> SECTION: 2.3.0-X adf-tests
>
> Date tests run Mojarra timestamp NWDIF
>
> 20150902 20150709-0007 16
> 20150909 20150709-0007 ---
> 20150911 20150709-0007 154
> 20150913 20150709-0007 11
> 20150916 20150709-0007 12
> 20150918 20150917-1754 11
> 20150920 20150918-0700 147
> 20150923 20150921-2304 134
>
> SECTION: 2.2.8-X adf-tests
>
> Date tests run Mojarra timestamp NWDIF
>
> 20150912 20150911-1541 29
> 20150915 20150914-1206 8
> 20150917 20150914-1206 29
> 20150919 20150914-1206 28
> 20150922 20150914-1206 15
> 20150924 20150914-1206 14
A few questions here:
Are there 11~15 tests failing all the time? What are they failing on?
What happened on 20150911? That's the highest number in this snapshot
of runs, but it went back to 11 two days after
> I inspected the commits to 2.3 master in the range 2015-09-15 -
> 2015-09-22. In that range, commits came from the following individuals
> (with the type of commit):
In the table it seems that things diverged from 20150920, and the last
run before that is 20150918. The Mojarra timestamp is a little
behind; 20150917.
How are the commits from 2015-09-15 related to this?
>
> Zhijun: javadoc
>
> Arjan: work on CDI producers
>
> Manfred: nbproject, pom, javadoc for BCP-47
>
> Ed: remove lambda from CdiExtension.processBean(), javadoc
>
> We can rule out Zhijun and Manfred as the possible causes. We can rule
> out Ed as a cause because the lambda change only impacted deployment on
> JDK1.8u60 and the ADF tests run on an earlier JDK.
Although unlikely, theoretically the fact that a bug didn't manifest
on JDK8u60, doesn't necessarily have to mean that the workaround
doesn't have consequences for earlier JDKs. But as mentioned, this
seems unlikely as the Mojarra tests would likely have failed before
that.
> Let me ask Manfred if we can modify the test job to pull a specific
> commit: cfea96318ea1b8de2615e7a450860fa7b017ae86. This was the last
> commit before the first of two commits from Arjan in that date range.
Okay, I'll await that. Depending on the outcome of that run, we could
see if removing some of the addBean calls in CDIExtension have any
effect, but it's perhaps too early to speculate about that. It would
help though to have an indication of what is exactly failing at the
moment.
Kind regards,
Arjan Tijms
>
> If we get a cleaner run with commit cfea963 then I'll ask you to revert
> those.
>
> Thanks,
>
> Ed
> --
> | edward.burns_at_oracle.com | office: +1 407 458 0017
> | 30 Business days til JavaOne 2015
> | 45 Business days til DOAG 2015