On Tue, Oct 13, 2015 at 4:31 PM, Edward Burns <edward.burns_at_oracle.com> wrote:
> First, I want to thank Arjan for his patience on this issue with the
> tree being closed, and his willingness to help determine the cause of
> the compatibility problem with ADF.
No problem ;)
> EB> SECTION: 2.3.0-X adf-tests
> EB>
> EB> Date tests run Mojarra timestamp NWDIF
> EB> or identifier
> EB>
> EB> 20150902 20150709-0007 16
> EB> 20150909 20150709-0007 ---
> EB> 20150911 20150709-0007 154
> EB> 20150913 20150709-0007 11
> EB> 20150916 20150709-0007 12
> EB> 20150918 20150917-1754 11
> EB> 20150920 20150918-0700 147
> EB> 20150923 20150921-2304 134
> EB> 20151004 2.3.0-m03 14
> EB> 20151009 2.3.0-m03 17
> EB> 20151011 2.3.0-m03 12
>
> EB> So now we know that something introduced between m03 and 2015-09-18 is
> EB> the cause of the problem.
>
> AT> There's one thing I don't quite understand here. If the problem was
> AT> introduced between m03 and 2015-09-18, what happened on 2015-09-02,
> AT> 2015-09-09, 2015-09-13, 2015-09-16 and 2015-09-18? Since that NWDIF
> AT> number was lower on those dates as well.
>
> We must accept those spikes as noise since they can be dismissed as not
> part of a trend.
From the given data, I think it's difficult to see a trend. Since m03,
there are 5 dates where the number is low (I assume --- means 0?), and
3 dates where the number is high. Neither one consecutive. I.e. it's
depending on what "---" means:
low - low - high - low - low - low - high - high
Just looking at these numbers I think both can be a spike. Notice
they're also alternating, which may tell us something else.
> Since you asked, I will do that presently. We should have some results
> by Friday.
Okay, so let's wait for that then.
> AT> One other thing that we could try is adding a quick switch [...]
> That sounds good, but isn't that effectively the same as running the
> tests against a build that doesn't have the functionality added?
Not entirely, since then a lot of other changes would not be there as
well. With a switch we can quickly remove an entire class of
functionality that was originally added via a few dozen commits
throughout the year.
Another important question; did anyone already look at what actually
failed when those high numbers were observed? If the failure would be
because of say a database connection timing out then this may rule out
a Mojarra commit.
Kind regards,
Arjan