dev@glassfish.java.net

Re: IMPORTANT: Triage on P3 bugs for GlassFish V2/9.1

From: Bill Shannon <bill.shannon_at_sun.com>
Date: Thu, 15 Feb 2007 21:46:00 -0800

vince kraemer wrote:
> I cc'ed the quality alias, since this is a pretty relevant discussion
> for that alias.
>
> Bill Shannon wrote:
>> Wrong, wrong, wrong.
>>
>> I see it's time for me to yet again explain how we are supposed to use
>> bug priorities. Haven't I done this at least every release for the
>> past, oh, I don't know, 8 years?
>>
>> A bug doesn't have an inherent priority. A bug isn't born with a
>> priority.
>
> I thought issues are created with a priority value by the filer....

Initially we considered having a separate field for the submitter
to suggest how important the bug was, and require the owner of the
bug to choose the priority based on the submitter's input and other
factors, but that just seemed unnecessarily complex. So the submitter
gets to suggest how important the bug is when they file it, but it's
up to the owner of the bug to adjust the priority as appropriate.

My point is that priority is a measure of a bug's importance at a
point in time, and it can change over time. It's not something
inherent to the bug.

>> The priority of a bug is *our* decision. Not just a technical decision.
>> Not just based on the impact of the bug. Not just based on the
>> severity of
>> the bug. But also based on resource, business decisions, and so on.
>
> A filer doesn't have visibility into these factors, so should all bugs
> get filed as P1/unconfirmed?
>
> This would basically mean that the bug must get evaluated and assigned a
> priority, before the product can ship?

See above. The submitter suggests an initial priority. The owner of
the bug should always be evaluating the bug to determine whether that
suggestion is appropriate or not.

>> The priority of a bug is the *output* of our decision making process;
>> it reflects what we *decided*, not what a bug *is*.
>>
>> If we think a bug absolutely, positively, *MUST* be fixed before FCS,
>> and if that one single bug is not fixed before FCS we will absolutely
>> hold up the release until that single bug is fixed, then we mark it as
>> a P3. Similarly for beta, we mark it as a P2. You can probably guess
>> how important a P1 bug is.
>
> Fix before next promoted build?

If not sooner. Generally it means "drop everything and work on this".

>> If we just *want* to fix the bug for FCS, that's a P4 bug.
>>
>> If you have a P4 bug, and you want to fix it for FCS, but you don't want
>> to lose track of it, fill in the "target release" field with the name of
>> the release where you intend to fix it.
>>
>> And if there are P4 bugs that you're not expecting to get to anytime
>> soon, well, that's why we have P5.
>
> I think this is a reasonable "process"... It is a bit different than the
> processes that the help associated with the GF issue tracker presents.
> We may want to publish an outline of the process on the wiki for the
> community, so filers do not misunderstand the meaning of their P2 bug
> being reclassified as a P4 bug...

Yes. I thought this was raised and dealt with a year ago when we really
started using Issue Tracker. The process I've described is the process
we're supposed to be using within Sun for Bugtraq.

>> Do not add keywords to the bugs.
>>
>> This is the way we manage 900 bugs. We divide them into categories that
>> are manageable. The bugs that deserve the bulk of our attention are
>> those that we absolutely have to fix no matter what. If there aren't
>> many less than 900 of those bugs, we're in deep trouble.
>>
>> Remember that when you submit a bug.
>
> It is very hard for filers to assign priorities. That is really the
> "job" of the person that evaluates the bug.
>
> But, that really means that all bugs that are unconfirmed need to be
> treated as high priority issue until they have been evaluated...

That's why Bugtraq has "dispatched" and "accepted" states. Until a
bug moves from dispatched to accepted, you shouldn't trust the priority.
Which is why bugs should remain in the dispatched state for as little
time as possible, preferably less than a week.

Issue Tracker doesn't match our process as well so I'm not sure exactly
how this should map.

> For example; in the GF issue tracker there are 375 unconfirmed issues,
> 291 of them are DEFECTS...
>
> As a person evaluating products based on the GF codebase, this is a very
> troubling issue.
>
> To me, at first glance, it looks like there are 291 possible show
> stopper issues that haven't been examined.

If there are 291 issues that no one has looked into even enough to
know whether the priorities seem correct, you should be worried.