users@glassfish.java.net

Re: Clustered Timer Service Behavior

From: <glassfish_at_javadesktop.org>
Date: Sat, 13 Dec 2008 11:02:35 PST

> I'm guessing (I don't know) that what will happen is
> that when the ServletContextListeners fire, you
> actually have a race condition.
>
> 1 of 2 things will happen.
>
> 1, each instance will think it "owns" the timer and
> execute the job (i.e. if you have 3 servers, it runs
> 3 times -- this would be bad).
>
> 2, when instance A creates the timer, if the timer
> already exists, that existing timer is "stomped on"
> and the new timer takes precedence. So, likely, if
> Instance A creates the timer, then Instance B creates
> the same timer, B tells A "I'm taking over timer X",
> and thus B "owns" it.

It's essentially 1), but if there's no check to see if the timer already exists it's even worse since a new one will be created every time the same server instance is restarted. Think of each call to TimerService.createTimer() as producing a separate row in a database. There is no special identity relationship between two calls to createTimer(), whether those two calls are made from the same JVM or a different JVM in the cluster. An enterprise bean can create any number of timers. Each is unique.

If what you want is one timer for the entire cluster, calling createTimer() within a
server instance specific callback is error prone. Even if you check first to see whether the
bean already has a timer associated with it, there is still the possibility of a race condition that would result in two separate server instances both creating a timer.

These are the very reasons we are adding support for automatic timer creation in EJB 3.1.
An automatically created timer will be specified via metadata. That way, the container
will ensure that for each automatic timer only one set of callbacks will take place, independent
of whether the application is deployed to a single server instance or to a cluster.

Otherwise, the burden is on the application to ensure "one per cluster" semantics.

>
> Mind, this all depends on how it's working.
>
> What I REALLY think happens is that nobody "owns" the
> timer, rather the timer notes the instance that
> started the timer in the DB.
>
> Then at regular intervals, the timer service "polls"
> the timer DB for timers that it owns.When a system
> fails, the cluster management finds any timers
> associated with the fail instance, and simply changes
> the owner to some other system in the cluster. Thus,
> when that system says "Give me timers I own",
> suddenly a new timer shows up.
>
> So, if you have 3 instances creating the same timer
> "at the same time", you really have 3 processes
> updating the timer db all at the "same time", and
> thus one will "win" just by being the last one to
> create the timer.
>
> So, whoever is last in line get caught with the bag
> and runs the timer.
[Message sent by forum member 'ksak' (ksak)]

http://forums.java.net/jive/thread.jspa?messageID=321591