Well we're actually not using one machine for each instance in the cluster.
We have a setup where we have two servers with two server instances on each machine. When one cluster is running we only have one server instance running on each machine. When we need to upgrade we start the other cluster and upgrade our application switch our load balancer and shut down the cluster with the old version. This means of course that we don't have all of the hardware resources available for the currently used cluster during upgrade.
[Message sent by forum member 'woel' (woel)]
http://forums.java.net/jive/thread.jspa?messageID=258195