It helps if I actually read the initial post.
My previous post is still relevant, but I want to explain what is actually happening here, and what's killing you.
The problem you are encountering is, yes, technically, a memory problem.
But in truth the problem is a swap problem.
Back in the day, when a Unix system is configured, you would typically allocate as much swap space as you had memory, sometime twice as much. Swap is important to a Unix system as its Virtual Memory tends to be backed by swap.
So, here's what's happening to you at this point.
As you can see from the stack trace, the installer is trying to execute a system process (that's what exec does). The problem here is that when the system tries to create a new process, it first calls the system call "fork". Fork makes an exact duplicate of the running process, and thats where you're running in to trouble.
In truth, the system doesn't physically duplicate the process. For example, if you have a 20MB process, the system does run out and grab 20MB more RAM, and copy your process in to that RAM. That would be far too expensive.
What it DOES do, tho, is it asks the Virtual Memory system to make a, well, virtual copy. The system goes out and allocates 20MB of Virtual Memory, and points it all at the pages of your current process, and then starts the new process. If the new process needs to change any of that memory, the system will incrementally copy the changed pages for use by the new process. This is called "Copy on write".
Now, the nut is that as soon as fork has completed all that work, exec is called which immediately wipes it all out to be replaced by a new process. But the system doesn't know that this is happening. It has to assume that when you fork a new process that you MIGHT want all of its memory. That's why it makes a complete, albeit virtual, copy of the system.
The problem you have is that your zone isn't giving you the swap space necessary to make the virtual copy of the Java system so it can fork this new process.
Here's why this is a potential problem long term.
Even if you work around the install issue, you may not be able to run glassfish on this machine. The obvious problem is JSPs. Glassfish will fork the server to compile a JSP. So it's easy to imagine you getting the server up and running, and then deploy a small Web app that fails on the JSP compile.
Java 6 remedies this situation as Glassfish will compile JSPs in place, but if you use Java 5 this will be a real concern. You can precompile your JSPs to get around this as well. And note, you don't need to build your software with Java 6, rather you just need to run Glassfish with Java 6.
Bottom line, if you can, talk to whoever manages your zone and get more swap. Unix systems with no (or little) swap are badly curtailed I think.
[Message sent by forum member 'whartung' (whartung)]
http://forums.java.net/jive/thread.jspa?messageID=267854