Hi Scott,
I've combined your idea of using jstack, with Harsha's previous
recommendation on lowering the number of files written to. I did 3
runs, using 20 files, 42 files, and 83 files written to, to benchmark,
and came up with some information.
Firstly, I've attached the "jstack" output for each of these runs
(jstack.zip). If you could take a look @ them, I'd appreciate it.
Files with .0 suffix is before the run began, .1 is while the number of
clients goes to about 100, and .2 and .3 are while the client load is
peaking. Please let me know if you find anything.
As far as web service call times, I found the following:
Average call to web service:
20 files: .2 seconds
42 files: 3.1 seconds
83 files: 61 seconds
As you can see, the increase is astronomical.
Please let me know if any more information is helpful.
-Bill
Scott Oaks wrote:
> Assuming that you have tuned the thread pools correctly (and it sounds
> like you've looked at that), then something is causing a bottleneck. It
> sounds almost like there's some sort of deadlock between your client(s)
> and server.
>
> One easy trick to find this is to make sure that you're using JDK 6
> (6_04 is best), and then to use the JDK 6 jstack command to get the
> stack of the appserver. [You can use various tools, including asadmin,
> to get stack dumps, but jstack will be fastest.] If threads are all
> blocking on a resource or something, you can usually tell from the
> stack dump: all of the GrizzlyWorkerThreads will be doing something
> other than calling waitForIoTask (or a similarly named method, depending
> on which precise version you're running).
>
> You can send me a stack dump if you have questions about it.
>
> -Scott
>
> On Wed, 2008-01-30 at 18:57, William Saxton wrote:
>> Hi all,
>>
>> Since the EJB issues I was having earlier appear to be due to a known
>> bug, I've switched over to using a Web Service to handle my application
>> server needs. The usage is the same, I have about 700 clients accessing
>> an application server, every 5 minutes, so around 3-4 clients will be
>> accessing it every second.
>>
>> It takes less than 1 second for a client to send the web service the
>> XMLified data, the application server to save the data to about 80
>> different files, and for it to return an "ok". When I turn on all of
>> the clients, though, it takes anywhere from 10 seconds to over 5 minutes
>> (which I timeout) for the request to be completed.
>>
>> I've tried these JVM tweaks:
>> http://weblogs.java.net/blog/sdo/archive/2007/12/a_glassfish_tun.html
>> as well as setting the HTTP acceptor threads to 16 and
>> request-processing threads to 16. These don't seem to have done
>> anything. There is very little CPU usage (~90% idle) and little local
>> disk I/O (where the data is saved) so I don't know where the bottleneck
>> is. I'm wondering whether the application server is simply queuing up
>> too many of the client requests when it could be handling more.
>>
>> Anyone have an ideas? Since I'm internal to Sun, I can always give
>> someone access to the app server if they wanted to do a quick glance.
>>
>> Thanks.
>>
>> -Bill
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>