There are plenty of test harnesses out there, but I'm no expert and have no standing to recommend one.
But, if the cause of the slowdown is indeed the Java Web Start downloads, you might try creating your own very simple test script that just executes the javaws command line to launch the client, then have a very small driver that starts some number of copies of the script.
For your tests, you would probably want to launch the client once manually and click on the Always Trust option on the certificate prompt screen. That will suppress prompts during the load test.
Remember that to simulate different clients launching the client you'll need to prevent test runs on the same system from using a common cache. Otherwise the first download will satisfy all the tests running on that system. Java SE 6 has an enhancement that allows you to turn off all caching. You can get to it using the Java control panel, click on Temporary Internet Files Settings, and uncheck the "Keep Temporary Internet FIles on my Computer" check box. Even if you are not using Java SE 6 for your production installation this enhancement would be a huge help for your test. Otherwise you'd have to run each test as a different user.
Please let us know what you find.
- Tim
[Message sent by forum member 'tjquinn' (tjquinn)]
http://forums.java.net/jive/thread.jspa?messageID=235280