Salut,
Bongjae Chang wrote:
> Hi,
>
> Then, how about using new ThreadPoolExecutor which is based on JDK's ThreadPoolExecutor?
>
> I have made and used ThreadPoolExecutor2 experimentally which can be modified easily from original ThreadPoolExecutor.
>
> When I just modified Executor#execute()'s impl simply in JDK's ThreadPoolExecutor, new ThreadPoolExecutor worked well as a common user expected in a practical manner.
>
> Moreover, ThreadPoolExecutor2 was faster than original ThreadPoolExecutor at some cases or had same performance at least when I tested it in various cases.
>
> I tried to improve the following points. This points are different from original ThreadPoolExecutor.
> - If all core threads are busy approximately, it increases a new work thread until max size.
> - It guarantees that a task is executed as fair as possible. Remember that original ThreadPoolExecutor doesn't guarantee task's fairness when queue is full because new thread always is created and execute the new task immediately under max size.
>
> If you are curious to know how it works, please see attached source.
>
> And I tagged prefix "[carryel]" in modified code, so I think that you can verify changed code easily.
>
> If poosible, and if this proposal could be accepted, I wish to contribute this simple logic to Grizzly. :-)
I like it! I will do a little more performance testing but so far it is
quite close to my original Pipeline I wrote for Grizzly 1.0
>
> If you are busy because of any prior issues, just skip and forget my attched code and you don't need to review it. It's OK!
>
> PS) I kept original comments of JDK's ThreadPoolExecutor.
I never liked those concurrent classes as I always found there are a
little slower that my original Pipeline....Can you file an RFE and
attach your code? We should target 1.9.16.
Thanks!
--Jeanfrancois
>
> Thanks.
>
> --
> Bongjae Chang
>
>
> ----- Original Message -----
> From: "rama" <rama.rama_at_tiscali.it>
> To: <dev_at_grizzly.dev.java.net>
> Sent: Friday, May 08, 2009 3:48 AM
> Subject: Re: Default thread pool size issue
>
>
>> I know that this could sound stupid, since i have do the request for
>> that, but... i can't agree more :D
>>
>> Also, having the pipelinethreadpool --> defaultthreadpool will
>> remove some "warning" like "huh, you can't enable stats 'cos you are
>> dumb" or something like that.
>> At last, having it working out of the box, could improve the
>> performance for heavy loaded env, without needing any workaround.
>>
>> The only issue that i see, at the moment, is that the
>> pipelinethreadpool is a bit more heavyweight rather than
>> defaultthread pool.
>> I know that this will sound stupid, but on my test, when the number
>> of connection is very low, defaultthreadpool is faster than
>> pipelinethreadpool (can't say why, just some ab test, not too much
>> profiling, probably someone would like to take a look to
>> pipelinethreadpool if he can squeeze out more performance, but i
>> suppose that the overhead is due to check if the worker thread are busy)
>>
>> anyway, let's go for pipelinethreadpool!! :)
>>
>>
>>
>>
>>> Hi,
>>>
>>> after latest thread pool related questions, I really start to be
>>> upset with DefaultThreadPool implementation, which is based on JDK
>>> ThreadPoolExecutor.
>>> Really, thread pool, which prefers to add tasks to queue until it
>>> gets full, IMHO, doesn't look useful for us.
>>>
>>> We do have another implementation PipelineThreadPool (called
>>> Pipeline because of former Grizzly 1.x thread pool API), which
>>> prefers to create new worker, until possible, and then start to add
>>> tasks to queue.
>>>
>>> May be we can do some "refactoring" and make PipelineThreadPool
>>> default one?
>>>
>>> What do you think?
>>>
>>> WBR,
>>> Alexey.
>>>
>>> On May 7, 2009, at 16:10 , Bongjae Chang wrote:
>>>
>>>> Hi Jeanfrancois,
>>>>
>>>> Thank you for your patch.
>>>>
>>>> I saw your patch code now. But I think that max-pool-size's
>>>> modification is maybe not enough because
>>>> java.util.concurrent.ThreadPoolExecutor only allows thread's
>>>> increment when its queue is full.
>>>> (Of course, when current thread count is not under core-pool-size)
>>>>
>>>> So I think that core-pool-size is also modified if Grizzly still
>>>> uses java.util.concurrent.ThreadPoolExecutor without thread
>>>> increment's policy.
>>>>
>>>> Thanks.
>>>>
>>>> --
>>>> Bongjae Chang
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Jeanfrancois Arcand" <Jeanfrancois.Arcand_at_Sun.COM>
>>>> To: <dev_at_grizzly.dev.java.net>
>>>> Sent: Thursday, May 07, 2009 10:52 PM
>>>> Subject: Re: Default thread pool size issue
>>>>
>>>>
>>>>> Salut,
>>>>>
>>>>> I've re-opened
>>>>>
>>>>> https://grizzly.dev.java.net/issues/show_bug.cgi?id=566
>>>>>
>>>>> And fixed the issue. Thanks Bongjae!
>>>>>
>>>>> -- Jeanfrancois
>>>>>
>>>>> Jeanfrancois Arcand wrote:
>>>>>> Salut,
>>>>>>
>>>>>> Justin Lee wrote:
>>>>>>> That's actually what I thought when I read his email, too.
>>>>>>> Makes sense.
>>>>>>> Should that formula be used in grizzly-config, too?
>>>>>> I would say yes. Look at Controller.autoConfigureCore() logic.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> -- Jeanfrancois
>>>>>>
>>>>>>
>>>>>>> Jeanfrancois Arcand wrote:
>>>>>>>> Salut,
>>>>>>>>
>>>>>>>> Bongjae Chang wrote:
>>>>>>>>
>>>>>>>>> Hi.
>>>>>>>>>
>>>>>>>>> When I tested TCP layer recently in grizzly, I met some
>>>>>>>>> problems of
>>>>>>>>> Cotroller's initialization which had been not occurred before.
>>>>>>>>>
>>>>>>>>> Here is some examples
>>>>>>>>> - ControllerStateListener's onReady() was not invoked in main
>>>>>>>>> Controller.
>>>>>>>>> - When I tried to connect a tcp connection, maybe timeout was
>>>>>>>>> occurred
>>>>>>>>> and I failed to send the packet to the remote server.
>>>>>>>>> - etc...
>>>>>>>>>
>>>>>>>>> When I debugged this problem, I could know that this was the
>>>>>>>>> side-effect of "Auto-configure the number of ReaderController
>>>>>>>>> based on
>>>>>>>>> OS/Core or the machine" issue(svn rev.3088, issue #566).
>>>>>>>>>
>>>>>>>>> I could know that 4 ReaderControllers was initialized when I
>>>>>>>>> had tested
>>>>>>>>> this on 4 CPUs. Before issue #566 was committed,
>>>>>>>>> ReaderController was
>>>>>>>>> not used because default read-thread-count was 0.
>>>>>>>>>
>>>>>>>>> As you know, default thread pool is shared between main
>>>>>>>>> Controller and
>>>>>>>>> ReaderController and default thread pool's core size and max
>>>>>>>>> size are 5.
>>>>>>>>>
>>>>>>>>> Then if gizzly is used on 3~4 more CPUs, I think that many
>>>>>>>>> tasks of
>>>>>>>>> grizzly can not be executing but queueing in thread pool
>>>>>>>>> executor.
>>>>>>>>>
>>>>>>>>> So, I think that default thread pool size is not enough and
>>>>>>>>> default
>>>>>>>>> size should also be configured automatically or at least
>>>>>>>>> warning message
>>>>>>>>> should be logged if the minimum thread's number for
>>>>>>>>> initialization was
>>>>>>>>> not guaranteed.
>>>>>>>>>
>>>>>>>>> When I tested this on my thread pool which had 11 core size
>>>>>>>>> and 11 max
>>>>>>>>> size, I could confim no problems.
>>>>>>>>>
>>>>>>>> Completely agree. I think the number of threads in that case
>>>>>>>> should be
>>>>>>>> core * default value. What do you think?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> -- Jeanfrancois
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Bongjae Chang
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> -----------------------------------------------------------------
>>>>>>>> ----
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>>>>>>>
>>>>>>>>
>>>>>>> ------------------------------------------------------------------
>>>>>>> ---
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>>>>>>
>>>>>> -------------------------------------------------------------------
>>>>>> --
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>>>>>
>>>>> --------------------------------------------------------------------
>>>>> -
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: dev-help_at_grizzly.dev.java.net