On 2 April 2012 19:23, Remy Maucherat <rmaucher_at_redhat.com> wrote:
> On Mon, 2012-04-02 at 15:17 +1000, Greg Wilkins wrote:
>> On 31 March 2012 02:38, Remy Maucherat <rmaucher_at_redhat.com> wrote:
>> >> No you can't. The max size of the write is imposed by the internal
>> >> buffer of the implementation and the writer does not get to chose the
>> >> size of the write. As you pointed out, the internal buffers are
>> >> likely to be small, so you probably can't increase it to 20K!. Even if
>> >> they know their entity is 4k+1 bytes, then they have to do it in 2
>> >> writes if can writes returns 4k.
>> >
>> > So, it is actually: yes I can.
>> >
>> > The container will do non blocking writes until it exhausts the 20KB
>> > buffer. Two options then:
>> > a) If it doesn't manage to write all 20K bytes, it flips the canWrite
>> > flag, and stores the remaining bytes that couldn't be read in a leftover
>> > buffer. This uses some amount of memory (< 20 KB), but the application
>> > buffer is given back to it (it may reuse it, or discard it, its choice).
>>
>> This approach breaks TCP/IP flow control as there is no back pressure
>> to a producer that is faster than the network and/or consumer. If
>> the writer thinks that the while 20K has been written, then there is
>> nothing stopping it writing another 20k and another etc. So your
>> internal copy will just grow and grow and grow!
>>
>> For writes you need to stop the producer somehow. Either you block it
>> in the call to write, or you return a length < the buffer size so that
>> it knows it is writing to fast and can stop producing until it get's a
>> call back.
>
> As the a) item of the algorithm indicates, the canWrite flag is flipped.
So you pass a large buffer to the IO layer and it flushes it
asynchronously and then calls back the application when it is
finished. That is more or less the NIO.2 style, but with a needless
copy! However it does have the ability to poll the IO layer after a
write operation to see if a another write can be done or if a callback
is needed - this is a good improvement (see my response to Oleksiy).
>> > Thanks for the clarification on the real purpose of what you are proposing.
>> huh? Why do you always think I'm plotting one thing or another. I'm
>> just trying to discuss the pros and cons of various async IO styles.
>
> Yes, I do think that. And I think I have reasons for that.
Geeze you are seriously paranoid. No plots here. No secret plans.
No hidden agendas. I'm not even sure what option I like best out of
a) no async IO; b) traditional canWrite style; c) NIO.2 Style; d) Some
new hybrid style. All I know is that we should discuss all the
proposals and test them with real examples and analysis.