On 29 March 2012 20:18, Remy Maucherat <rmaucher_at_redhat.com> wrote:
> Looking about this new reply makes me doubt your sincerity a bit.
As always your attitude and inability (or unwillingness) to comprehend
the point of view of others makes it very tiresome to deal with you.
But in most previous cases you have eventually seen the light after
many months of repetition, so I will persist.
> I keep the rest of the buffer, say it was written (so that the
> application does not have to deal with it),
Firstly you can't just keep the rest of the buffer, as you have no
idea as to it's volatility or otherwise after you return from the
write method. You have to copy the rest of the buffer.
> and flip the canWrite flag
> to false (so that the application knows it should stop writing until
> told otherwise using a callback). When it is possible to write without
> blocking (so when the socket polling says so) the remaining bytes will
> be written, the canWrite flag is flipped back to true, and the
> application will get its notification. There is NO internal backlogging.
Sure you can try to write the passed buffer directly to the channel,
and then only copy the remaining data. But producers and consumers
rarely run at the same speed so either you will be producing faster
than the network can consume, in which case the copy will be frequent,
or the production is slower, in which case it is made even more
inefficient by the need to pass in the data in smaller chunks than the
network could otherwise consume.
Besides, it is much better to just let the app pass the entire content
buffer and then try to write as much as possible. The app can then
keep the unwritten data and it knows the volatility of the buffer, so
it can copy it or not as needed.
> I can give an example for better understanding:
> while (os.canWriteCount() > 4KB) {
> os.write(my4KBbuffer);
> os.flush();
> }
> // write callback
> while (os.canWriteCount() > 4KB) {
> os.write(my4KBbuffer);
> os.flush();
> }
> // write callback
> while (os.canWriteCount() > 4KB) {
> os.write(my4KBbuffer);
> os.flush();
> }
Well that's ugly and inefficient! The application has to both break
the data up into little consumable chunks and be prepared to write it
either in a loop or via call back style.
It is a needless complication to limit the amount that can be written
to the amount that the implementation is prepared to buffer in the
event that the network is blocked. It is far more efficient and far
simpler to do either:
os.write(my20KBuffer, completionhandler);
or
os.write(my20KBuffer); // 8k of the 20k is written
// write callback
os.write(my20KBuffer); // next 8k of the 20k is written
// write callback
os.write(my20KBuffer); // remaining 4k is written
> The flush will cause the actual write, since the Servlet layer has its
> own buffer, I'm using it for clarity so that it is known when it occurs.
But you were just saying that you would try to directly write the
content passed in the write and only buffer it it was unwriteable?
So I don't see what the flush is doing - either the data was already
written, or it was buffered because it could not be written.
> Apparently, your issue with Rajiv's proposal is that it is possible to
> implement it inefficiently,
No - my problem is that it requires the implementation to provide an
un-knowable number - how many bytes can be written without blocking.
Thus implementations will have to guess a number that is likely to be
much less than what can be written (or if it is large will contribute
to buffer bloat). This will constrain applications to chunk their
own content and always write less that could actually be written.
> while your proposal is always inefficient.
You have not established that. Both proposals that I have advocated
have less data copying, less system calls, larger writes and fewer EE
wrapped callbacks than the current proposal.
> Of course, if both were equally inefficient, then the NIO 2 API style is
> simpler and should be chosen. But that's really not the case.
>
> Last, and you may have missed it, it has been clarified that each of
> these callbacks should have the full EE environment set. So in addition
> to the trip to the thread pool, they will have a cost, and being able to
> minimize them is good.
Exactly! In this example the NIO2 style has the least callbacks.
Both canwrite as boolean and canwrite as int can have the same number
of callbacks, but canwrite as int requires the application to
implement a complex loop inside a callback aproach.
> Simple read/write boolean flags are probably enough, and that's what I
> have right now. My problem with your argumentation is that using an int
> doesn't really change anything to efficiency (the boolean is actually
> still there anyway if the application prefers it !), it simply gives an
> extra information to the application if, as in my implementation, it
> wants to avoid using leftovers (the extra copy you didn't like). So what
> is the problem ?
The problem is that the int is an artificially created limit, above
which the application cannot write. Instead of saying
write(bigbuffer) and letting the impl take as much data as possible,
the app has to loop doing while(canWrite)write(chunk) for absolutely
no gain.
Finally, I have no idea why you feel the need to question my sincerity
or motivations for discussing the various options. Even if I had been
wrong in the points that I raised, perhaps I had misunderstood the
proposed design and this could have been pointed out in polite dialog.
We need to get this right and there is no harm taking a bit of time
to discuss the actual usage and implementation of the proposed
designs. Instead you appear extremely annoyed at any suggestion that
might take the design away from what you have already implemented -
perhaps it is you that has the agenda?
We can't have a process that is a) design proposed b) Remy says he has
implemented it c) finished!
I note that few others are joining discussion in the EG and I
suggest that it is your caustic attitude towards alternate points of
view that is a disincentive to others asking questions or
contributing.