On 2/15/12 10:35 AM, Matthew Swift wrote:
> Hi Ryan,
>
> Thanks for looking into this.
>
> I've just tried testing the fix but unfortunately I'm hitting an
> exception while running our unit tests which I'm pretty sure is
> unrelated to your fix but, instead, triggered by some other changes
> which have been made to Grizzly since 2.1.7. Before spending too much
> time myself figuring out what's happening, perhaps if I explain what's
> going wrong you may be able to figure it out immediately. Here goes:
>
> In one of our tests we create a client and server. The client connects
> to the server using a plain connection and then dynamically installs
> an SSL filter as part of a StartTLS request. The client then
> authenticates using the DIGEST-MD5 SASL mechanism with SASL connection
> security enabled (it is possible to have two security layers installed
> in LDAP!), the result being that a new SASL filter is added to the
> filter chain. The client then sends an LDAP search query over the
> connection but it fails deep inside the Grizzly stack with an
> ArrayIndexOutOfBoundsException:
>
> Feb 15, 2012 7:06:32 PM
> org.glassfish.grizzly.filterchain.DefaultFilterChain execute
> WARNING: Exception during FilterChain execution
> java.lang.ArrayIndexOutOfBoundsException: 3
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain$FiltersState.clearState(DefaultFilterChain.java:640)
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.checkStoredMessage(DefaultFilterChain.java:523)
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:216)
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:149)
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:127)
> at
> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.write(DefaultFilterChain.java:404)
> at org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:354)
> at org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:346)
> at
> com.forgerock.opendj.ldap.LDAPConnection.searchAsync(LDAPConnection.java:748)
>
> Our code which dynamically adds the SSL/SASL filters looks like this:
>
> void installFilter(final Filter filter)
> {
> synchronized (stateLock)
> {
> // Updates to the filter chain cannot be synchronized with
> traversals, so
> // copy on write.
> FilterChain oldFilterChain = (FilterChain)
> connection.getProcessor();
> FilterChain newFilterChain = new DefaultFilterChain(oldFilterChain);
>
> // Install the filter in the new filter chain beneath the LDAP
> filter.
> newFilterChain.add(newFilterChain.size() - 1, filter);
> connection.setProcessor(newFilterChain);
> }
> }
>
> Initially every connection shares the same filter chain and
> installation of a filter causes it to be copied on write (see above).
> Running through the debugger shows that the FiltersState being cleared
> only contains 3 elements and is not in sync with the total number of
> filters now in the chain.
>
> In addition to the ArrayIndexOutOfBoundsException, I find it a little
> worrying that the FilterState exists at all since it is invalid once
> the FilterChain has been changed (e.g. the Filter order changed). Are
> we doing something wrong?
> This all works just fine in 2.1.7.
No, I think, like you said, you've just hit a side-effect of a fix that
was made to prevent memory leaks.
Would you mind opening an issue on this and either Alexey or myself can
look into it asap?
>
> Matt
>
>
> On 13 February 2012 23:31, Ryan Lubke <ryan.lubke_at_oracle.com
> <mailto:ryan.lubke_at_oracle.com>> wrote:
>
> Hi Matt,
>
> As a follow-up, I've committed an initial fix for parallel write
> issue in blocking mode.
> It should be available in tonight's 2.1.9-SNAPSHOT and
> 2.2.2-SNAPSHOT nightly builds (if you feel like confirming).
>
> -rl
>
>> Awesome - thanks for the info. :-)
>>
>> I'll take a look at this tomorrow (getting late here now) and
>> come back with any questions I have, but it looks pretty
>> straightforward from what you describe.
>>
>> Cheers,
>>
>> Matt
>>
>> On 6 February 2012 23:00, Ryan Lubke <ryan.lubke_at_oracle.com
>> <mailto:ryan.lubke_at_oracle.com>> wrote:
>>
>> Further details...
>>
>> <snip>
>>
>> Presumably there's some sort of bounded write queue.
>> </snip>
>>
>> In addition to the PushBackHandler, there are a couple of
>> ways to configure the size of the queue.
>>
>> See: AsyncQueueWriter.setMaxPendingBytesPerConnection(int) [1]
>>
>> By default, the queue will be limited to four times the size
>> of the socket write buffer size.
>>
>> This value may be initially set via
>> TCPNIOTransportBuilder.setMaxAsyncWriteQueueSizeInBytes(int) [2].
>>
>>
>>
>> [1]
>> http://java.net/projects/grizzly/sources/git/content/modules/grizzly/src/main/java/org/glassfish/grizzly/asyncqueue/AsyncQueueWriter.java?rev=15de8fef0a4f138636e222f861ae6cb1fcf111f9
>>
>> [2]
>> http://java.net/projects/grizzly/sources/git/content/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOTransportBuilder.java?rev=15de8fef0a4f138636e222f861ae6cb1fcf111f9
>>
>>
>
>