Hi Chris,
Jersey is internally blocking - if you dig deeper into the code, you'll
see that request and response entities are represented as Java
Input/OutputStream. This is coming from JAX-RS API and we currently
don't have any simple way how to get rid of blocking approach.
(FYI: there is a planned feature for JAX-RS 2.1 which would allow to
write and access REST resources in non-blocking fashion)
Executing requests in "worker" thread is standard approach by any http
containers - usually there are two thread pools - selector and worker.
Netty itself does what selector thread is supposed to do, but there is
no worker thread pool. If Jersey would just use Netty threads to run
itself, it would mean that the server cannot process more parallel
requests than Netty pool, because all threads would be blocked by Jersey
blocking operations.
Anyway, since you already started with some analysis, can you use some
profiling tools to see where is the bottleneck?
Thanks and regards,
Pavel
On 02/01/2017 17:04, Chris Berry wrote:
> Greetings,
>
> Upfront — thanks very much for the excellent Jersey project.
> We’ve been using it for years at my day job. It is a godsend!
>
> So my question…
>
> I recently built a “dropwizard-netty-bundle”. And I was extremely
> pleased that I could incorporate Jersey onto Netty so cleanly.
> It boiled down to a relatively small amount of code.
>
> To accomplish this; I used this:
> https://github.com/jersey/jersey/tree/master/containers/netty-http/src/main/java/org/glassfish/jersey/netty
>
> And based it on this:
> https://github.com/jersey/jersey/tree/master/examples/helloworld-netty
>
> And it works great — except, sadly, it performs terribly.
>
> I compared my code with a Jetty/Jersey version of exactly the same
> Resource
> (A simple GET Resource that returns; “OK”)
>
> #### Jetty/Jersey -- using the alive page and the `wrk` program
> ```
> cberry@localhost ~ $ wrk -d30s --latency http://localhost:8080/alive.txt
> Running 30s test @ http://localhost:8080/alive.txt
> 2 threads and 10 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency 339.17us 1.56ms 43.71ms 98.87%
> Req/Sec 20.83k 3.38k 28.61k 74.88%
> Latency Distribution
> 50% 194.00us
> 75% 235.00us
> 90% 292.00us
> 99% 2.14ms
> 1246019 requests in 30.10s, 231.72MB read
> Requests/sec: 41393.01
> Transfer/sec: 7.70MB
> ```
>
> #### Netty/Jersey (with an apples to apples alive.txt)
> ```
> cberry_at_localhost ~ $ wrk -d30s --latency
> http://localhost:8007/v1/alive.txt
> Running 30s test @ http://localhost:8007/v1/alive.txt
> 2 threads and 10 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency 649.52us 4.52ms 118.48ms 98.77%
> Req/Sec 16.49k 1.74k 19.55k 81.83%
> Latency Distribution
> 50% 284.00us
> 75% 304.00us
> 90% 342.00us
> 99% 8.05ms
> 984994 requests in 30.03s, 84.54MB read
> Requests/sec: 32801.29
> Transfer/sec: 2.82MB
> ```
>
> ~21% less throughput. 2.75X slower.
>
>
> #### BUT when I run it WITHOUT Jersey
> ```
> cberry@localhost ~ $ wrk -d30s --latency http://localhost:8007/helloworld
> Running 30s test @ http://localhost:8007/helloworld
> 2 threads and 10 connections
> Thread Stats Avg Stdev Max +/- Stdev
> Latency 138.29us 59.75us 5.40ms 97.83%
> Req/Sec 35.96k 4.76k 41.01k 74.09%
> Latency Distribution
> 50% 124.00us
> 75% 149.00us
> 90% 178.00us
> 99% 208.00us
> 2153924 requests in 30.10s, 199.25MB read
> Requests/sec: 71558.68
> Transfer/sec: 6.62MB
> ```
> a whopping _72K requests/sec_ (as I’d expect — 1.7X the throughput —
> at a much faster 99%tile)
>
> I _think_ that this code:
> https://github.com/jersey/jersey/blob/2.25/containers/netty-http/src/main/java/org/glassfish/jersey/netty/httpserver/JerseyServerHandler.java#L112
> is the reason why!
>
> It looks like it is firing another Thread each time for Jersey??
> So it makes sense why Netty/Jersey is slightly slower !!
> (At least by my read)
>
> So, my question, Am I doing something terribly wrong?
> Meaning — Should Netty/Jersey perform as raw Netty is expected too??
> Or are there fundamental reasons why it will not??
> Or do I have it configured/built incorrectly??
> (I realize that I’d need to send code to truly answer that question
> :~) — But, in general?
>
> Thank you for your assistance.
> Cheers,
> — Chris