Hi Tiran,
can you pls. create a simple client/server reproducer for the problem,
which includes SSL configuration, so I know I have the same scenario as
you do?
IMO we have to understand what exactly JSSE doing that takes so long time.
Thank you.
WBR,
Alexey.
On 07.07.15 06:49, Meltser Tiran wrote:
>
> Hi Alexey,
>
> In theory you are right.
>
> Unfortunately we don’t control the IMAP clients and from what we see
> in the field, they perform short sessions…
>
> Do you know perhaps on any special settings in Grizzly/Java that can
> optimize the SSL handshake (SSL acceleration)?
>
> *Tiran Meltser***
> System Architect
> Global Products & Operations
> *Comverse* – /Making Your Network Smarter/
>
> T +972-3-7678381
> M +972-54-5639381
> Tiran.Meltser_at_comverse.com <mailto:Tiran.Meltser_at_comverse.com>
> *www.comverse.com <http://www.comverse.com/> *
>
> *P****Please think of the environment before printing this email***
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Tuesday, July 07, 2015 4:32 PM
> *To:* users_at_grizzly.java.net
> *Subject:* Re: High CPU Usage on SSL utils
>
> Sorry, I meant IMAP. AFAIU there is keep-alive mechanism IMAP IDLE,
> which could be used to keep connection open and receive server
> notification once there is a new email.
> I believe the main problem you have is initial SSL handshake, which
> really could be expensive, and I think you can rid off, or reduce the
> effect of the expensive SSL handshake by using IMAP keep-alive mechanism.
>
> WBR,
> Alexey.
>
> On 06.07.15 14:53, Ben-Yosef Efrat wrote:
>
> Hi Alexey,
>
> We are not using SMTP, we are using IMAP Protocol.
>
> We also noticed that this issue does not occur if we reduce the
> key size (at first it was 2048, and we reduced it to 512).
>
> Of course we cannot run with 512. Any Ideas?
>
> Thanks
>
> Efrat
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Thursday, July 02, 2015 6:44 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: High CPU Usage on SSL utils
>
> Hi guys,
>
> IMO what you're currently measuring is SSL handshake performance
> vs. no-handshake in plain connection.
> I quickly googled for SMTP protocol and looks like there is
> possibility to keep connection alive, IMO it's essential,
> especially for secured use case.
>
> To be more fair, try to run Echo benchmark, but not with 10
> messages, but for example 100K or more, it should remove impact of
> SSL handshake.
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 30.06.15 16:05, Ben-Yosef Efrat wrote:
>
> Sorry, couldn’t attach the file. Please see it in
> ftp://transfer.comverse.com/outgoing/SSLVsPlain/SSLVsPlain.zip
>
> *From:*Ben-Yosef Efrat [mailto:Efrat.Ben-Yosef_at_comverse.com]
> *Sent:* Tuesday, June 30, 2015 4:41 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* RE: High CPU Usage on SSL utils
>
> Hi Alexey,
>
> We spent that last week on profiling this issue. We did the
> following:
> 1. Run our application on plain vs. Ssl when the application
> basically does nothing (respond with hard coded OK)
>
> 2. Run Grizzly SSL/Plain echo example under load
>
> 3. run Jprofiler, VisualVm and netbeans profiler
>
> All showed the same:
>
> * CPU increases dramatically – over 60% on some cases
>
> * memory increase by 40% and more on some cases
>
> * org.glassfish.grizzly.ssl.SSlUtils.executeDelegatedTask() is
> the most problematic methods CPU wise
>
> We cannot run our application this way at all. On ssl
> everything crashes after 20 min.
>
> Attached are some screenshots and the code for the ssl/Plain
> echo load test we run.
>
> Please advise
>
> Efrat
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Friday, June 19, 2015 8:18 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: High CPU Usage on SSL utils
>
> Hi Tiran,
>
> I see your concern, but IMO it's similar to what you found
> w.r.t. "selector", if you enable "core classes" profiling -
> you'll see that it's core SSL implementation consumes CPU, not
> Grizzly itself. Once you have the profiling results with core
> classes, we can check what exactly consumes that much time.
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 18.06.15 20:07, Meltser Tiran wrote:
>
> Hi Alexey (working with Efrat and Uri) J,
>
> First we don’t keep the connections open as this is an
> IMAP server (or proxy).
>
> Second, I agree that SSL channel should have some
> performance price but what we experience is far beyond a
> reasonable price, after making another check we see we
> have response times slower *600* times than plain
> communication…
>
> Third, we are running on a 16 cores machine in which SSL
> should hardly be noticed.
>
> Forth, we have just compared the SSL experience to our
> previous server written in C++ some more than 10 years ago
> and operated on machines with about ¼ CPU power (or even
> less), and load results show response time was doubled at
> max (*not 600 times worse*).
>
> Can you have a look on the hot method the profiler is
> showing and see if something can be done there (or maybe
> give us some guideline)?
>
> *Tiran Meltser***
> System Architect
> Global Products & Operations
> *Comverse* – /Making Your Network Smarter/
>
> T +972-3-7678381
> M +972-54-5639381
> Tiran.Meltser_at_comverse.com <mailto:Tiran.Meltser_at_comverse.com>
> *www.comverse.com <http://www.comverse.com/> *
>
> *P****Please think of the environment before printing this
> email***
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Thursday, June 18, 2015 8:16 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: High CPU Usage on SSL utils
>
> Hi Efrat,
>
> do you keep connections alive?
> In general SSL is not for free and it's expected to be
> slower than plain communication.
>
> WBR,
> Alexey.
>
> On 18.06.15 18:14, Ben-Yosef Efrat wrote:
>
> Also, we see higher response times then before (around
> 50 times longer)
>
> *From:*Ben-Yosef Efrat
> [mailto:Efrat.Ben-Yosef_at_comverse.com]
> *Sent:* Thursday, June 18, 2015 6:54 PM
> *To:* users_at_grizzly.java.net
> <mailto:users_at_grizzly.java.net>
> *Cc:* Broide Uri; Meltser Tiran
> *Subject:* High CPU Usage on SSL utils
>
> Hi Alexey,
>
> Continuing Uri’s query about the CPU, we moved to
> java8 profiler and as you said, indeed not the
> select() was the issue.
>
> Now, we added the SSL to the picture (utilizing
> Grizzly’s SSLfilter) and as you can see in the
> attached screenshot, the executeDelegatedTask() method
> consumes a lot of CPU (we sort by self-tile CPU).
>
> Please advise,
>
> Thanks,
>
> Efrat
>
> cid:image001.png_at_01D0A9F5.776D0720
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that constitutes
> proprietary information of Comverse Inc. or its
> subsidiaries. If you are not the intended recipient of
> this message, you are hereby notified that any review,
> use or distribution of this information is absolutely
> prohibited and we request that you delete all copies
> and contact us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that constitutes
> proprietary information of Comverse Inc. or its
> subsidiaries. If you are not the intended recipient of
> this message, you are hereby notified that any review,
> use or distribution of this information is absolutely
> prohibited and we request that you delete all copies
> and contact us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial
> or privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you
> are not the intended recipient of this message, you are
> hereby notified that any review, use or distribution of
> this information is absolutely prohibited and we request
> that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>.
> Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you are
> not the intended recipient of this message, you are hereby
> notified that any review, use or distribution of this
> information is absolutely prohibited and we request that you
> delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you are
> not the intended recipient of this message, you are hereby
> notified that any review, use or distribution of this
> information is absolutely prohibited and we request that you
> delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any
> review, use or distribution of this information is absolutely
> prohibited and we request that you delete all copies and contact
> us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any review,
> use or distribution of this information is absolutely prohibited and
> we request that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com. Thank You.”