users@jersey.java.net

[Jersey] Re: threads left hanging in jersey 2.14

From: Jon Marston <jon.marston_at_englishcentral.com>
Date: Tue, 6 Jan 2015 11:49:31 -0500

Thanks for moving this forward.

I’ll keep an eye on the HK2 ticket but for now will stay on Jersey 2.13. The stuck threads tend to make it difficult to redeploy quickly to our development environments.

jon

> On Jan 6, 2015, at 9:42 AM, Jakub Podlesak <jakub.podlesak_at_oracle.com> wrote:
>
> FYI: John has filed https://java.net/jira/browse/HK2-241 <https://java.net/jira/browse/HK2-241>
>
> ~Jakub
>
>> On 06 Jan 2015, at 12:01, Jakub Podlesak <jakub.podlesak_at_oracle.com <mailto:jakub.podlesak_at_oracle.com>> wrote:
>>
>> Hi Jon,
>>
>> The thread locals you see in the log have been created by the HK2 framework.
>> Could you please report the issue there, ideally accompanied with a reproducible test case?
>> Well you might just copy the P.S. section bellow now :-)
>>
>> https://java.net/jira/browse/HK2 <https://java.net/jira/browse/HK2>
>>
>> I am ccing John Wells, the lead of the HK2 project.
>>
>> @John: the actual HK2 version used in Jersey is 2.4.0-b06, could you please have a look at this?
>> Maybe we should be cleaning something up in Jersey and we do not do that properly at the moment?
>> HK2 version used in Jersey 2.13 had been 2.3.0-b10
>>
>> Thank you in advance!
>>
>> ~Jakub
>>
>> P.S. I have seen the hanging thread locals also when running the following Jersey example on Tomcat:
>> https://github.com/jersey/jersey/tree/master/examples/helloworld-webapp <https://github.com/jersey/jersey/tree/master/examples/helloworld-webapp>
>> war file: http://search.maven.org/remotecontent?filepath=org/glassfish/jersey/examples/helloworld-webapp/2.14/helloworld-webapp-2.14.war <http://search.maven.org/remotecontent?filepath=org/glassfish/jersey/examples/helloworld-webapp/2.14/helloworld-webapp-2.14.war>
>>
>> Here is how to reproduce:
>>
>> cp ~/Downloads/helloworld-webapp-2.14.war ~/opt/apache-tomcat-8.0.15/webapps
>> # wait until the above deploys, then
>> curl http://localhost:8080/helloworld-webapp-2.14/helloworld <http://localhost:8080/helloworld-webapp-2.14/helloworld>
>> # and undeploy:
>> rm ~/opt/apache-tomcat-8.0.15/webapps/helloworld-webapp-2.14.war
>>
>> tail ~/opt/apache-tomcat-8.0.15/logs/catalina.out
>>
>> 06-Jan-2015 11:41:53.824 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [helloworld-webapp] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$3] (value [org.jvnet.hk2.internal.Utilities$3_at_5ae72909]) and a value of type [java.util.WeakHashMap] (value [{protected org.glassfish.jersey.server.internal.process.ServerProcessingBinder$ContainerRequestFactory(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_5602000}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>> 06-Jan-2015 11:41:53.824 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [helloworld-webapp] created a ThreadLocal with key of type [org.jvnet.hk2.internal.FactoryCreator$1] (value [org.jvnet.hk2.internal.FactoryCreator$1_at_286b7cfc]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>> 06-Jan-2015 11:41:53.824 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [helloworld-webapp] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$2] (value [org.jvnet.hk2.internal.Utilities$2_at_12a4525b]) and a value of type [java.util.WeakHashMap] (value [{}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>
>> this war does not cause the left over: http://search.maven.org/remotecontent?filepath=org/glassfish/jersey/examples/helloworld-webapp/2.13/helloworld-webapp-2.13.war <http://search.maven.org/remotecontent?filepath=org/glassfish/jersey/examples/helloworld-webapp/2.13/helloworld-webapp-2.13.war>
>>
>>
>>> On 04 Jan 2015, at 02:30, Jon Marston <jon.marston_at_englishcentral.com <mailto:jon.marston_at_englishcentral.com>> wrote:
>>>
>>> Hello,
>>>
>>> I’m running a Jersey application using Java 7 and Tomcat 7.0.55 container.
>>>
>>> When I upgrade the dependencies from Jersey 2.13 to 2.14, i wind up getting lots of hanging threads when I shut down my server. Boot up and operation are fine, but I have problems stopping the server cleanly.
>>>
>>> With 2.14, is there something additional I need to shut down in my Bootstrap contextDestroyed handler?
>>>
>>> here are the sample thread warnings i see when shutting down my server
>>>
>>>
>>>
>>>
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$2] (value [org.jvnet.hk2.internal.Utilities$2_at_4df24a4f]) and a value of type [java.util.WeakHashMap] (value [{}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.FactoryCreator$1] (value [org.jvnet.hk2.internal.FactoryCreator$1_at_26ce3318]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$3] (value [org.jvnet.hk2.internal.Utilities$3_at_2b7c808f]) and a value of type [java.util.WeakHashMap] (value [{public org.glassfish.jersey.internal.ContextResolverFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_7adc429, public org.glassfish.jersey.servlet.WebComponent$HttpServletRequestReferencingFactory(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_1297c4d, protected org.glassfish.jersey.server.internal.process.ServerProcessingBinder$ContainerRequestFactory(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_29fdd45}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$2] (value [org.jvnet.hk2.internal.Utilities$2_at_4df24a4f]) and a value of type [java.util.WeakHashMap] (value [{}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.FactoryCreator$1] (value [org.jvnet.hk2.internal.FactoryCreator$1_at_26ce3318]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$2] (value [org.jvnet.hk2.internal.Utilities$2_at_4df24a4f]) and a value of type [java.util.WeakHashMap] (value [{}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.FactoryCreator$1] (value [org.jvnet.hk2.internal.FactoryCreator$1_at_26ce3318]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>> Jan 03, 2015 8:13:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
>>> SEVERE: The web application [/bridge] created a ThreadLocal with key of type [org.jvnet.hk2.internal.Utilities$3] (value [org.jvnet.hk2.internal.Utilities$3_at_2b7c808f]) and a value of type [java.util.WeakHashMap] (value [{public org.glassfish.jersey.message.internal.XmlJaxbElementProvider$General(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3b65f83, public org.glassfish.jersey.server.internal.inject.BeanParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3a38935, private javax.ws.rs.container.ResourceContext org.glassfish.jersey.server.model.internal.VoidVoidDispatcherProvider.resourceC, private javax.inject.Provider org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher, private org.glassfish.jersey.server.model.internal.ResourceMethodDispatcherFactory org.glassfish.jersey.server.model.ResourceMethodInvoker$Builder.dispatcherProvi, public org.glassfish.jersey.server.internal.inject.HeaderParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_56f0796, private org.glassfish.jersey.process.internal.RequestScope org.glassfish.jersey.server.ServerRuntime$Builder.requestScope=org.jvnet.hk2.in, public org.glassfish.jersey.message.internal.SourceProvider$SourceWriter(org.glassfish.hk2.api.Factory,org.glassfish.hk2.api.Factory)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3a96f64, public org.glassfish.jersey.message.internal.XmlRootElementJaxbProvider$General(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_d028bb7, public org.glassfish.jersey.message.internal.XmlRootElementJaxbProvider$App(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_2222a20, public com.babelcentral.bridge.JerseyApplication(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_fbd14fd, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider, protected javax.ws.rs.ext.Providers com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider._providers=org.jvnet.hk2.i, private javax.inject.Provider org.glassfish.jersey.internal.JaxrsProviders.workers=org.jvnet.hk2.internal.Uti, public org.glassfish.jersey.server.internal.RuntimeExecutorsBinder$BackgroundSchedulerFactory(org.glassfish.jersey.spi.RuntimeThreadProvider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_221a3de, public org.glassfish.jersey.server.internal.inject.FormParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_68d5139, private javax.inject.Provider org.glassfish.jersey.server.internal.inject.AbstractContainerRequestValueFactor, org.glassfish.jersey.server.internal.JerseyResourceContext(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_dc9767, public org.glassfish.jersey.server.internal.inject.AsyncResponseValueFactoryProvider(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_1257eba, public org.glassfish.jersey.server.internal.inject.ParamConverters$AggregatedProvider(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_4a2ee21, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.server.internal.inject.ParamInjectionResolver.locator=org., public org.glassfish.jersey.message.internal.XmlCollectionJaxbProvider$General(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_5d81d78, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.server.internal.inject.BeanParamValueFactoryProvider.locat, private javax.ws.rs.core.Configuration org.glassfish.jersey.server.ServerRuntime$Builder.configuration=org.jvnet.hk2.i, public org.glassfish.jersey.message.internal.XmlJaxbElementProvider$App(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_7528419, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.server.model.ResourceMethodInvoker$Builder.locator=org.jvn, private org.glassfish.jersey.spi.ExceptionMappers org.glassfish.jersey.server.ServerRuntime$Builder.exceptionMappers=org.jvnet.hk, public org.glassfish.jersey.message.internal.SourceProvider$DomSourceReader(org.glassfish.hk2.api.Factory)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_1481556, org.glassfish.jersey.server.model.internal.ResourceMethodDispatcherFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_370da60, private javax.inject.Provider org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.containerReques, private javax.inject.Provider org.glassfish.jersey.message.internal.DocumentProvider.tf=org.jvnet.hk2.interna, private javax.inject.Provider org.glassfish.jersey.message.internal.DocumentProvider.dbf=org.jvnet.hk2.intern, public org.glassfish.jersey.message.internal.XmlRootObjectJaxbProvider$App(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_592bf41, public org.glassfish.jersey.message.internal.XmlRootObjectJaxbProvider$Text(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_54a4f77, public org.glassfish.jersey.server.internal.routing.RuntimeModelBuilder(org.glassfish.jersey.server.model.ResourceMethodInvoker$Builder,org.glassfish.hk2.api.ServiceLocator,org.glassfish.jersey.message.MessageBodyWorkers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_1a73960, private org.glassfish.jersey.process.internal.RequestExecutorFactory org.glassfish.jersey.server.ServerRuntime$Builder.asyncExecutorFactory=org.jvne, public org.glassfish.jersey.server.internal.inject.WebTargetValueFactoryProvider(org.glassfish.hk2.api.ServiceLocator,javax.ws.rs.core.Configuration)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_7e46ee2, public org.glassfish.jersey.message.internal.SourceProvider$SaxSourceReader(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_62becb7, private java.util.concurrent.ScheduledExecutorService org.glassfish.jersey.server.ServerRuntime$Builder.backgroundScheduler=org.jvnet, org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_5303d9d, public org.glassfish.jersey.message.internal.XmlCollectionJaxbProvider$App(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_21d3e37, public org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorFactory(org.glassfish.jersey.server.internal.inject.ParamConverterFactory)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_5df9211, public org.glassfish.jersey.server.internal.inject.DelegatedInjectionValueFactoryProvider(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_ee9d1a3, public org.glassfish.jersey.server.internal.inject.QueryParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_18c22bf, public org.glassfish.jersey.internal.ExceptionMapperFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_73d5786, public org.glassfish.jersey.server.internal.inject.MatrixParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_43d5fad, public org.glassfish.jersey.message.internal.XmlCollectionJaxbProvider$Text(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_154613c, private javax.inject.Provider org.glassfish.jersey.internal.JaxrsProviders.mappers=org.jvnet.hk2.internal.Uti, public org.glassfish.jersey.server.internal.inject.PathParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_44687a7, public void org.glassfish.jersey.message.internal.AbstractJaxbProvider.setConfiguration(javax.ws.rs.core.Configuration)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3273e72, public org.glassfish.jersey.message.internal.XmlJaxbElementProvider$Text(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_574b830, public org.glassfish.jersey.message.internal.XmlRootObjectJaxbProvider$General(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_1e8938e, private javax.inject.Provider org.glassfish.jersey.internal.JaxrsProviders.resolvers=org.jvnet.hk2.internal.U, private org.jvnet.hk2.internal.DynamicConfigurationServiceImpl(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_6c0764f, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.internal.inject.ContextInjectionResolver.serviceLocator=or, private javax.inject.Provider org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher, org.glassfish.jersey.internal.inject.JerseyClassAnalyzer(org.glassfish.hk2.api.ClassAnalyzer,org.glassfish.hk2.api.IterableProvider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_2e09805, org.glassfish.jersey.server.internal.inject.ParamConverterFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_71c9ac5, public org.glassfish.jersey.server.ContainerMessageBodyWorkersInitializer(javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_6f5075d, public org.glassfish.jersey.message.internal.XmlRootElementJaxbProvider$Text(org.glassfish.hk2.api.Factory,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_b69744e, org.glassfish.jersey.server.internal.inject.EntityParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3852b2b, public org.glassfish.jersey.server.internal.process.ServerManagedAsyncExecutorFactory(org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3d54718, private org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory org.glassfish.jersey.server.model.ResourceMethodInvoker$Builder.invocationHandl, public org.glassfish.jersey.server.internal.inject.CookieParamValueFactoryProvider(org.glassfish.jersey.server.internal.inject.MultivaluedParameterExtractorProvider,org.glassfish.hk2.api.ServiceLocator)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_3d22a9b, public org.glassfish.jersey.server.internal.inject.JaxbStringReaderProvider$RootElementProvider(javax.inject.Provider,javax.ws.rs.ext.Providers)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_2eb3462, public org.glassfish.jersey.message.internal.MessageBodyFactory(org.glassfish.hk2.api.ServiceLocator,javax.ws.rs.core.Configuration)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_7bc0f6d, org.glassfish.jersey.server.internal.process.ReferencesInitializer(org.glassfish.hk2.api.ServiceLocator,javax.inject.Provider)=org.jvnet.hk2.internal.Utilities$SoftAnnotatedElementAnnotationInfo_at_398cd61, private javax.ws.rs.core.Configuration org.glassfish.jersey.server.model.ResourceMethodInvoker$Builder.globalConfig=or, private org.glassfish.hk2.api.ServiceLocator org.glassfish.jersey.server.ServerRuntime$Builder.locator=org.jvnet.hk2.interna}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>>>
>>
>