PerimeterX leaks threads on webapp shutdown

[mads1980]

The async HTTP client is not shutdown, so many threads remain live on shutdown.
This also happens with the TimerConfigUpdater Timer thread which is instantiated, but is never stopped.

01-May-2019 12:25:21.131 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [pool-51-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.141 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.150 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.159 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.168 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.177 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 4] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.187 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 5] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.196 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 6] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.205 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 7] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.215 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 8] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.224 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 9] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.233 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 10] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.242 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 11] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.251 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.260 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 13] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.270 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 14] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.279 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 15] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.288 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 16] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.297 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 17] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.308 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 18] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.318 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 19] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.327 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 20] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.337 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 21] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.346 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 22] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.356 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 23] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.366 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 24] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.375 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 25] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.384 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 26] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.394 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 27] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.403 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 28] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.413 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 29] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.423 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 30] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.432 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 31] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
01-May-2019 12:25:21.441 WARNING [tomcat-http-36] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [I/O dispatcher 32] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

[pxjohnny]

Hi @mads1980,
Thank you for taking the time to notify us on this issue. We will look into it ASAP and let you know of any changes.
Thanks!

[mads1980]

Threads also leak from the following classes:

• TimerValidateRequestsQueue (timer task is never cancelled)
• PXHttpClient: httpClient, asyncHttpClient, nHttpConnectionManager,

[ricardonevesmindera]

@mads1980 I am having some trouble with CPU leak while using this SDK. However after a quick inspect in class PXHttpClient all resources (httpClient, asyncHttpClient) seem to be closed either with a finally statement or with a try with resources since those implement Closeable interface.
Can you give some more details with about the problem that you found in nHttpConnectionManager?
Also what is the issue with the TimerValidateRequestsQueue?

Regards,
Ricardo Neves

[mads1980]

perimeterx_patched_classes.zip

[mads1980]

See above the patched classes that fix the different thread leakage issues. Those were patched against v6.1.4 of the Java SDK.

[ricardonevesmindera]

@mads1980 Thank you very much for the response. I guess at the time that you made those fixes, you haven't created a PR to the SDK, right? I see some improvements on 6.4.5 version, but I still gave thread leakage issues. I'll try to create a custom version from the 6.4.5 version and make some tests.

Thanks again!

[ricardonevesmindera]

@pxjohnny If you have the availability also take a look at those files