tracker issue : CF-4205271

Vikram, Can you pls ask the customer to try using the following flag in JVM config -Dcoldfusion.http.usepooling=false

Comment by Piyush K.

We are seeing similar issues on 2016 Patch 12 - does this work around apply to that version as well?

Comment by Mike P.

This issue is occurring in our environments as well. We will try the JVM argument provided and see if the behavior changes.

Comment by dakota c.

Can anyone from Adobe clarify what this argument is doing? and how new is it? New to this update, or been there for years? Also, is this specific to hangups in cfhttp processing, or some more generic hangup?

Comment by Charlie A.

Charlie, I am wondering the same thing. My bug report was closed in favor of this one and I don't even know if they are the same. We need some feedback from Adobe on what to do. I have a bunch of ColdFusion servers that are stuck on ColdFusion 2018 Update 4 because of this.

Comment by Dave C.

And also please clarify if this suggestion is for 2016 Patch 12 as well as 2018 Patch 5.

Comment by Mike P.

Mike, This JVM flag is only applicable to CF2018. You should not face this issue in CF2016. If you are still experiencing Server hang issue after applying CF 2016 Update 12, it might be because of a different reason. Please reach out to ColdFusion support @ [cfinstal@adobe.com|mailto:cfinstal@adobe.com] with more information so that we can investigate this issue further. -Nimit

Comment by Nimit S.

Dave/Dakota, The JVM argument in discussion, that is suggested as a stopgap measure here, disables HTTP connection pooling, as is evident from it's name. This means that every time an HTTP call is made, instead of reusing a readily available connection from a pool of connections, a new one will be created (and subsequently terminated once it has served it's purpose). For further details you may reference the "Pooling connection manager" section at https://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/connmgmt.html This is not suggested as a solution for any CF server may be running into performance issues. A CF server can become unresponsive for any number of reasons. The best way to determine the root cause is to collect thread dumps of the underlying JVM. We've suggested this workaround only for the folks, for whom, we've examined the logs and found HTTP connection allocation mechanism to be the problem. Here's an example of the stacktrace from a connection starved thread: "http-nio-10192-exec-9" #163 daemon prio=5 tid=0x00007fdc408cf800 nid=0x34e7 waiting on condition [0x00007fdb9bcdd000] java.lang.Thread.State: WAITING (parking) JavaThread state: _thread_blocked - jdk.internal.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise) - parking to wait for <0x00000000d9088e08> (a java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject) - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=194 (Compiled frame) - java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() @bci=42, line=2075 (Compiled frame) - org.apache.http.pool.PoolEntryFuture.await(java.util.Date) @bci=48, line=138 (Compiled frame) - org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(java.lang.Object, java.lang.Object, long, java.util.concurrent.TimeUnit, org.apache.http.pool.PoolEntryFuture) @bci=479, line=306 (Compiled frame) - org.apache.http.pool.AbstractConnPool.access$000(org.apache.http.pool.AbstractConnPool, java.lang.Object, java.lang.Object, long, java.util.concurrent.TimeUnit, org.apache.http.pool.PoolEntryFuture) @bci=8, line=64 (Compiled frame) - org.apache.http.pool.AbstractConnPool$2.getPoolEntry(long, java.util.concurrent.TimeUnit) @bci=15, line=192 (Compiled frame) - org.apache.http.pool.AbstractConnPool$2.getPoolEntry(long, java.util.concurrent.TimeUnit) @bci=3, line=185 (Compiled frame) - org.apache.http.pool.PoolEntryFuture.get(long, java.util.concurrent.TimeUnit) @bci=45, line=107 (Compiled frame) - org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(java.util.concurrent.Future, long, java.util.concurrent.TimeUnit)

Comment by Piyush K.

Hi Piyush, Is this a stopgap measure for CFHTTP randomly hanging as well? My stacktrace is much different than the one above. See bug report: https://tracker.adobe.com/#/view/CF-4205439

Comment by Dave C.

Dave, I'm sorry if I wasn't clear before. It is especially for CFHTTP. Although HTTPClient would normally be the underlying library used in anything that involves making an HTTP call. I don't see the full thread dumps attached with CF-4205439, but the single thread stack trace shared in the report there, looks like the thread is waiting for a connection from the pool, same as the thread in the example I quoted earlier. "org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)"  

Comment by Piyush K.

Hi Piyush, Thanks for the explanation. Is there a fix coming for this in the next patch? If so, how close is it?

Comment by Dave C.

Dave, That's the plan. No timelines yet. Can you pls share the file mentioned in your threadDump (E:\WEBROOT\americanfloormats.com\global\extensions\components\lloyd_mats.cfc:21)). I would like to see how the HTTP call is being made in your code. You may mail that to me at pnayak@adobe.com

Comment by Piyush K.

Thanks for the explanation, Piyush. (Somehow I didn't get or didn't see the notifications about these last few replies.) I see that update 6 is due to fix this: https://coldfusion.adobe.com/2019/11/preview-builds-coldfusion-2018-release-update-6-and-coldfusion-2016-release-update-13-released/ Can you confirm that those who added this arg should be able to remove it, after applying that fix?

Comment by Charlie A.

Also, shouldn't the status and fixed in build values be changed here?

Comment by Charlie A.