tracker issue : CF-3934737

select a category, or use search below
(searches all categories and all time range)
Title:

CFHTTP does not always abort after timeout interval

| View in Tracker

Status/Resolution/Reason: Closed/Withdrawn/CannotReproduce

Reporter/Name(from Bugbase): Tom Chiverton / Tom Chiverton (Tom Chiverton)

Created: 02/05/2015

Components: Performance

Versions: 11.0

Failure Type: Memory Leak

Found In Build/Fixed In Build: CF11_Final /

Priority/Frequency: Normal / Some users will encounter

Locale/System: ALL / Linux All

Vote Count: 2

Even with setting a timeout on CFHTTP calls, sometimes the thread sticks and doesn't terminate untill CF is restarted.

We set a timeout of 30, but have requests that run into tens of thousands of seconds.

In an example calling the Twitter API (https://api.twitter.com/1.1/statuses/user_timeline.json?....) Fusion Reactor reports the stack trace as :
java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)[Native Method]
java.net.SocketInputStream.read(SocketInputStream.java:152)
java.net.SocketInputStream.read(SocketInputStream.java:122)
sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
sun.security.ssl.InputRecord.read(InputRecord.java:480)
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
 - locked <0x18a21f4> (a java.lang.Object)
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
 - locked <0x1bb2431> (a java.lang.Object)
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275)
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254)
org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:123)
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:318)
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
coldfusion.tagext.net.HttpTag.createConnection(HttpTag.java:2160)
coldfusion.tagext.net.HttpTag.connHelper(HttpTag.java:1077)
coldfusion.tagext.net.HttpTag.doEndTag(HttpTag.java:1372)

Locked ownable synchronizers: 
    - java.util.concurrent.ThreadPoolExecutor$Worker@1fb61ed

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3934737

External Customer Info:
External Company:  
External Customer Name: Tom
External Customer Email:

Attachments:

Comments:

With provided information, I was unable to repro the case. Http request on timeout does not lead to issue mentioned. Please provide more details to repro.
Comment by Akhila K.
8581 | February 17, 2015 07:07:57 AM GMT
It doesn't happen with every request. I suspect you need the remote end to fail to respond in just the right way, e.g. keeping the TCP/IP connection open but not transferring anything
Comment by External U.
8582 | February 17, 2015 07:43:35 AM GMT
Yes We have that issue when I calling Adobe API from my coldfusion site
Vote by External U.
8586 | April 02, 2015 09:09:18 AM GMT
Please consider the following example: testCFHttpTimeout.cfm (this script has to execute for longer than 1 second, increase index if necessary): <cfloop index="randomindex" from="1" to="500000" step="1"> <cfoutput>a</cfoutput><cfflush> </cfloop> testAG.cfm: <cfsetting requesttimeout="30" showdebugoutput="Yes" enablecfoutputonly="Yes"> <cfoutput> #now()# <cfhttp url="https://www.berlitzvirtualclassroom.com/testing/testCFhttpTimeout.cfm" method="GET" timeout="1"></cfhttp> <h1>https://www.berlitzvirtualclassroom.com/testing/testCFhttpTimeout.cfm Timeout Test : timeout seconds: 1</h1> #now()# </cfoutput> When running testAG.cfm the result is: {ts '2015-04-13 09:33:23'} https://www.berlitzvirtualclassroom.com/testing/testCFhttpTimeout.cfm Timeout Test : timeout seconds: 1 {ts '2015-04-13 09:33:32'} expected result (script should take 1 second only): {ts '2015-04-13 09:33:23'} https://www.berlitzvirtualclassroom.com/testing/testCFhttpTimeout.cfm Timeout Test : timeout seconds: 1 {ts '2015-04-13 09:33:24'} This is a problem for us because we can hit the same issue in case of poor networking conditions.
Comment by External U.
8583 | April 13, 2015 08:11:15 AM GMT
This happens when having poor network conditions. Socket seem to be stuck in read state. This leads to thread pool exhaustion eventually when multiple requests are stuck this way.
Vote by External U.
8587 | April 13, 2015 08:12:45 AM GMT
Tried with provided information. We were not able to reproduce this issue and it is working as expected. RequestTimedOut Exception was thrown. Result after executing testAG.cfm and testCFHttpTimeout.cfm : The request has exceeded the allowable time limit Tag: cfhttp We are closing this bug as we were not able to repro, if you face same issue again please get in touch with us with more information. We will re-open the bug by then. Thanks.
Comment by Subir J.
8584 | September 24, 2015 01:53:39 AM GMT
Here's another stack trace with the same symptom (a CFHTTP with timeout=1 goes on for thousands of seconds). FusionReactor claims there is a lock : Lock 0: java.util.concurrent.ThreadPoolExecutor$Worker@1f17460 If you can't reproduce, at least take a look at the stack and try to figure out what is causing the hang ? java.net.SocketOutputStream.socketWrite0(SocketOutputStream.java:???)[Native Method] java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109) java.net.SocketOutputStream.write(SocketOutputStream.java:153) org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:123) org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:157) org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115) org.apache.http.entity.ByteArrayEntity.writeTo(ByteArrayEntity.java:112) org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:155) org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:149) org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:236) org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121) org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254) org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86) org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) coldfusion.tagext.net.HttpTag.createConnection(HttpTag.java:2169) coldfusion.tagext.net.HttpTag.connHelper(HttpTag.java:1077) coldfusion.tagext.net.HttpTag.doEndTag(HttpTag.java:1372)
Comment by External U.
8585 | September 29, 2015 05:00:17 AM GMT