tracker issue : CF-3366102

select a category, or use search below
(searches all categories and all time range)
Title:

throttled CFHTTP

| View in Tracker

Status/Resolution/Reason: Closed/Won't Fix/Workaround

Reporter/Name(from Bugbase): Shuie Yankelewitz / Shuie Yankelewitz (Shuie Yankelewitz)

Created: 11/15/2012

Components: Net Protocols, HTTP

Versions: 10.0

Failure Type: Performance Issue

Found In Build/Fixed In Build: Final /

Priority/Frequency: Major / Some users will encounter

Locale/System: English / Linux Ubuntu 10.10

Vote Count: 3

Problem Description: Multiple CFHTTP request executed in succession result in each subsequent CFHTTP call being throttled approximately 5 seconds more than the prior call. 

Steps to Reproduce: Create a loop that runs 10 times and calls and outputs the results of a CFHTTP tag.

Actual Result: From the logs:

HTTP request completed {Status Code=200 ,Time taken=28525 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-18 	
HTTP request completed {Status Code=200 ,Time taken=22752 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-23 	
HTTP request completed {Status Code=200 ,Time taken=17549 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-17 	
HTTP request completed {Status Code=200 ,Time taken=12320 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-22 	
HTTP request completed {Status Code=200 ,Time taken=7137 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-25 	
HTTP request completed {Status Code=200 ,Time taken=4302 ms}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-23 	
Starting HTTP request {URL='http://www.google.com', method='GET'}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-17 	
Starting HTTP request {URL='http://www.google.com', method='GET'}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-22 	
Starting HTTP request {URL='http://www.google.com', method='GET'}
Sep 10, 2012 	5:13 PM 	Information 	ajp-bio-8012-exec-19 	
Starting HTTP request {URL='http://www.google.com', method='GET'}

Expected Result:
Should be returning results without the delay

Any Workarounds: downgrade to CF9

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3366102

External Customer Info:
External Company:  
External Customer Name: shudini1
External Customer Email:  
External Test Config: Ubuntu Server 12.04. Apache 2.4

Attachments:

Comments:

posted in forum before submitting bug report: http://forums.adobe.com/message/4850062
Comment by External U.
17187 | November 15, 2012 12:53:08 PM GMT
I am also stuck downgrading. A process that I run on a CF9 rhel5 system ran in just over 6 minutes. On a CF10 rhel6 system with MUCH better hardware It takes about 68 minutes. Tcpdumps on the servers shows that the response time of the website is about the same (within ms of each other), however cfhttp times on CF10 are all about 5 seconds. I want the new javascript stuff in CF10 but can't until this is fixed.
Vote by External U.
17202 | November 15, 2012 01:20:00 PM GMT
Also happens on RHEL systems.
Comment by External U.
17188 | November 15, 2012 01:20:35 PM GMT
This bug is seriously affecting automated page preview captures our software does when designers are implementing designs. Something which used to be sub second is now taking 20+seconds and frequently timing out.
Vote by External U.
17203 | November 15, 2012 05:28:22 PM GMT
This may sound crazy, but I've seen weirder things in ColdFusion. It just might be that what you is expected behaviour, assuming single-threaded processing. Say, you have 10 cfhttp tags in a loop. When the loop starts, ColdFusion first processes a number of cfhttp start-tags, perhaps up to all 10 of them, possibly without strictly respecting order. Since the processing is single-threaded, ColdFusion executes the tag bodies in series, one after the other. This would explain why the successive execution times are cumulative.
Comment by External U.
17189 | November 16, 2012 06:36:31 AM GMT
This may sound crazy, but I've seen weirder things in ColdFusion. It just might be that what you observe is expected behaviour, assuming single-threaded processing. Say, you have 10 cfhttp tags in a loop. When the loop starts, ColdFusion first processes a number of cfhttp start-tags, perhaps up to all 10 of them, possibly without strictly respecting order. Since the processing is single-threaded, ColdFusion executes the tag bodies in series, one after the other. This would explain why the successive execution times are cumulative.
Comment by External U.
17190 | November 16, 2012 06:37:32 AM GMT
See Ben Nadel's blog on using cfthread with cfhttp. [ http://www.bennadel.com/blog/749-Learning-ColdFusion-8-CFThread-Part-II-Parallel-Threads.htm ] There he solves the problem by using multiple, asynchronous threads, like this: <cfloop index="intGet" from="1" to="10"> <!--- Start a new thread for this CFHttp call. ---> <cfthread action="run" name="objGet#intGet#"> <cfhttp method="GET" url="some_url" result="THREAD.Get#intGet#"> </cfthread> </cfloop> <!--- Now, we have to wait for all of concurrent threads to be joined before we can use the CFHttp results. ---> <cfloop index="intGet" from="1" to="10"> <cfthread action="join" name="objGet#intGet#" /> </cfloop>
Comment by External U.
17191 | November 16, 2012 06:48:59 AM GMT
tested using the multi threaded solution suggested by Ben Nadel [ http://www.bennadel.com/blog/749-Learning-ColdFusion-8-CFThread-Part-I I-Parallel-Threads.htm ]. Does not seem to make a difference. CF9: We Got 1000 Results in 0.72 seconds using CFHttp and CFThread CF10: We Got 1000 Results in 45.02 seconds using CFHttp and CFThread
Comment by External U.
17192 | November 16, 2012 08:08:24 AM GMT
Also seen on our CF10 update 3 instance. The only workaround we have is to use over aggressive caching and accept the slow performance.
Vote by External U.
17204 | November 17, 2012 04:40:33 AM GMT
Upgraded to CF10 update 4 with no change. Still being throttled.
Comment by External U.
17193 | November 19, 2012 11:30:57 AM GMT
A significant pattern has arisen: this seems to occur only on Linux systems.
Comment by External U.
17194 | November 20, 2012 02:49:36 AM GMT
On taking a thread dump we observed that in linux systems while executing http calls, the apache http client tries to generate some secure random number. In java generation of secure random generation is an expensive operation dependent on the "entropy" on the machine. This "entropy" is low in some newly installed linux systems. Hence the extra time to execute the call. The solution is to do either of the following 1. Set this system property to your JVM – if you are using standalone CF installation, you would set it in jvm.config. “-Djava.security.egd=file:/dev/./urandom” or 2. In $JAVA_HOME/jre/lib/security/java.security file, change the value of securerandom.source to file:/dev/./urandom You can also refer to this post by Shilpi and engg from the ColdFusion team which talks about this issuehttp://www.shilpikhariwal.com/2012/04/random-number-generation-in-unix.html (Comment added from ex-user id:aahmad)
Comment by Adobe D.
17195 | November 20, 2012 08:49:53 AM GMT
using option #1 below does resolve the issue on my new Ubuntu 12.04 machine.
Comment by External U.
17196 | November 21, 2012 08:19:01 AM GMT
I'm another person now left with the only option to downgrade to CF9 from CF10 here and we are using Update 12. We're running Centos and neither of the 'fix' workarounds have worked for me either.
Comment by External U.
17197 | December 18, 2013 05:24:08 AM GMT
@AlastairAcquire, could yoiu please confirm that the changes suggested by Altamish were indeed applied and it did not help? We have tried this on multiple machines and this seems to have resolved the issue. Check out this link for more details. http://www.shilpikhariwal.com/2012/04/random-number-generation-in-unix.html
Comment by Rupesh K.
17198 | December 23, 2013 09:54:54 AM GMT
When urandom setting was made as mentioned by Rupesh and Altamish, this issue was resolved.
Comment by Akhila K.
17199 | March 02, 2014 07:43:06 PM GMT
This has been a significant issue for us. Using urandom is not optimal since this it is less secure or random than random. However, it looks like this was updated recently to recommend downgrading instead of the insecure workaround of using urandom. Hopefully Adobe will do the right thing and reopen this bug and fix it. At least resolving it in CF11. Coldfusion competitive products do not seem to have this issue.
Comment by External U.
17200 | April 25, 2014 07:19:12 PM GMT
It is *critically* important that you use the strange-looking syntax file:/dev/./urandom. Since Java 5, there is a check in the source that looks for /dev/urandom and assumes you meant /dev/random so it ignores it! The /./ in the middle fakes out the special check so it will actually use /dev/urandom.
Comment by External U.
17201 | August 05, 2015 10:18:17 PM GMT