tracker issue : CF-4038087

Issue occurs in development and production, on both ColdFusion 10 and 11. Stock JRE or JRE 1.8.0_45 - memory leak still exists. No problem in ColdFusion 9 or earlier - all sites were upgraded to ColdFusion 10 and 11 over the last couple years. Microsoft JDBC Driver 4.0 may be a sufficient workaround, but we need more long term testing before we would be comfortable using a different database driver in production. The driver included with ColdFusion should work, as it used to pre-ColdFusion 10.

Comment by External U.

Hi Brieanna, Please share below mentioned information: 1. Settings Summary 2. Is there a specfic query which is introducing this memory leak? 3. Is it possible if you can share some Sample website using which we can try to replicate this issue? 4. OS version used 5. Database server version used 6. JDK version used 7. Have you tried using Microsoft's sql server driver? If yes, did you observe any memory leak issue?

Comment by Nimit S.

Hi Brieanna, Please provide me below mentioned information so that we can start investigating this issue.

Comment by Nimit S.

ColdFusion Summary: System Information Server Details Server Product ColdFusion Version 11,0,06,295053 Tomcat Version 7.0.54.0 Edition Enterprise Operating System Windows Server 2008 R2 OS Version 6.1 Update Level C:/ColdFusion11/cfusion/lib/updates/chf11000006.jar Adobe Driver Version 5.1.3 (Build 000094) JVM Details Java Version 1.8.0_45 Java Vendor Oracle Corporation Java Vendor URL http://java.oracle.com/ Java Home C:\Program Files\Java\jdk1.8.0_45\jre Java File Encoding Cp1252 Java Default Locale en_US File Separator \ Path Separator ; Line Separator Chr(13) User Name CFService User Home C:\Users\CFService User Dir C:\ColdFusion11\cfusion\bin Java VM Specification Version 1.8 Java VM Specification Vendor Oracle Corporation Java VM Specification Name Java Virtual Machine Specification Java VM Version 25.45-b02 Java VM Vendor Oracle Corporation Java VM Name Java HotSpot(TM) 64-Bit Server VM Java Specification Version 1.8 Java Specification Vendor Oracle Corporation Java Specification Name Java Platform API Specification Java Class Version 52.0 CF Server Java Class Path <removed big block of text> Java Class Path C:\ColdFusion11\cfusion\lib\oosdk\lib; C:\ColdFusion11\cfusion\lib\oosdk\classes; C:\ColdFusion11\cfusion\bin\..\runtime\bin\tomcat-juli.jar; C:\ColdFusion11\cfusion\bin\cf-bootstrap.jar Java Ext Dirs C:\Program Files\Java\jdk1.8.0_45\jre\lib\ext;C:\Windows\Sun\Java\lib\ext Printer Details Default Printer Microsoft XPS Document Writer Printers Microsoft XPS Document Writer ------------------------------------- We have narrowed down one instance of the problem to the following query. <cfquery datasource = "#request.dsn#" name = "local.blog"> SELECT * FROM BLOG B LEFT JOIN BLOG_CATEGORY BC ON ( BC.ID_BLOG_CATEGORY = B.ID_BLOG_CATEGORY AND BC.SOFTDELETE = <cfqueryparam cfsqltype = "cf_sql_bit" value = "False" /> ) ORDER BY B.RELEASE_DATE DESC </cfquery> If cfqueryparam is removed from this query, the memory leak goes away. The below works: <cfquery datasource = "#request.dsn#" name = "local.blog"> SELECT * FROM BLOG B LEFT JOIN BLOG_CATEGORY BC ON ( BC.ID_BLOG_CATEGORY = B.ID_BLOG_CATEGORY AND BC.SOFTDELETE = 'false' ) ORDER BY B.RELEASE_DATE DESC </cfquery> ------------------------------------- OS is Windows Server 2008 R2. Database server is SQL 2008 R2 Express. JDK is 1.8.0_45 as listed above. We have tried Microsoft's SQL driver and ColdFusion no longer appeared to have any DeleteOnExitHook references hanging out in the heap. All the information in this paragraph was already provided when the ticket was opened. As mentioned, we have this problem in multiple environments, with both CF10 and CF11, and with both the included JRE, and 1.8.0_45 which we have upgraded to. We have a stripped down website running on a virtual machine with this error (one blank page that hits the database, empty database with exception of two records in one table). We can coordinate getting Adobe access to the environment if needed.

Comment by External U.

<cfqueryparam cfsqltype = "cf_sql_bit" value = "False" /> has a memory leak <cfqueryparam value = "False" /> appears to be fine Any known issues with cfsqltype?

Comment by External U.

Hi Brieanna, Is it possible if you can generate heapdump when OOME occurs and share with us for further investigation? Please use jvm flags mentioned below in order to capture heapdump whenever OOME occurs in your development server where you are able to repro this issue. -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${DOMAIN_HOME}/logs/ Please share your jvm.config file as well. You can send jvm.config file at nimsharm@adobe.com. Regards, Nimit

Comment by Nimit S.

E-mail sent with access details for PRODUCTION heap dump created yesterday, and jvm.config.

Comment by External U.

Any update on this bug?

Comment by External U.

We have supplied Adobe with a heap dump, jvm.config, a stripped website, and a stripped database. The website has one functioning page - it does nothing but connect to the database and perform a simple query. The one page load leaves one java.io.DeleteOnExitHook in the heap. Refreshing the page 1000 times leaves 1000 entries. The supplied environment continues to have this memory leak in a freshly installed environment with Windows 2008 R2 X64 and default ColdFusion 11 production install with all Windows and ColdFusion updates applied. An employee with Adobe responded via E-mail at the end of October stating they are unable to replicate the issue, and that they are working with their database-driver vendor to investigate the issue. We were supposed to receive an "update on this issue soon", but it has been 4 weeks since last communication. We tried Microsoft's SQL Server driver - while it doesn't have this memory leak, it has other inconsistencies that cause problems in our testing. We would prefer a fix from Adobe rather than update code for compatibility with Microsoft's SQL Server driver. Those are the updates that we have on this database driver bug.

Comment by External U.

Any update on this bug? We are seeing this in production on our ColdFusion 11 servers with the latest hotfix installed. Eclipse Memory Analyzer is pointing at java.io.DeleteOnExitHook's LinkedHashMap. Example entry key: C:\Users\cfusion\AppData\Local\Temp\ddtb2194090160619159364.tmp

Comment by External U.

We have sent test environment login details to Adobe so they can see the issue first-hand. We are now waiting to hear back from the person handling this bug report.

Comment by External U.

ColdFusion 11 Update 9 released yesterday appears to have fixed this memory leak! 120,000 requests and increasing in our CF11 testing environment and no sign of "class java.io.DeleteOnExitHook" entries in the dominator tree. ColdFusion 10 Update 20 and ColdFusion 2016 Update 2 build 299200 were also released.

Comment by External U.

I'll keep a look out to make sure it is actually resolved. We have multiple servers that don't run into this issue at over 120,000 requests and other servers that run into it at 120,000 requests. It appears to only happen for specific situations. I'm working on applying the latest hotfix to all of our servers.

Comment by External U.

Thanks Brieanna for the information. We will wait for your confirmation on this issue.

Comment by Nimit S.

We are up to 3.3 million requests and there are no heap issues. Dominator tree is clean.

Comment by External U.

ColdFusion 11 Update 9 has fixed this issue for us. As requested, the ticket can be closed.

Comment by External U.

Thanks Brieanna for the confirmation. I am closing this ticket.

Comment by Nimit S.