Status/Resolution/Reason: Closed/Fixed/HaveNewInfo
Reporter/Name(from Bugbase): Dave DeVol / Dave DeVol ()
Created: 04/20/2017
Components: Charting/Graphing, Server
Versions: 2016,2018
Failure Type: Memory Leak
Found In Build/Fixed In Build: Latest / CF2018U5
Priority/Frequency: Normal / Some users will encounter
Locale/System: English / Win 2012 Server x64
Vote Count: 0
Problem Description:
Previous versions of CF used to have the ability to set Threads in the CFAdmin, CF2016 does not, although this is still available in the adminapi. The docs says you can set this from 1 to 5. If CF2016 supports it, you should add it back to the CFAdmin.
You can't set a custom Disk Cache location in CFAdmin. I can't set it in CFAdmin, the neo-graphing.xml, or the adminapi. It always resets back to the default location.
Changing the settings CFAdmin-> Server Settings -> Charting requires a restart to take affect. There should be an alert saying so.
The Time-To-Live only appears to work when cache type is set to Disk Cache and only for the images on disk. This setting appears to be ignored when set to Memory. If Time-To-Live is set in CFAdmin, it should apply to both Memory and Disk. Since you are using Ehcache to cache the charts, it seems that the "TimeToLiveSeconds" in Ehcache should be set to what's in CFAdmin. Maybe have a cleanup process that runs evictExpiredElements() function in Ehcache. Right now, Ehcache will not expire elements until the Max Elements In Memory is hit. Might want to add a MaxBytesInMemory setting.
When you set to Disk Cache, EHCache Max Elements In Memory are set to 1 and Disk Persistence: true. The timeToLiveSecods is 0, which means forever and timeToIdleSeconds is 0, also forever. Although "Maximum number of cached images" don't seem to apply to the Disk Cache, but it does seem to apply to the Memory Cache.
It doesn't seem to matter if you set it to Disk Cache or Memory Cache, the cfchart cache images seem to generate in both places. This means disk and memory activity. I'd think it would be one or the other. When set to Disk Cache, you set "MaxelementsInMemory" to 1 and "OverflowToDisk" in Ehcache, but I've noticed that the Ehcache elements in memory often exceed the 1. Once it does this, there seems to be no limit. This can cause a "memory leak", since there is no real limit.
Although you can create the exact same chart over and over, it never really uses the previous chart from the cache like its suppose to, it keeps creating new ones. I can create hundreds of the same charts that get "cached" to Ehcache.
Even if you set timeToIdleSeconds or timeToLiveSeconds in EHcache, there is no thread that will come along and expire those unless they get accessed with a get(), then they return null and get expired. This isn't happening with the charts, so they just sit in the CF_Chart_CacheManager. It then waits to hit your maxElementsInMemory for EHcache to expire them. Ultimately what's happening is that all these cfcharts are getting added to the EHcache Memory and not being removed until you hit a max. This is causing memory issues on my server.
I also suspect that there may be a memory leak in the jvm heap related to all of this. I haven't gotten proof yet, but I did a heap dump with JProfiler and I had about 5G's of charts in memory. Even if I maxed out my Max Elements In Memory, it shouldn’t have used all that memory. Maybe caching charts to Ehcache just takes up a lot of memory?
Steps to Reproduce:
Try to set a custom Disk Cache Path for Charts in CFAdmin. Try to set a time-to-live for memory charts and then review the timeToLiveSeconds in Ehcache configuration. Generate a chart and watch it get cached to disk and memory, no matter what the setting is. Try to generate the same chart multiple times and watch how it keeps creating new ones on disk and memory. Try changing Threads in the CFAdmin.
Actual Result:
Expected Result:
Any Workarounds: For me, I set a low "Max" cached charts (less than 100) in the CFAdmin to limit the amount of memory it uses. I've not let the server run long enough yet to test this in production to see if there is a true memory leak or if previously it was just that 1500 Max cached charts used 5GB of memory.
Attachments:
Comments: