tracker issue : CF-3850033

select a category, or use search below
(searches all categories and all time range)
Title:

Unable to import large CF9 settings (~1500 data sources)

| View in Tracker

Status/Resolution/Reason: Closed/Fixed/

Reporter/Name(from Bugbase): Tom Chiverton / Tom Chiverton (Tom Chiverton)

Created: 11/11/2014

Components: Installation/Config, Config

Versions: 11.0

Failure Type: Incorrect w/Workaround

Found In Build/Fixed In Build: CF11_Final / 300103

Priority/Frequency: Normal / Few users will encounter

Locale/System: ALL / Linux All

Vote Count: 0

Listed in the version 2016.0.03.300466 Issues Fixed doc 
Problem Description:
When importing my CF9 settings, the migration wizard hung for exactly 50 minutes, then stopped with a time out.
Restarting the wizard the same thing happens again.

Steps to Reproduce:
Create a CF9 install with many data sources.
Choose to import this during CF11 initial install.

Actual Result:
Unable to migrate settings, unable to access CFIDE after install.

Expected Result:
CF should process in batches so as not to hit the timeout, or run with no timeout during the migration.
CF should mark each DSN/setting as migrated so it can at least quickly find out where it got to last time and continue.

Any Workarounds:
Import speed can be improved by setting -Djava.security.egd=file:/dev/./urandom in jvm.config. If this isn't enough, you may have to change
<var name='timeoutRequests'><boolean value='true'/>
in neo-runtime.xml to be false. 
All by hand because the CFIDE isn't accessible.
If the migration does succeed (around 2 and a half hours in our case) the CFIDE login will have timed out, so when you press 'next' after the list of migrated items the whole process starts again after you log back in.
Edit adminconfig.xml to prevent the migration going off again.

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3850033

External Customer Info:
External Company:  
External Customer Name: ChivertonT
External Customer Email:  
External Test Config: My Hardware and Environment details:

Attachments:

Comments:

I am unable to observe the issue when migrating the settings from CF10 (with ~2000 DSNs) to CF11. When you run the CF installer it creates a folder called cf10settings/cf9settings in <cfroot>/cfusion/lib. When the the migration wizard runs it merges the xml files in that folder with the xml files in the new cf installation in cfusion/lib. In my case the neo-datasource.xml created was ~7MB in size, and the migration wizard took a few minutes to migrate the settings and I did not observe any errors. I am not sure at this point if the DSNs are verified for availability at the time of migration or not. But can you confirm a few things? Do you have a stanalone or a J2EE installation of CF9? Are you importing any other settings (that can possibly be causing the issue) besides the the DSNs? Were there any errors reported in the process. You can probably check the migration.log file. What is the size of your neo-datasources.xml file.
Comment by Piyush K.
10179 | November 14, 2014 08:37:41 AM GMT
Standalone CF9. Yes, I am importing all settings from CF9 to CF11, as we're upgrading. There were no errors, other than those listed, and left alone it completes fine. It is the DSNs holding it up as the stack traces I obtain from FusionReactor are in setDsn() in the admin API, and it's the neo-datasource.xml that is being changed. Nothing in migration.log other than the time out errors from old runs. CF9 neo-datasource.xml is 10757563 CF11 one post import is 12481457
Comment by External U.
10180 | November 14, 2014 10:17:51 AM GMT
Did you happen to retain those stack trace from FusionReactor? Can you please share that? Can you also confirm if the neo*.xml config files were all copied to the CF9settings folder in cf_root of the new CF server installation.
Comment by Piyush K.
10181 | December 03, 2014 06:22:45 AM GMT
Yes, the files were correctly copied. Were you migrating on CentOS 5 ? I can't get a stack trace now, because we've got our CF9 system sync'ing DSN changes to our CF11 system so we don't have to wait for hours during an eventual upgrade.
Comment by External U.
10182 | December 04, 2014 03:30:54 AM GMT
can we get the setup and thread dump?
Comment by Deepraj J.
10183 | September 25, 2015 07:51:48 AM GMT
9 months later ? No, sorry, we don't have that machine any more.
Comment by External U.
10184 | September 26, 2015 02:57:49 AM GMT
lol
Comment by External U.
10185 | September 28, 2015 02:10:32 AM GMT
If you are looking for fix for this, please contact support. Details of the fix: ---------------------- When there are large set of data sources to import (Say more than 1000 data sources), import fails with timeout repetitively. This can happen for 1 ColdFusion reason and 2 external environmental reasons: One is more time (Many Minutes together) is taken to import. Default Request Time out is set to 60 seconds. And import is not completed by this time This can happen on any platforms say Windows or Linux OSes. Fix is such that the request is completed in less than a minute even when there are 1000+ data sources. Two is: The default Entropy (-Djava.security.egd=/dev/urandom) set in jvm.config is not available in the Linux OS. Changing it to -Djava.security.egd=file:/dev/./urandom fixes it. Since this is OS specific you will have to set this. Because of encryption of passwords is involved for data source import the time is getting multiplied. Third is: On Linux machines there are defaults limits on how many threads can be created by a user. And for each data source there is a daemon thread created. This limit has to be increased depending on your number of data sources. If import fails because of this reason, the error thrown in exception.log is: java.lang.OutOfMemoryError: unable to create new native thread Fix for this is: Change the limits in the file: /etc/security/limits.conf for the ColdFusion's runtime user. In the following example, nobody is the runtime user of ColdFusion server is running on. nobody soft nproc 3000 nobody hard nproc 3000
Comment by Nitin K.
10186 | November 13, 2015 06:20:59 AM GMT
Since the fix has implications on the behaviour when the dsns are edited from admin/loaded while starting, this fix would kick in with system property flag based. Flag to add in jvm.config : -Dcoldfusion.migrate.dsnloadonce=true Also datasources changes persist after restarting CF (#CF-4101765).
Comment by Nitin K.
10187 | September 01, 2016 05:22:19 AM GMT
Hi Adobe, Not that I plan to monitor this ticket, but I have a suggestion: You should document all of CF's JVM args somewhere. I don't think tracker.adobe.com serves as CF's documentation, so how would people know about -Dcoldfusion.migrate.dsnloadonce? Thanks!, -Aaron
Comment by Aaron N.
27272 | April 03, 2018 05:52:36 AM GMT