tracker issue : CF-4205448

select a category, or use search below
(searches all categories and all time range)
Title:

Connector problems: "Tomcat is down or refused connection"

| View in Tracker

Status/Resolution/Reason: To Track//PRNeedInfo

Reporter/Name(from Bugbase): A. B. / ()

Created: 10/16/2019

Components: Installation/Config, Connector

Versions: 2018

Failure Type: Non Functioning

Found In Build/Fixed In Build: 2018.0.05.315699 /

Priority/Frequency: Normal / Most users will encounter

Locale/System: / Win 2016

Vote Count: 3

Problem Description:
We are unable to launch web requests on one of our ColdFusion innstances. The error in the isapi_redirect.log reads:

[Wed Oct 16 11:14:36.380 2019] [4732:12764] [info] ajp_connection_tcp_get_message::jk_ajp_common.c (1425): (t1studiereaderci) can't receive the response header message from tomcat, tomcat (127.0.0.1:8020) has forced a connection close for socket 1524
[Wed Oct 16 11:14:36.381 2019] [4732:12764] [error] ajp_get_reply::jk_ajp_common.c (2402): (t1studiereaderci) Tomcat is down or refused connection. No response has been sent to the client (yet)
[Wed Oct 16 11:14:36.382 2019] [4732:12764] [error] ajp_service::jk_ajp_common.c (2975): (t1studiereaderci) sending request to tomcat failed (unrecoverable),  (attempt=1)
[Wed Oct 16 11:14:36.382 2019] [4732:12764] [error] HttpExtensionProc::jk_isapi_plugin.c (2739): service() failed with http error 502

(Thanks to my colleague, Ben Ferdinandus)

We believe that this issue is related to the ISAPI connector issue discussed in: 

https://tracker.adobe.com/#/view/cf-4205252 
https://www.carehart.org/blog/client/index.cfm/2019/9/26/hold_off_on_sep_24_cf_updates 

We tested the patched isapi_redirect.dll which your team suggests, namely

https://coldfusion.adobe.com/2019/10/error-accessing-coldfusion-administrator-using-connector-port/ 
https://www.dropbox.com/sh/q2hv32vuw25oiho/AADpPdNBKC82IXXIxBM60m-ya?dl=0 

but these didn’t help.

We would therefore be grateful if you could send us a suitable isapi_redirect.dll patch.


Steps to Reproduce:
The change we made was from Update 4 to  Update 5 of ColdFusion 2018.

Actual Result:
No web requests processed. Instead, errors obtained as described above.

Expected Result:
Requests being processed normally, without any Tomcat/connector errors.

Any Workarounds:
None yet known

Attachments:

Comments:

Hi,   Can you share the isapi_redirect.log file? Also, is it happening for all your instances? Or is it happening just in 1 instance? Did you restart the Application Pool after the dll was changed (manually)?   Thanks, Kailash
Comment by Kailash B.
31612 | October 16, 2019 01:10:48 PM GMT
Hi Kailash, I attached the log file. Is it correct that the connector version numbert didn't change after replacing the dll? We did restart the application pool. We even did an iisreset.
Comment by Dik K.
31613 | October 16, 2019 03:23:01 PM GMT
Hi,   Can you change the connector port and try again? * Goto your connector magic folder ([CF Home]\config\wsconfig\[Magic folder] * Open workers.properties and change worker.cfusion.port (suppose its 8025 after the change) * Open server.xml ([CF Home]\cfusion\runtime\conf) * Change the connector port present in the block (for example, make the value 8018 to 8025): * <Connector port="_*8018*_" packetSize="65535" protocol="AJP/1.3" redirectPort="8451" tomcatAuthentication="false" maxThreads="500" connectionTimeout="60000"/> * Restart ColdFusion server, as well as the IIS Application Pool * Try to access web pages again   Thanks, Kailash  
Comment by Kailash B.
31614 | October 16, 2019 04:05:26 PM GMT
Hi Kailash, seems like changing the portnumber solved the connector problem. We will continue testing and let you know if the 502 errors disappeared.
Comment by Dik K.
31621 | October 16, 2019 06:13:09 PM GMT
Hi,   It seems the port you were using was blocked somehow. Changing it got it working for you. I have closed the bug for now. Please get back to us if you hit this issue again, and we will reopen this bug   Thanks, Kailash
Comment by Kailash B.
31623 | October 17, 2019 06:14:25 AM GMT
Hi Kailash, Thank you for your assistance on this. Sorry, we spoke too soon. Changing the port did work. But for just the first few requests. We encountered the issue in subsequent requests. We would therefore be grateful if you could reopen this bug. I have attached the current isapi_redirect log. Please let me know if uou need more information. Regards
Comment by A. B.
31624 | October 17, 2019 07:39:16 AM GMT
Hi,   Looking at the logs it seems you are using the older version of isapi_redirect.dll Can you please re-download it from the link: [https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3A5cfec515-c332-4777-a502-49a5f5b36093 ?|https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3A5cfec515-c332-4777-a502-49a5f5b36093]   Thanks, Kailash
Comment by Kailash B.
31625 | October 17, 2019 08:49:56 AM GMT
Thanks, Kailash. We shall now give it another go.
Comment by A. B.
31626 | October 17, 2019 10:09:51 AM GMT
Hi Kailash, We have given the new file a go. Alas, that didn't help either. We also experimented with different values for the connector (in worker.properties and server.xml), and with upgrading using the wsconfig tool. Same issue every time. Nevertheless, we had earlier decided to switch on DEBUG mode for isapi_redirect logging. (Thanks to my colleague, Dik Klop). So all of these tests have been logged, down to the last detail. I have uploaded the file for your convenience to the above link [|https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3A5cfec515-c332-4777-a502-49a5f5b36093]. The file is called isapi_redirect_DEBUG_MODE.log. Please treat its contents as confidential. Again, thanks for your assistance.
Comment by A. B.
31627 | October 17, 2019 11:00:27 AM GMT
Hi,   You have given an incorrect location for the uploaded debug file. Can you recheck?   Thanks, Kailash
Comment by Kailash B.
31628 | October 17, 2019 11:30:38 AM GMT
Hi Kailash, Sorry about that. I have re-uploaded the file. The new link is : https://documentcloud.adobe.com/link/track?uri=urn%3Aaaid%3Ascds%3AUS%3A6f6fe0c2-166b-491e-b3ce-1e4c8d1d987b Regards,
Comment by A. B.
31629 | October 17, 2019 11:54:55 AM GMT
Hi Kailash, I have sent you an urgent e-mail via support@coldfusionsupport.zendesk.com Regards
Comment by A. B.
31634 | October 17, 2019 02:10:45 PM GMT
Hi,   Looking at the logs, it seems you are still using the older dll and not the patched dll I have attached the dll in the bug itself. * Please rename the file to isapi_redirect.dll from isapi_redirect.dll1 * Take a backup of the original isapi_redirect.dll file and delete it from this folder * Copy this new dll to the [CF Home]\config\wsconfig\[Magic Folder] * Restart your IIS Website * Restart the Application pool related to your IIS Website * Retry I am assuming you are using IIS 64 bit with ColdFusion 2018 here If you have a different configuration, please let me know   Thanks, Kailash
Comment by Kailash B.
31645 | October 18, 2019 06:26:33 AM GMT
A.BAKIG, Can you pls. comment on Kailash's last comment.
Comment by Piyush K.
31650 | October 19, 2019 06:55:46 AM GMT
Hi Piyush and Kailash, yes, we are using CF2018 on Windows 2016 64 bit. we took the steps as described by Kailash. Is there an email address where I can send the log file? I don't want to attach the log file to this ticket, because debug logging was on an it contains sensitive information.
Comment by Dik K.
31651 | October 19, 2019 07:47:10 AM GMT
I've attached a regular log file. If you want to have the debug logfile as well, please let me know.
Comment by Dik K.
31652 | October 19, 2019 08:03:43 AM GMT
Hi Kailash, Hi Piyush, Any development on this issue? Any patches or fixes we can test? Thanks. Regards,
Comment by A. B.
31678 | October 23, 2019 07:47:15 AM GMT