tracker issue | What iT iS dESign studios

Title:

cfhtmltopdf outputs special characters as question marks (?)

| View in Tracker

Status/Resolution/Reason: Closed/Fixed/

Reporter/Name(from Bugbase): Roberto Anania / Roberto Anania (Roberto Anania)

Created: 08/24/2015

Components: Document Management, PDF Generation (CFHTML2PDF)

Versions: 11.0

Failure Type: Data Loss

Found In Build/Fixed In Build: CF11_Final /

Priority/Frequency: Major / All users will encounter

Locale/System: English / Win 2008 Server

Vote Count: 0

Listed in the version 2016.0.0.297996 Issues Fixed doc

Verification notes: verified_fixed on May 18, 2018 using build 2016.0.01.298513

Problem Description: cfhtmltopdf outputs special characters as question marks (?)

Steps to Reproduce: Insert a special character in between cfhtmltopdf tag.

Actual Result: ?

Expected Result: If I put "à", I expect à. Instead I get ?.

Any Workarounds: Use &agrave instead of à.

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	4042294

External Customer Info:
External Company:  
External Customer Name: Roberto
External Customer Email:  
External Test Config: My Hardware and Environment details:



Windows 2008.



Coldfusion 11 Update 5.

Attachments:

December 05, 2015 00:00:00: 1_4042294_20151204.pdf
April 19, 2016 00:00:00: 2_error.txt
April 19, 2016 00:00:00: 3_test.cfm.txt
April 19, 2016 00:00:00: 4_production-jvm.config.txt
April 19, 2016 00:00:00: 5_local-jvm.config.txt
April 20, 2016 00:00:00: 6_testPDF.pdf

Comments:

I had tried:

-<cfprocessingdirective pageencoding="UTF-8">
-<InvalidTag charset="utf-8">
-<InvalidTag http-equiv="Content-Type" content="text/html; charset=utf-8">

Comment by External U.

6152 | August 24, 2015 12:35:54 PM GMT

Hi Roberto,

I have tried using "à" and I am able to print pdf without any issue.

Please provide below mentioned information:

1. Settings summary 
2. Code snippet

Comment by Nimit S.

6153 | September 13, 2015 11:52:32 AM GMT

Here is a code snippet:

<cfhtmltopdf>
	<!doctype html>
	<html>
		<head>
			<InvalidTag charset="utf-8">
			<title>Test</title>
		</head>
		<body>
			à
		</body>
	</html>
</cfhtmltopdf>

This returns a pdf with a ? as the only character.

Comment by External U.

6154 | September 15, 2015 10:12:19 AM GMT

This code is working fine at our end.

Can you also please share Settings summary?

Comment by Nimit S.

6155 | September 15, 2015 10:33:55 AM GMT

What sort of settings would you require?

Comment by External U.

6156 | September 15, 2015 10:38:06 AM GMT

Please follow steps mentioned below:

Login into ColdFusion Administrator
Navigate to Server Settings > Settings Summary
Click Save As PDF on top right corner.
Share the PDF file with us.

Comment by Nimit S.

6157 | September 15, 2015 10:54:25 AM GMT

Is there a specific setting we are looking for?

I would prefer not uploading my entire server setup summary.

Comment by External U.

6158 | September 15, 2015 11:51:51 AM GMT

Yes, I need information mentioned below:

1. OS Name(32/64 bit)
2. ColdFusion server build number
3. JDK(1.7/1.8) along with update details

You can send these details to my email address: nimsharm@adobe.com

Comment by Nimit S.

6159 | September 15, 2015 12:00:12 PM GMT

Verified in build: 296152
Test case added at: //depot/qa/cf/regression/coretests/coldfusion/tags/extensibility/cfpdf/bugs/CF-4042294/

Comment by Kailash B.

6160 | October 29, 2015 02:11:40 AM GMT

Adobe,

How do we reproduce this issue? What is the repro case?

Thanks!,
-Aaron

Comment by External U.

6161 | December 04, 2015 05:51:14 AM GMT

Hi Aaron,

You can use:
<cfhtmltopdf>
	<!doctype html>
	<html>
		<head>
			<InvalidTag charset="utf-8">
			<title>Test</title>
		</head>
		<body>
			ààààààààààààààà
		</body>
	</html>
</cfhtmltopdf>
 
for repro.

Comment by Kailash B.

6162 | December 04, 2015 06:07:54 AM GMT

Hi Kailash,

Thanks but the PDF correctly displays ààààààààààààààà

How do I get it to display the question marks?

Thanks!,
-Aaron

Comment by External U.

6163 | December 04, 2015 07:31:10 AM GMT

Hi Kailash,

Attached CF-4042294_20151204.pdf is my result when running your repro code. It is correct.

So, how do we repro the issue?

Thanks!,
-Aaron

Comment by External U.

6164 | December 04, 2015 04:21:51 PM GMT

Hello Aaron,

I am still experiencing this issue on a server with this configuration:

ColdFusion Version 	11,0,07,296330
ColdFusion Edition 	        Standard  
Operating System 	        Windows Server 2008

Can it be the edition that is the issue? Are you testing with Enterprise or Developer edition? Next week I will test this on Windows 2012 with developer edition. I will let you know what happens.

Thanks,

Robert

Comment by External U.

6165 | December 04, 2015 04:55:13 PM GMT

Hi Roberto,

My test was done w/ CF11 Update 7 (build 11,0,07,296330) w/ Java version 1.8.0_66 on Windows 10. And also w/ CF11 Update 6 (build 11,0,06,295053) w/ Java 1.8.0_66 on Windows 2012.

I see the issue: cfhtmltopdf incorrectly ignores cfprocessingdirective and displays question mark:
<cfprocessingdirective pageencoding="utf-8">
<cfhtmltopdf>à</cfhtmltopdf>

cfdocument honors cfprocessingdirective and correctly displays character:
<cfprocessingdirective pageencoding="utf-8">
<cfdocument format="pdf">à</cfdocument>

Another workaround is add -Dfile.encoding=UTF-8 on CF Admin's "Java and JVM" page in the "JVM Arguments" and then restart CF. Then cfdocument and cfhtmltopdf both correctly display the character w/o needing the above cfprocessingdirective.

Thanks!,
-Aaron

Comment by External U.

6166 | December 07, 2015 01:08:57 AM GMT

Hi Kailash,

cfhtmltopdf ignores <cfprocessingdirective pageencoding="utf-8">.

Is that being fixed as part of this ticket? cfdocument honors <cfprocessingdirective pageencoding="utf-8">, so cfhtmltopdf should do the same.

Thanks!,
-Aaron

Comment by External U.

6167 | December 07, 2015 01:10:27 AM GMT

Thank you Aaron for the CF Admin fix!

Comment by External U.

6168 | December 07, 2015 08:30:34 AM GMT

Hi Roberto,

YVW!

@Adobe, question remains: will cfhtmltopdf honor <cfprocessingdirective pageencoding="utf-8"> as part of this ticket's fix?

Thanks!,
-Aaron

Comment by External U.

6169 | December 07, 2015 10:12:19 AM GMT

################################################
##ERROR

Detail  Reason: 
ERROR_PDF_CONVERSION

Message 
Error occurred while generating PDF.

StackTrace  
coldfusion.document.webkit.PDFgErrorHandler$ServiceManagerConversionException: Error occurred while generating PDF. 
    at coldfusion.document.webkit.PDFgErrorHandler.handleConversionError(PDFgErrorHandler.java:188) 
    at coldfusion.document.webkit.HttpPDFRequestHandler.requestPDFGeneration(HttpPDFRequestHandler.java:170) 
    at coldfusion.tagext.lang.HtmlToPdfTag.processPDFgRequest(HtmlToPdfTag.java:1331) 
    at coldfusion.tagext.lang.HtmlToPdfTag.handlePDFgConversionRequest(HtmlToPdfTag.java:1466) 
    at coldfusion.tagext.lang.HtmlToPdfTag.convertToPDF(HtmlToPdfTag.java:1413) 
    at coldfusion.tagext.lang.HtmlToPdfTag.doEndTag(HtmlToPdfTag.java:1551) 
    at cftest2ecfm1038184189.runPage(W:\wwwroot\com.mysite\test.cfm:5) 
    at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:246) 
    at coldfusion.tagext.lang.IncludeTag.handlePageInvoke(IncludeTag.java:736) 
    at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:572) 
    at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65) 
    at coldfusion.filter.IpFilter.invoke(IpFilter.java:45) 
    at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:466) 
    at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:42) 
    at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40) 
    at coldfusion.filter.PathFilter.invoke(PathFilter.java:142) 
    at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)

Comment by External U.

6170 | April 19, 2016 06:17:43 AM GMT

######################################################
##PROD jvm.config **fails

#
# VM configuration
#
# Where to find JVM, if {java.home}/jre exists then that JVM is used
# if not then it must be the path to the JRE itself

java.home=C:\\Program Files\\Java\\jdk1.8.0_77\\jre

#
# If no java.home is specified a VM is located by looking in these places in this
# order:
#
#  1) ../runtime/jre
#  2) registry (windows only)
#  3) JAVA_HOME env var plus jre (ie $JAVA_HOME/jre)
#  4) java.exe in path
#

# Arguments to VM
java.args=-server  -Xms9216m -Xmx9216m -XX:MaxPermSize=192m -XX:+UseParallelGC -Xbatch -Dfile.encoding=UTF-8 -Dcoldfusion.home={application.home} -Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.JavaUtilLog -Duser.language=en -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -Dcoldfusion.classPath={application.home}/lib/updates,{application.home}/lib,{application.home}/lib/axis2,{application.home}/gateway/lib/,{application.home}/wwwroot/WEB-INF/flex/jars,{application.home}/wwwroot/WEB-INF/cfform/jars -javaagent:C:/FusionReactor/instance/cfusion.cf11/fusionreactor.jar=name=cfusion.cf11,address=8088 -agentpath:C:/FusionReactor/instance/cfusion.cf11/frjvmti_x64.dll

# Comma separated list of shared library path
java.library.path={application.home}/lib,{application.home}/jintegra/bin,{application.home}/jintegra/bin/international,{application.home}/lib/oosdk/classes/win

# Comma separated list of shared library path for non-windows
java.nixlibrary.path={application.home}/lib

java.class.path={application.home}/lib/oosdk/lib,{application.home}/lib/oosdk/classes

Comment by External U.

6171 | April 19, 2016 06:17:54 AM GMT

###############################################
##LOCAL jvm.config **works
#
# VM configuration
#
# Where to find JVM, if {java.home}/jre exists then that JVM is used
# if not then it must be the path to the JRE itself

java.home=C:\\Program Files\\Java\\jdk1.8.0_77\\jre

#
# If no java.home is specified a VM is located by looking in these places in this
# order:
#
#  1) ../runtime/jre
#  2) registry (windows only)
#  3) JAVA_HOME env var plus jre (ie $JAVA_HOME/jre)
#  4) java.exe in path
#

application.home=C:/ColdFusion11/ezCF1

# Arguments to VM
java.args=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5006 -server -Xms256m -Xmx512m -XX:MaxMetaspaceSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Djava.awt.headless=true -Duser.language=en -Dcoldfusion.rootDir={application.home} -Djava.security.policy={application.home}/lib/coldfusion.policy -Djava.security.auth.policy={application.home}/lib/neo_jaas.policy -Dcoldfusion.classPath={application.home}/lib/updates,{application.home}/lib,{application.home}/lib/axis2,{application.home}/gateway/lib/,{application.home}/wwwroot/WEB-INF/cfform/jars,{application.home}/wwwroot/WEB-INF/flex/jars,{application.home}/lib/oosdk/lib,{application.home}/lib/oosdk/classes -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.JavaUtilLog 

java.class.path={application.home}/lib/oosdk/lib,{application.home}/lib/oosdk/classes

# Comma separated list of shared library path
java.library.path={application.home}/lib,{application.home}/jintegra/bin,{application.home}/jintegra/bin/international,{application.home}/lib/oosdk/classes/win

Comment by External U.

6172 | April 19, 2016 06:18:19 AM GMT

CF 11 patched up to update 7 and I'm still running into a pdf conversion issue when special characters are present `á, é, í, ó, ú, ü, ñ` etc.... 

Below is a simple snippet that fails on production but works on my local VM. Still searching for what setting aside, from Win 2012 (local) & Win 2008 (prod), that might be different in my local setup vs production.

Current Setup: 
(local => windows 2012) (production => windows 2008 R2)
CF 11 -> update 7
JDK -> jdk1.8.0_77
JVM Arg Added ->   -Dfile.encoding=UTF-8

#############################################################################
## TEST.cfm  --- **note: adding <cfprocessingdirective pageEncoding="utf-8" /> doesn't help...
#############################################################################
<cfhtmltopdf
    name="pdfFile"
    destination="_testPDF.pdf" overwrite="yes">

    <h1>PDF THAT FAILS</h1>

    <h4>PERSONs NAME: with special characters like á, é, í, ó, ú, ü, ñ </h4>
</cfhtmltopdf>

<cfdump var="#pdfFile#">
<cfabort>

Comment by External U.

6173 | April 19, 2016 06:18:39 AM GMT

Another side note... I'm using CF 11 Enterprise, but it works on CF 11 Developer. Not sure if this is the difference between my local and production server that is causing this code to fail.

Not sure if it's relevant but Roberto asked " Are you testing with Enterprise or Developer edition" and it made me wonder if this was possibly part of the issue...

Comment by External U.

6174 | April 19, 2016 06:31:10 AM GMT

Tim, if you remove the special characters does the PDF generate?

Comment by External U.

6175 | April 19, 2016 07:45:07 AM GMT

Hmm.. It doesn't. Weird. However, the cf server is creating PDFs all the time in the main app that are far more complicated than this, but it is not throwing an error except on my test file. 

I'm very confused now.

Someone questioned, wondered if this might require a certain font on the server to succeed?

Comment by External U.

6176 | April 19, 2016 12:37:55 PM GMT

What is the exact code you are running?

Is this in your file?

#############################################################################
## TEST.cfm --- **note: adding <cfprocessingdirective pageEncoding="utf-8" /> doesn't help...
#############################################################################

If so, remove one pound sign from the first and last line.

Comment by External U.

6177 | April 19, 2016 01:41:56 PM GMT

No that was just a divider comment for the post..

content below:

<cfhtmltopdf destination="testPDF.pdf" overwrite="yes">
á, é, í, ó, ú, ü, ñ
</cfhtmltopdf>

^^ the above was failing with the error : Reason: ERROR_PDF_CONVERSION because I didn't have write permissions on the directory. fixed that now I just get a PDF with question marks despite being on update 7 and adding the JVM argument to the jvm.config

see attached testPDF.pdf

Comment by External U.

6178 | April 19, 2016 02:25:39 PM GMT

Well still getting the question marks so I'm not sure what's going on there still, but I did realize that I'm trying to make the file name using the person's name. Therefore, the file name has the special characters so that is what is actually causing my ERROR_PDF_CONVERSION.

Ironically the same GENERIC error is thrown when previously the cfhtmltopdf tag didn't have write permissions on the directory. The generic nature of the error hasn't helped the ambiguity of this bug.

@Roberto - adding the JVM argument fixed the question marks for you? 
Did you have to make sure you were using a unicode font in the document?

Not sure if it's relevant but on a machine that it works on my 
Adobe Driver Version is : 5.1.3 (Build 000094)  

The Adobe Driver Version is 5.1.1 (Build 0001) - on the remote production server that is failing with special chars in the file name and producing ? marks in the document    

All things being equal this one of the only things I can see that's different about the installs

Comment by External U.

6179 | April 19, 2016 02:50:21 PM GMT

Yes, I did what Aaron suggested below and it worked. I have since applied the same fix to another server that had the same issue and it worked also.

add -Dfile.encoding=UTF-8 on CF Admin's "Java and JVM" page in the "JVM Arguments" and then restart CF

Comment by External U.

6180 | April 19, 2016 02:57:06 PM GMT

Hmm. I'm stumped. I've added that. I'll try to restart the service again tonight if I can stay up late enough.

Do you know if the Adobe Driver Version is 5.1.1 (Build 0001) would make any difference or is that just related to JDBC driver stuff? it's the only thing I can see that is different about my setups.

I have to strip the special chars out of the file name as this throws an error:

This fails:
<cfhtmltopdf destination="testPDFwithSpecialCharsInNámé.pdf" overwrite="yes">
á, é, í, ó, ú, ü, ñ
</cfhtmltopdf>

Comment by External U.

6181 | April 19, 2016 03:10:25 PM GMT

Maybe the PDF service needs to be restarted also? It runs as a separate service from the CF Application from what I understand.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-g-h/cfhtmltopdf.html

Comment by External U.

6182 | April 19, 2016 03:18:02 PM GMT

If you remove the cfhtmltopdf tags, do the characters display properly in a browser?

Comment by External U.

6183 | April 19, 2016 03:19:43 PM GMT

Nope... hmm. Does this mean it's a font issue?

Comment by External U.

6184 | April 19, 2016 03:37:55 PM GMT

It could be your editor. Which editor are you using?

Comment by External U.

6185 | April 19, 2016 03:45:05 PM GMT

Sublime Text locally and then notepad on the server. I just manually made/edited the test.cfm page using notepad on the server.

Comment by External U.

6186 | April 19, 2016 03:56:33 PM GMT

Try making a new page in Notepad and setting the encoding to UTF-8. Any difference?

Comment by External U.

6187 | April 19, 2016 04:00:10 PM GMT

Seriously... That did the trick. Sorry for all the noise on this closed ticket. Appreciate the help. Beating my head against the wall for no reason.

The ? marks are gone and the source of my error is that you can't have any special characters in the destination path. Just would be nice if the Error message wasn't so generic and gave some indication that it wasn't to do with the special characters in the document, but more a result of the special characters in the file name.

Comment by External U.

6188 | April 19, 2016 07:56:42 PM GMT

@Tim - glad it works now.

Comment by External U.

6189 | April 19, 2016 08:17:10 PM GMT

Hi Tim,

So the issue was that there were some special characters in the destination file path. Am I correct?

We will check this case and get back to you if there is a way to provide some extra information in the exception.

Comment by Chinoy G.

6190 | April 19, 2016 10:45:06 PM GMT

Hi all,

Nice work! Chinoy, here is a repro:

<cfscript>
  fileWrite(expandPath("./myfilé1.txt"), "my file");//creates file (good)
  cfdocument(format="pdf", filename=expandPath('./myfilé2.pdf'), overwrite=true) {writeOutput("my file");}//creates file (good)
  cfhtmltopdf(destination=expandPath('./myfilé3.pdf'), overwrite=true) {writeOutput("my file");}//throws coldfusion.document.webkit.PDFgErrorHandler$ServiceManagerConversionException (bad)
</cfscript>

Issue: cfhtmltopdf fails to write if filename contains special characters.

Since filewrite() and cfdocument do not have the same issue, then cfhtmltopdf should be fixed?

Thanks!,
-Aaron

Comment by External U.

6191 | April 20, 2016 12:53:52 AM GMT

Hi Aaron,

Thanks for the additional info and repo case. Can you raise a new bug for this issue

Comment by Chinoy G.

6192 | April 20, 2016 01:03:29 AM GMT

Hi Aaron,

We have raised a bug for the filename with special characters issue. Bug ID is #CF-4142993

Comment by Kailash B.

6193 | April 21, 2016 03:24:52 AM GMT

Hi Chinoy and Kailash,

You're welcome and thanks very much!  I see CF-4142993 is already marked Fixed. Awesome.

Thanks!,
-Aaron

Comment by External U.

6194 | May 09, 2016 10:51:35 PM GMT

Hi Adobe,

I've verified this is fixed in CF2016 Update 1 (build 2016.0.01.298513). With -Dfile.encoding=UTF-8 JVM arg removed, the following creates PDF successfully:

<cfprocessingdirective pageencoding="utf-8">
<cfhtmltopdf>à</cfhtmltopdf>

Thanks!,
-Aaron

Comment by Aaron N.

27852 | May 18, 2018 12:37:33 AM GMT

tracker issue : CF-4042294

cfhtmltopdf outputs special characters as question marks (?)