tracker issue : CF-3561029

select a category, or use search below
(searches all categories and all time range)
Title:

SerializeJSON Does Not Correctly Handle Control Characters

| View in Tracker

Status/Resolution/Reason: Closed/Won't Fix/

Reporter/Name(from Bugbase): Joseph Witthuhn / Joseph Witthuhn (Joseph Witthuhn)

Created: 05/15/2013

Components: Language, Serialization

Versions: 9.0.1

Failure Type:

Found In Build/Fixed In Build: 11.0 / CF10_Update14

Priority/Frequency: Normal / Few users will encounter

Locale/System: English / Windows 7

Vote Count: 2

Problem Description:
The SerializeJSON function does not correctly handle control characters in Strings.  It passes them through as-is, instead of escaping them using \uXXXX (where each X is a hexadecimal digit).  This means that some JSON parsers (which insist that the data follows the rules), including Jackson, cannot handle the output.  This is described at http://json.org or more formally in RFC 4627 at http://www.ietf.org/rfc/rfc4627.txt?number=4627

The form required me to pick a platform, but it seems to be a bug on all platforms (see test configuration below).

Steps to Reproduce:
Create a Struct, and load some String values in that contain control characters.  Pass this into SerializeJson and examine the output.

Actual Result:
The control characters are not encoded.

Expected Result:
The control characters should be escaped with \uXXXX as described above.  

Any Workarounds:
First use a RegEx to strip all control characters (for example, using REReplace).
Example of workaround: REReplace(SerializeJSON(inputStruct), "[\x00-\x08\x0B\x0C\x0E-\x1F]+", " ", "all")

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3561029

Deployment Phase:	Release Candidate

Keywords:
FixTested


External Customer Info:
External Company:  
External Customer Name: joseph.witthuhn
External Customer Email:  
External Test Config: Reproduced on two configurations:



Windows 7 Professional Service Pack 1

Adobe ColdFusion 9.0.1

Running as a single server on IIS 



SUSE Linux

Adobe ColdFusion 9.0.1 with Cumulative Hotfix 2

Running as a WAR file on Tomcat 6

Attachments:

Comments:

This also effects CF10 in the same way. Some control characters are escaped correctly, but not others so the result can be invalid JSON.
Vote by External U.
15371 | July 17, 2013 12:57:13 PM GMT
This is supposed fixed in 10,0,14,291717 (see http://helpx.adobe.com/coldfusion/kb/bugs-fixed-coldfusion-10-update.html) but it is not. Repro: <cfset st = {special=chr(7) & chr(9) & chr(13) & chr(27) & chr(128)}> <cfset j = serializeJson(st)> <cfoutput>#j#</cfoutput> Returns: {"SPECIAL":"\t\rx80;"} What about the rest of them?
Comment by External U.
15363 | October 14, 2014 03:25:03 PM GMT
The below testcase is outputing the expected result I assume. It also handles the hexadecimal char well. <cfscript> o = StructNew(); o.surname = "ANCU+1234"; writeOutput(serializeJson(o)); </cfscript> So can you be more sepcific about the cases where you find it not working?
Comment by Awdhesh K.
15364 | October 28, 2014 01:36:13 AM GMT
Other than the code in the comment you're replying to, you mean? The code marked "repro"? The repro code that demonstrates the very thing you're asking for more specifics about? I don't quite know how to react to that, TBH, Awdhesh.
Comment by External U.
15365 | October 28, 2014 02:39:05 AM GMT
The fix for this bug is incorrect and caused bugs #CF-3941059 and #CF-3837347.
Comment by External U.
15366 | March 18, 2015 04:56:11 AM GMT
As part of the fix that was made for the bug #CF-3941059, the fix for this bug has been reverted. Hence closing this bug as NeverFix. Thanks!
Comment by S P.
15367 | October 20, 2015 05:00:41 AM GMT
Can you please explain your "thinking" there? There is still an issue here. That your supposed "fix" for this had issues doesn't mean you still don't need to fix this issue, but just not cause *other* issues. The fix for this should NOT have caused the issues reported in the other tickets!
Comment by External U.
15368 | October 20, 2015 06:16:46 AM GMT
Unfortunately it sounds like we will have to wait for the next update to know for sure one way or the other if this was actually fixed or if the buggy "fixes" were just reverted. It's possible that they fixed it as part of the other bug fixes, but you are right Adam that if that is the case, it shouldn't be marked as NeverFix... in which case there should have been some explanation as to why they are fixing THIS bug.
Comment by External U.
15369 | October 20, 2015 10:17:14 AM GMT
FIX JSON ISSUES, FIX JSON ISSUES, FIX JSON ISSUES, FIX JSON ISSUES, FIX JSON ISSUES, FIX JSON ISSUES
Vote by External U.
15372 | October 20, 2015 10:17:43 AM GMT
Base64 encoding is being used to send a binary file( gpeg, png images) to browser wherein the “u+” is a valid two chars sequence which is not a Unicode. With this fix 'u+' was always being converted to '\u' due to which a lot of customers had reported concerns about image not being rendered correctly onto the browser from the server, because it could not identify the conversion of 'u+' to '\u', thereby leading to rendition of corrupted image. Hence the fix was reverted. Also this behavior can be tested on the Pre-release update builds that are out, for your testing purposes, but please make sure that you do not deploy it on your production server. Thanks!
Comment by S P.
15370 | October 26, 2015 11:29:33 PM GMT