tracker issue : CF-3476661

select a category, or use search below
(searches all categories and all time range)
Title:

deserializeJson does not properly handle high-ascii characters

| View in Tracker

Status/Resolution/Reason: Closed/Withdrawn/NotABug

Reporter/Name(from Bugbase): Adam Tuttle / Adam Tuttle (Adam Tuttle)

Created: 01/15/2013

Components: AJAX, Plumbing

Versions: 10.0

Failure Type: Data Corruption

Found In Build/Fixed In Build: Final /

Priority/Frequency: Trivial / All users will encounter

Locale/System: English / Platforms All

Vote Count: 2

Problem Description:

If a JSON string includes non-standard ASCII characters, such as ñ or é, they are corrupted during deserialization. (E.g. Peña becomes something similar to Pe?±a) This does not appear to affect serializeJson() -- serialized strings maintain their high-ascii characters.

Steps to Reproduce:

<cfdump var="#deserializeJson('{"foo":"Peña"}')#" />

Actual Result:

foo: Pe?±a

Expected Result:

foo: Peña

Any Workarounds:

None that I've found yet!

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3476661

External Customer Info:
External Company:  
External Customer Name: Adam.Tuttle
External Customer Email:  
External Test Config: My Hardware and Environment details:



Tested on OSX 10.7.5 / Apache2.2 and Windows Server 2003 / IIS 7.

Attachments:

Comments:

Compatibility issues continue to plague the JSON functions and should be addressed as a priority.
Vote by External U.
16766 | January 15, 2013 04:59:23 PM GMT
This is important. It affects some of the things I do, all of the time.
Vote by External U.
16767 | January 15, 2013 05:00:19 PM GMT
To save you the trouble, I've tested the same input with JSONUtil.cfc from Nathan Mische and it is likewise affected. Unfortunately I see _no_ workaround for this currently.
Comment by External U.
16759 | January 15, 2013 05:07:26 PM GMT
Cannot replicate on Windows 7 Home Premium 64-bit. Either on CF10.0.7 or CF9.0.2 It all seems to work fine on that setup. -- Adam
Comment by External U.
16760 | January 15, 2013 05:14:24 PM GMT
Actually, I wasn't able to replicate this on Win 7 with CF 9 or 10.
Comment by External U.
16761 | January 15, 2013 05:27:12 PM GMT
I think this is JVM version related. I'm on 1.6.0_37.
Comment by External U.
16762 | January 15, 2013 06:24:37 PM GMT
This is a platform encoding issue. There are several ways you could address this. One option is to use cfprocessingdirective. <cfprocessingdirective pageEncoding = "UTF-8" /> <cfdump var="#deserializeJson('{"foo":"Peña"}')#" />
Comment by External U.
16763 | January 15, 2013 09:47:21 PM GMT
Just noting the JVM arg is -Dfile.encoding=UTF-8. Perhaps CF should just read-in files as UTF-8 by default, instead of the OS's default encoding. After all, it reads URL and Form input as UTF-8 by default. And it encodes output as UTF-8 by default. Thanks!, -Aaron
Comment by External U.
16764 | January 22, 2013 04:24:50 AM GMT
Not reproducible. the characters show up perfectly. As noted by other folks as well in the thread, this is not a bug with DeserializeJSON. Usually high-ascii characters would come from some external source and would not be hardcoded in the code and in that case everything would work perfectly.
Comment by Rupesh K.
16765 | September 04, 2013 08:38:05 AM GMT