tracker issue : CF-3337394

select a category, or use search below
(searches all categories and all time range)
Title:

Function SerializeJSON() converts employee's last name ("No") to boolean false in JSON output

| View in Tracker

Status/Resolution/Reason: Closed/Fixed/

Reporter/Name(from Bugbase): James Partin / James Partin (James Partin)

Created: 09/26/2012

Components: AJAX, Plumbing

Versions: 11.0,10.0

Failure Type: Data Corruption

Found In Build/Fixed In Build: Final / CF11 Update4,CF10 Update16

Priority/Frequency: Critical / All users will encounter

Locale/System: English / Win 2008 Server R2 64 bit

Vote Count: 13

Listed in the version 11.0.04.293328 Issues Fixed doc 
Problem Description:
When using SerializeJSON() to convert objects to JSON, one employee has their last name of 'No' converted into a boolean false value, which breaks a lot of things.
I have tried javacasting as a string, placing in a cfc as a property with string type, always gets converted back to boolean false. There needs to be an optional flag for SerializeJSON() to ignore converting Yes/No strings to boolean values. I just finished getting the university to purchase ColdFusion 10 and this is killing me.

Steps to Reproduce:
Create a struct with one of the keys having the string value 'No'. run the struct through SerializeJSON()

Actual Result:

value is boolean false.

Expected Result:

value should be string "No"

Any Workarounds:

Currently appending "__" to every instance where I must use the employee's last name prior to serialization, then performing a Replace() to remove any occurrences of "__" in the result serialized JSON string

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3337394

External Customer Info:
External Company:  
External Customer Name: Jim-sama
External Customer Email:  
External Test Config: My Hardware and Environment details:

ColdFusion 10, Window 2008 x64

Attachments:

Comments:

@Adobe, perhaps serializeJSON() should respect query column type? writeOutput(serializeJSON(queryNew("lastName", "varchar", [["No"]]))); Actual Result: {"COLUMNS":["LASTNAME"],"DATA":[[false]]} Expected Result: {"COLUMNS":["LASTNAME"],"DATA":[["No"]]}
Comment by External U.
17888 | September 26, 2012 11:49:25 AM GMT
@Jim-sama, I understand you are working with structs. I've just created another ticket to resolve this when dealing with queries. #CF-3337487 Not sure if that'd help you or not. Just mentioning it as FYI. Thanks!, -Aaron
Comment by External U.
17889 | September 26, 2012 12:13:26 PM GMT
I also have this problem with queries, (thats where the data originally comes from in my case) but it comes down to the same serialization engine.
Comment by External U.
17890 | September 28, 2012 01:33:28 PM GMT
We run travel sites that rely heavily on JSON data and ISO country codes. The ISO country code for Norway is NO and we've had to code a hack around this issue in a few places. The code below illustrates how the bug affected us: <!--- Bug: SerializeJSON turn "NO" into false ---> <cfset testStruct = {"test" = "NO"}> <cfdump var="#testStruct#"> <!--- the test key is "NO" ---> <cfset json = SerializeJSON(testStruct)> <cfdump var="#json#"> <!--- Test is now false --->
Vote by External U.
17933 | January 11, 2013 10:35:12 AM GMT
We run travel sites that rely heavily on JSON data and ISO country codes. The ISO country code for Norway is NO and we've had to code a hack around this issue in a few places. The code below illustrates how the bug affected us: <!--- Bug: SerializeJSON turn "NO" into false ---> <cfset testStruct = {"test" = "NO"}> <cfdump var="#testStruct#"> <!--- the test key is "NO" ---> <cfset json = SerializeJSON(testStruct)> <cfdump var="#json#"> <!--- Test is now false --->
Comment by External U.
17891 | January 11, 2013 10:35:28 AM GMT
This is hilarious. However as we also deal with the Norwegian language and ISO codes and stuff, this has potential to be an issue for us too. -- Adam
Vote by External U.
17934 | June 19, 2013 08:28:47 AM GMT
SerializeJSON would honour the datatype from the query or in the CFC.
Comment by Rupesh K.
17892 | September 04, 2013 07:37:16 AM GMT
Can you pls provide a CFC-based example of this, Rupesh? I tried this, and it doesn't work: // C.cfc component acccessors=true { property string title; property string surname; } // test.cfm o = new C(); o.title = "Doctor"; o.surname = "No"; writeOutput(serializeJson(o.surname)); output: false Tested A-OK on queries though. Good work. -- Adam
Comment by External U.
17893 | October 14, 2014 03:36:20 PM GMT
Adam, you are supposed to serialized the CFC instance itself not its individual properties. Property's datatype is being inferred by the underlying CFC meta-data. Same will not be available if you pass an individual property value in serialize function. Try the following // test.cfm o = new C(); o.title = "Doctor"; o.surname = "No"; writeOutput(serializeJson(o)); Note: Currently you need to provide the CFC name in uppercase, which we are taking care off // C.cfc component acccessors=true { property string Title property string Surname }
Comment by Awdhesh K.
17894 | October 28, 2014 12:13:18 AM GMT
Gotcha. I think I tried that initially, but had some issues. Will try again and report back if necessary. Cheers for the follow-up.
Comment by External U.
17895 | October 28, 2014 05:02:53 AM GMT
Hi all, Just noting the CFC property issue is fixed in CF11 Update 3 per #CF-3845642. Repro: CF11 Final (11,0,0,289822): writeDump(serializeJson(o));//returns string: {"SURNAME":false,"TITLE":"Doctor"} writeDump(deserializeJSON(serializeJson(o)));//returns struct: {SURNAME:"NO",TITLE:"Doctor"} writeDump(deserializeJSON(serializeJson(o)).surname);//returns: NO CF11 Update 3 (11,0,03,292245(PreRelease)): writeDump(serializeJson(o));//returns string: {"SURNAME":"No","TITLE":"Doctor"} writeDump(deserializeJSON(serializeJson(o)));//returns struct: {SURNAME:"No",TITLE:"Doctor"} writeDump(deserializeJSON(serializeJson(o)).surname);//returns: No Thanks!, -Aaron
Comment by External U.
17896 | November 23, 2014 08:51:12 PM GMT
This has regressed in CF 11 update 3. Repro: // Villian.cfc component accessors=true { property string firstName; property string lastName; property string title; } <cfscript> // villian.cfm villian = new Villian(); villian.setFirstName("Julius"); villian.setLastName("No"); villian.setTitle("Doctor"); json = serializeJSON(villian); writeOutput(json); </cfscript> CF10 update 15: {"lastName":"No","firstName":"Julius","title":"Doctor"} CF11 update 2: {"lastName":"No","firstName":"Julius","title":"Doctor"} CF11 update 3: {"lastName":false,"firstName":"Julius","title":"Doctor"} I'm puzzled that you were seeing it all OK, Aaron? Can you pls try my repro here? Awdhesh... thoughts?
Comment by External U.
17897 | December 22, 2014 05:22:19 PM GMT
This issue is fixed and the fix will be available as part of the upcoming update of ColdFusion 11.
Comment by S P.
17898 | January 23, 2015 03:11:25 AM GMT
Hi Adam, I just now saw your comment, sorry! Ugh, I had a typo. I had acccessors=true instead of accessors=true. I now see the same result you see here in CF11 Update 3 (last name is false). I also now see the correct result in CF11 Update 4. Very sorry about that false confirmation!! -Aaron
Comment by External U.
17899 | January 27, 2015 05:54:10 AM GMT
Regarding "Ugh, I had a typo." ..actually, I was using the repro discussed earlier in this ticket. Thanks, -Aaron
Comment by External U.
17900 | January 28, 2015 04:53:17 AM GMT
This is still failing on a struct value, it is working if you have it in a cfc as a property. See: https://github.com/foundeo/cfmltests/blob/master/tests/functions/SerializeJSON.cfc for a test case.
Comment by External U.
17901 | September 02, 2015 02:53:06 PM GMT
Not fixed in CF11u6 or or CF10u17 when the "No" or "Yes" value is in a struct, it is only fixed if the value is in a CFC property.
Vote by External U.
17935 | September 02, 2015 03:12:05 PM GMT
This needs to be fixed, we deal with many Japanese users and this is common over there....
Vote by External U.
17936 | September 02, 2015 03:23:50 PM GMT
As mentioned by Pete Freitag the bug still exists in CF10U16 is serializing a struct outside a CFC
Vote by External U.
17937 | September 02, 2015 04:36:02 PM GMT
The above changes is limited and valid for CFC and queries only where type info could be derived. In case of struct, data type information is not present, hence would not work for struct.
Comment by Awdhesh K.
17902 | September 02, 2015 11:11:35 PM GMT
@Awdhesh - since it is impossible to know if the string should be treated as a boolean or a string, wouldn't it make sense to treat it as a string by default, and if it should be a boolean let the developer javaCast("boolean", value) which will always be treated properly as a boolean with no ambiguity? This is a problem because there is no way to treat "No" or "Yes" as a string (there is a hack that seams to work by prepending chr(2) to the string, but that is not something I would want to rely on) because javaCast("string", "NO") still shows up as a boolean false in the JSON. I do understand the flip side is that when a developer has some code like this: { isOver65 = age GT 65 } that isOver65="YES" and the json representation would be {"ISOVER65":true}
Comment by External U.
17903 | September 03, 2015 01:41:18 PM GMT
You speak good sense Peter: it's inconvenient but understood that CF can't infer data types in situations like this, so - on that basis - *don't try*. If you know you can't do something... don't then go ahead and try do it anyhow.
Comment by External U.
17904 | September 03, 2015 02:11:04 PM GMT
All I want is a flag or setting to force SerializeJSON to treat "No" as a string. The issue is not fixed, and should not be closed. I'm not asking for interpretation of Structs to be fixed, just a flag to have a more sane SerializeJSON for intelligent people that use true/false or 0/1 to indicate true/false. I can't believe it's 2015 and this is still a problem.
Comment by External U.
17905 | September 03, 2015 03:45:49 PM GMT
Hi Awdhesh, I agree w/ Peter, Adam and James. SerializeJSON() should stop converting "yes" to true and "no" to false, or provide a flag/setting to prevent that conversion, since "In case of struct, data type information is not present". Thanks!, -Aaron
Comment by External U.
17906 | September 03, 2015 09:14:48 PM GMT
I just want to add that the JSON features are really important for today's web developers. Every app I have written in the last several years has had to work with JSON, more and more. This is a key feature to get right. I will also add that serializing a struct is very common too, I have never serialized a CF query directly, I serialize arrays of structs because that is how I want to work with the data in JavaScript.
Comment by External U.
17907 | September 04, 2015 09:51:43 AM GMT
Question - why is this bug still Closed/Fixed? After a few days of folks saying it isn't fixed, it should be reopened by now? I'm with Pete on this - having JSON broken liken this is a huge issue imo. It essentially makes the *entire* feature, the *entire* use of CF for client-side apps, unsafe. I know it is a small thing, but really, unless you know every single row of data in your database, then you can't use Ajax w/ CF safely.
Comment by External U.
17908 | September 04, 2015 01:57:56 PM GMT
As an FYI, a structure will also have an issue: z = {"productkey":"89900909130939081290830983019819023"}; when output, the string is converted to a number
Comment by External U.
17909 | September 04, 2015 03:31:59 PM GMT
Please explain in 25 characters or more how this bug impacts productivity and why you are adding a vote.
Vote by External U.
17938 | September 14, 2015 01:27:02 PM GMT
one of the worst bugs in CF
Vote by External U.
17939 | January 20, 2016 06:32:27 PM GMT
Awdhesh... multiple people asked you a question 4.5 months ago. The courteous and professional thing to do in these situations is to *acknowledge that* by replying to them. Can you please up your game?
Comment by External U.
17910 | January 23, 2016 09:11:54 AM GMT
How about this: Adobe, if you have decided to *not* do anything about this, say so. I - and many others - will disagree with you. But at least *say* something. Say we're wrong. Say we don't know what we're talking about. But for the love of God, just freaking say something. Anything.
Comment by External U.
17911 | January 23, 2016 09:24:50 AM GMT
still broken in 11,0,07,296330
Vote by External U.
17940 | February 04, 2016 10:18:14 AM GMT
Sorry for the delayed response. We are planning to re-look into the struct serialization. Will share the initial syntax and design with you people shortly.
Comment by Awdhesh K.
17912 | March 30, 2016 03:06:30 AM GMT
Floated the syntax to wherein data type of a the struct key's value can be provided. Please check the prelease post and share your feedback. https://prerelease.adobe.com/project/forum/thread.html?cap={05d198f3-b43f-49ee-950a-8643486df6d4}&forid={e7caebde-a8f5-4f52-b22d-e94600b746e8}&topid={8903d794-0ff0-49ad-a377-db8d069ec8cd} Syntax proposed: <cfset mystruct = StructNew() > <cfset mystruct.setMetadata("lastname": "String", "age": "number") > In the above example, the "lastname" property is marked as "String", so ColdFusion will always treat it as String data type rather casting it as Boolean for some cases.
Comment by Awdhesh K.
17913 | April 19, 2016 11:51:42 PM GMT
How would that work for an array of structs?
Comment by External U.
17914 | April 20, 2016 06:49:56 AM GMT
Do you think the type name for age should be "numeric" not "number" to match CFML type names? Also the function is called setMetadata, but it only allows one property per key, so if you were going to call it setMetadata I would expect it have a struct of meta data per key, eg: myStruct.setMetadata( { { "lastname": {type:"String" } } ); You can then add other keys, such as a key that respects the case... eg: myStruct.setMetadata( { { "lastname": {type:"String", name:"lastName" } } ); If you are not going to allow expansion of other metadata, why not just call the member function what it is setDataType, eg myStruct.setDataType( { "lastname":"String" } );
Comment by External U.
17915 | April 20, 2016 08:46:51 AM GMT
Hi Awdhesh, I agree w/ Pete: 1) The type names should match existing CFML type names (as described by Pete's code). 2) It'd be useful if we can also set case-respecting key names (as described by Pete's code). Hi Pete, I also think a getter is needed so that custom serializers can get this metadata, as well as allow us to inquire a struct's ordering. mystruct.getMetadata() suggestion: -------------------------------------------- returns: {ordered="insertion|unordered", keys={lastname={type="string", name="lastName"}}} If the getter is named myStruct.getMetadata(), then I'm thinking the correponding setter should be named myStruct.setMetadata(). mystruct.setMetadata() suggestion: -------------------------------------------- syntax: mystruct.setMetadata(required struct keys) example: mystruct.setMetadata(keys={lastname={type="string", name="lastName"}}) Thanks!, -Aaron
Comment by External U.
17916 | April 20, 2016 07:48:25 PM GMT
@Ray Camdon: It will work the way an array of CFC gets serialized today. While iterating over the an array, we will get a struct as an element and then while serializing this struct, we can retrieve its metadata to know the right data types. @Pete/Aaron 1. "number" was just an oversight and it will be "numeric" a CFML data type. 2. I see your point where a struct can be passed as comprehensive metadata. But the original syntax also has advantage of being simple to write and will the job most of the times. What we can do is we can support both the syntax and its up to user to choose what information they want to set as metadata. Example: mystruct.setMetadata( "lastname": "String"); --------------------------------------------------------------------------------------- OR mystruct.setMetadata(keys={lastname={type="string", name="lastName"}}) The value of a key's can be a string or a struct. If its a string, then it will be considered as data type and if its a struct then we will look for "type" key to retrieve its data type. @getMetadata: We will discuss and will add this functionality accordingly.
Comment by Awdhesh K.
17917 | April 20, 2016 11:39:54 PM GMT
Hi Awdhesh, Perfect! Thanks very much for considering! -Aaron
Comment by External U.
17918 | April 22, 2016 12:02:56 AM GMT
I am definitely looking forward to this solution!
Comment by External U.
17919 | April 22, 2016 08:52:49 AM GMT
Thanks Awdhesh. BTW your example: mystruct.setMetadata( "lastname": "String"); should probably be mystruct.setMetadata( { "lastname": "String" } ); right? (Missing {} ) or is there some way you can pass implicit structs without the brackets? I don't think it is correct to say the value of keys is a string or a struct, it is always a struct but it is either a struct with string values or a struct with struct values ... correct?
Comment by External U.
17920 | April 22, 2016 10:07:21 AM GMT
Hi Pete, I interpreted this: "The value of a key's can be a string or a struct." as meaning this: "The value of a key, within the 'keys' struct, can be a string or a struct." Hi Awdhesh, I hope I understood the above correctly. Also, Pete makes a good point regarding the {}. We will wrap the struct w/ {} right? Thanks!, -Aaron
Comment by External U.
17921 | April 23, 2016 12:53:16 AM GMT
Aaron you got it right. Peter thanks for pointing the missing {}, around the keys.
Comment by Awdhesh K.
17922 | May 02, 2016 11:59:33 PM GMT
test note
Comment by CFwatson U.
17923 | June 07, 2016 04:19:49 AM GMT
The fix for this bug is available as part of the early-access build for ColdFusion 2016 Update 2.
Comment by CFwatson U.
17924 | June 07, 2016 04:26:46 AM GMT
Is the fix apply to the return of the restul? I have the issue from restful, which Yes become True
Vote by External U.
17941 | June 12, 2016 08:13:33 PM GMT
This bug is not fixed at all in Updater 2
Comment by External U.
17925 | June 17, 2016 03:02:53 AM GMT
Confirmed working in CF2016 u2: <cfscript> myStruct = {}; myStruct.FName = "Doctor"; myStruct["LName"] = "No"; WriteDump(myStruct); myJSON = SerializeJSON(myStruct); WriteDump(myJSON); // Evals to false myStruct.setMetadata( { "LName": "String" } ); //fix myJSON = SerializeJSON(myStruct); WriteDump(myJSON); //evals as String "No" </cfscript> Output: struct FNAME Doctor LName No {"LName":false,"FNAME":"Doctor"} {"LName":"No","FNAME":"Doctor"} Thank you. Is this also applied to CF10 or 11?
Comment by External U.
17926 | June 17, 2016 08:33:51 AM GMT
It is not, see this: <cfset data = {string_no: 'NO', string_no_case: 'No', string_yes: 'YES', string_yes_case: 'Yes', boolean_false: FALSE, boolean_true: TRUE} /> <cfdump var="#data#" /> SerializeJSON: <cfdump var="#SerializeJSON(data)#" />
Comment by External U.
17927 | June 17, 2016 08:49:59 AM GMT
"Yes" is also still wrong... not only "No"
Comment by External U.
17928 | June 17, 2016 08:51:24 AM GMT
@FirstName LastName: You did see that you need to specify metadata for the struct, right? To be clear, the fix isn't "It magically works", but "you have to do X to make it work right".
Comment by External U.
17929 | June 17, 2016 08:53:29 AM GMT
Firstname Lastname - did you try adding the required extra code such as: data.setMetadata( { "string_no": "String" } ); before your SerializeJSON() call?
Comment by External U.
17930 | June 17, 2016 08:57:30 AM GMT
Sh**. Got it - that works. Thanks for noting this.
Comment by External U.
17931 | June 17, 2016 09:22:50 AM GMT
I don't understand why this is closed when this is still very much broken. This is forcing us to write horrible hacky workarounds and it should be fixed.
Vote by Tim S.
17942 | October 17, 2017 10:56:15 AM GMT
I hope you are using setMetaData to set the data type for the key?
Comment by Vamseekrishna N.
17932 | October 23, 2017 09:40:20 AM GMT
Has there been any further update on this functionality in a later update? Do we still need to set the metadata purposely so that it doesn't attempt to convert simple text of yes or no to a boolean answer? Is there a planned update for this in CF2018? Thanks!
Comment by Adam M.
29359 | July 16, 2018 12:09:48 PM GMT