tracker issue : CF-3505249

select a category, or use search below
(searches all categories and all time range)
Title:

DeSerializeJSON of null values causes strange structure formatting

| View in Tracker

Status/Resolution/Reason: Closed/Withdrawn/AsDesigned

Reporter/Name(from Bugbase): Josh Souza / Josh Souza (Josh_Souza_ViaWest)

Created: 02/22/2013

Components: Core Runtime

Versions: 10.0

Failure Type:

Found In Build/Fixed In Build: Final /

Priority/Frequency: Normal / All users will encounter

Locale/System: English / Win All

Vote Count: 5

Problem Description: When you deserialize a json structure with 'null' values, such as: {"text":null} instead of a blank string or otherwise, the documentation states: The JSON null value becomes the string null.
In previous versions of Coldfusion (confirmed with 8 and 9) it becomes either a blank string or the actual text "null". However in CF10 it becomes 'undefined' in a very odd way.
You end up with a structure with all of the keys that were passed in, but they may not have values at all. So if you did: <cfset test=DeserializeJSON('{"good":"string","bad":null}>
You would be able to cfoutput: #test.good# but you would receive an error if you tried to output #test.bad#, because the value doesn't exist.
StructKeyExists reports that the value does not exist, yet StructKeyList still returns the key.

Steps to Reproduce:
Put the following code into a test page in CF9 and CF10 and observe the differences.

<cfset testString='{"description":null,"testText":"nothing"}'>
<cfset ts=DeserializeJson(testString)>
<cfoutput>
This is before I fix anything. Here is the given string: #testString#
<br>This is what it deserializes into
<cfdump var="#ts#">
<br>These are the struct key list entries: #StructKeyList(ts)#
<br>This is the result of StructKeyExists(ts,"description"): #StructKeyExists(ts,"description")#
</cfoutput>
<cfloop list="#StructKeyList(ts)#" index="key">
	<cfif not StructKeyExists(ts,key)>
		<cfset ts[key]="">
	</cfif>
</cfloop>
<cfoutput>
<br>This is after I fix things.
<br>These are the struct key list entries: #StructKeyList(ts)#
<br>This is the result of StructKeyExists(ts,"description"): #StructKeyExists(ts,"description")#
<br>And the final resulting deserialized struct:
<cfdump var="#ts#">
</cfoutput>
<cfabort>



Actual Result: CF error when attempting to dereference structure values that have keys.

Expected Result: You should be able to dereference all structure keys that are returned from StructKeyList

Any Workarounds:
By performing a loop such as the following after all dereferences, it kinda works around the problem for us (replicating CF9 behavior)

<cfloop list="#StructKeyList(ts)#" index="key">
	<cfif not StructKeyExists(ts,key)>
		<cfset ts[key]="">
	</cfif>
</cfloop>

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	3505249

External Customer Info:
External Company:  
External Customer Name: Josh_Souza_ViaWest
External Customer Email:  
External Test Config: CF10 in Windows and Linux

Attachments:

Comments:

Note that my workaround does not descend into nested structures etc.... a problem we ran into
Comment by External U.
16218 | February 22, 2013 04:50:41 PM GMT
Our entire system (700+ sites) revolves around an internal JSON base API and this bug causes almost every data call to fail. Restoring this functionality to match the documentation is imperative for us. I believe we are going to have to hold off our upgrade to CF 10 until this is corrected.
Vote by External U.
16223 | June 04, 2013 01:14:17 PM GMT
This bug is greatly affecting our ability to upgrade to CF10. We've had a codebase that worked with the expected doc usage through multiple older versions of CF. Implementing a workaround in our use cases would be quite a hindrance at our stage and scale.
Vote by External U.
16224 | June 04, 2013 01:43:43 PM GMT
We have a system built in CF 9 that heavily utilizes JSON Serialization. When testing for CF 10, most of our site stops working because of the deserialization bug. Please make this a priority! Thank you.
Vote by External U.
16225 | June 04, 2013 03:16:07 PM GMT
We have developed for null values as per the docs and this has worked in CF8 and CF9 without issue. With CF10, to have to do a StructKeyExists for every var in a structure before doing anything else with that var will require a massive amount of additional coding for us. This bug should have a much higher priority in my opinion.
Vote by External U.
16226 | June 06, 2013 06:48:20 PM GMT
To me it sounds like what you're describing is simply expected behaviour when a struct key's value contains a null. The null-handling behaviour in JSON (de)serialisation in CF8 & CF9 was *broken* : it should never have been converting a NULL to "null" (ie: a string containing the word null), that was a bug (not to mention bloody stupid). What CF is doing - correctly - in CF10 is to make the value actually NULL. And it's standard / expected CF behaviour that if you access your #test.bad# that you'd get an error. Demonstrating expected behaviour: <cfset struct = { good = "string", bad = javacast("null", "") }> <cfdump var="#{ struct = struct, keyArray = structKeyArray(struct), isNull = isNull(struct.bad) }#"> <cftry> <cfoutput>struct.bad: #struct.bad#</cfoutput> <cfcatch> <cfdump var="#cfcatch#"> </cfcatch> </cftry> This errors with "Element BAD is undefined in STRUCT." And this is expected. This should NOT be fixed, as it's NOT a bug. -- Adam
Comment by External U.
16219 | June 26, 2013 03:57:03 AM GMT
I have to agree with Adam here. Reverting back to CF's previously broken behavior is the wrong thing to do, IMO. While I sympathize with the pain that this may cause some in having to modify their applications to deal with the "new" behavior, legitimate bugs should not be left unfixed due to convenience. JavaScript has no problem handling nulls, and neither does the JSON format (remember that the "J" in JSON stands for JavaScript). Why should ColdFusion be forced to coerce nulls into "null" (a string) when there is no real need to? Since there was probably a bug filed against CF9 and/or CF8 to get null fixed in DeserializeJSON, wouldn't rolling back to the previously "broken" behavior be an affront to the developers who had been patiently waiting to get this fixed? -Carl V.
Comment by External U.
16220 | June 26, 2013 10:01:57 AM GMT
I've run into the same problem when upgrading from CF9 to CF10. I fixed the error by performing a check, like this: <cfif structKeyExists(deserializedJSON, key)> <cfset realValue = deserializedJSON[key]/> <cfelse> <!--- process as null value ---> <cfset realValue = "null"/> </cfif> Of course, I only had to do that in one spot; I can imagine it would be a nightmare if you have to do that in a lot of places.
Vote by External U.
16227 | June 26, 2013 10:59:35 AM GMT
NB: this is the bug that was fixed which is causing this confusion: https://bugbase.adobe.com/index.cfm?event=bug&id=CF-3043777. I stress CF-3043777 was the correct thing to do. This ticket - CF-3505249 - is user misconception, and not a bug. -- Adam
Comment by External U.
16221 | June 27, 2013 02:24:23 AM GMT
+1. Null values in JSON should not be converted to "null" or empty string. Withdrawing the bug
Comment by Rupesh K.
16222 | June 27, 2013 12:13:22 PM GMT