tracker issue | What iT iS dESign studios

Title:

DeserializeJSON does not maintain the correct order of elements

| View in Tracker

Status/Resolution/Reason: Closed/Withdrawn/NotABug

Reporter/Name(from Bugbase): Martin Parry / Martin Parry (Martin Parry)

Created: 09/20/2015

Components: Language, Serialization

Versions: 11.0

Failure Type:

Found In Build/Fixed In Build: CF11_Final /

Priority/Frequency: Major / All users will encounter

Locale/System: English / Windows 10 64 bit

Vote Count: 0

When deserializing a JSON object the elements withing the object are sorted alphabeticall instead of maintaing the correct order. 

I am using Alpaca form builder ( http://www.alpacajs.org/demos/form-builder/form-builder.html )  to design forms for use within our app - Alpaca exports the JSON with the correct order of the fields I have created. e.g. first name, middle names and last_name. You can check out the JSON by clicking on the code tab.

After deserializing, the order of the fields are in alphabetical rather than the correct order I have drag and dropped them into. Thus, when presenting the user with the form to fill in, the fields are not in the correct order.

The result after deserializing the above object is in the attachments >>>>

To reproduce:-

<cfsavecontent variable="theJSON">
{
    "schema": {
        "required": false,
        "type": "object",
        "properties": {
            "_en_first_name": {
                "required": false,
                "type": "string",
                "properties": {}
            },
            "_en_middle_names": {
                "required": false,
                "type": "string",
                "properties": {}
            },
            "_en_last_name": {
                "required": false,
                "type": "string",
                "properties": {}
            }
        }
    },
    "options": {
        "disabled": false,
        "collapsible": true,
        "legendStyle": "button",
        "helpers": [],
        "focus": false,
        "showMessages": true,
        "validate": true,
        "type": "object",
        "fields": {
            "_en_first_name": {
                "attributes": {},
                "renderButtons": true,
                "autocomplete": false,
                "disallowEmptySpaces": false,
                "disabled": false,
                "data": {},
                "helpers": [],
                "disallowOnlyEmptySpaces": false,
                "showMessages": true,
                "label": "First name",
                "validate": true,
                "type": "text",
                "allowOptionalEmpty": true,
                "fields": {}
            },
            "_en_middle_names": {
                "attributes": {},
                "renderButtons": true,
                "autocomplete": false,
                "disallowEmptySpaces": false,
                "disabled": false,
                "data": {},
                "helpers": [],
                "disallowOnlyEmptySpaces": false,
                "showMessages": true,
                "label": "Middle name(s)",
                "validate": true,
                "type": "text",
                "allowOptionalEmpty": true,
                "fields": {}
            },
            "_en_last_name": {
                "attributes": {},
                "renderButtons": true,
                "autocomplete": false,
                "disallowEmptySpaces": false,
                "disabled": false,
                "data": {},
                "helpers": [],
                "disallowOnlyEmptySpaces": false,
                "showMessages": true,
                "label": "Last Name",
                "validate": true,
                "type": "text",
                "allowOptionalEmpty": true,
                "fields": {}
            }
        }
    },
    "data": {}
}
</cfsavecontent>

<cfdump var="#deserializeJSON(theJSON)#" label=" "><cfabort>


Any Workarounds:

Not that I'm aware of

----------------------------- Additional Watson Details -----------------------------

Watson Bug ID:	4060037

External Customer Info:
External Company:  
External Customer Name: Martin Parry
External Customer Email:  
External Test Config: Not relevant

Attachments:

September 20, 2015 00:00:00: 1_json.png

Comments:

I forgot to mention - Even if you don't dump the variable and immediately serializeJSON and dump the output of that the field order is incorrect.

Comment by External U.

5819 | September 20, 2015 01:07:43 AM GMT

OK - After some probing around and using some code I found here http://jsonutil.riaforge.org/index.cfm?

I modified line 54 and 421 of jsonUtil.cfc to use createObject("java", "java.util.LinkedHashMap").init() instead of structnew() and after deserializing and serializing the order is preserved. 

As this could break existing functionality for other users, if you introduced this as an option it may be something like .. DeserializeJSON(JSONVar[, strictMapping, useCustomSerializer, preserveOriginalSorting])

I found the solution at http://jean.ugal.com/notes/key-order-in-coldfusion-structures

Comment by External U.

5820 | September 20, 2015 01:49:57 AM GMT

How is the order relevant? You access by name.

Comment by External U.

5821 | September 20, 2015 05:49:23 AM GMT

Bernhard, perhaps you don't understand what I was doing. In the form builder I drag and drop the fields that I want to display in a particular order first, middle then last name. I export the JSON, then when I read it into CF and deserialize, the order is changed to first, last, middle - So when I display the input fields they're the wrong way round. By using the linkedhashMap, it preserves the order when re-serializing or looping over a StructKeyList and passing back to the form tool. 

<cfset tJSON=deserializeJSON(theJSON)>
<cfoutput>#structKeyList(tjson.options.fields)#</cfoutput><cfabort>
Result:  _en_first_name,_en_last_name,_en_middle_names

But after running through my modified version of JSONUtil it looks like this

<cfinvoke component="com.jsonutil" method="deserializeFromJSON" returnvariable="tjson" jsonvar="#theJSON#">
<cfoutput>#structKeyList(tjson['options']['fields'])#</cfoutput><cfabort>

Result: _en_first_name,_en_middle_names,_en_last_name

Comment by External U.

5822 | September 21, 2015 12:29:57 AM GMT

This is not a bug. JavaScript (more precisely the ECMA standard) specifically states that object properties are not ordered. Accordingly JSON has no sense of property ordering, and CFML certainly does not have sense of ordering in struct keys (or object properties, for that matter). And the CMFL engine should not perceive any ordering in a JSON object it is deserialising, because there isn't any.

Any ordering that you perceive is coincidental to the language implementation, and this can vary from implementation to implementation, and there's certainly no guarantee that an implementation of JavaScript will coincidentally match the implementation of a completely different data structure in a different language.

Object properties and struct elements are not designed to be accessed in any sort of assured sequence: they're designed to be accessed by key name. That's you're doing so / relying on being able to do so is a design flaw in the implementation code.

If you need the data to be ordered: use a data structure that has a notion of ordering (as you yourself suggest: a LinkedHashMap for example), however it seems to me like the problem originates on the Alpaca form builder end of things, as they're not exposing the form schema in an order-aware fashion. You should probably be chasing *them* up.

ECMA spec Ref:
http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf (go down to 4.3.3)

Comment by External U.

5823 | September 21, 2015 07:49:10 AM GMT

Like the below comment points out, order of elements cannot be maintained during deserialization. 
Closing the bug as NotABug.

Comment by Immanuel N.

5824 | October 20, 2015 04:58:25 AM GMT

tracker issue : CF-4060037

DeserializeJSON does not maintain the correct order of elements