Diff filter doesn't work on object arrays?

Nathan Cox

[Screen Recording 2022-09-21 at 14.07.26.mov]I have two JSON arrays, and I'm trying to use the diff function to identify unique objects in the array.

This works fine on a simple array of numbers, but on an array of objects, it says "an unknown error has occurred"

Please see the video attached. First I show you that both datasets are formatted the same. Secondly, I activate the 'diff' filter and the error occurs.

Any help would be appreciated thanks!

Ray Deck

Diff filter doesn't work on objects. This is true in a lot of languages because of how objects (and vector data types generally) are stored in memory. 

So to figure out "what's new" you'd need to roll your own.

How I would think about it: loop over the objects to build parallel arrays of JSON strings. Then diff the arrays of strings (because strings can be diffed). Take the resulting exclusionary set, loop over it to decode the json from each element and add those to arrays. Boom, you have your diffed array.  

Sean Montgomery

 we'll check out the error. It feels like this should be supported. We definitely should have a better error message though. Will let you know what we find.

Nathan Cox

 Incredible thanks.

Chris Coleman

Hey  thanks for your patience on this. We've updated to add support for diffs and intersections on arrays to work with nested objects instead of only scalars. Thanks for reporting this!

Nathan Cox

 Thanks for this Chris, it does work now but I was hoping to return the whole object where the difference is, rather than just the key value from that object that changed. Is that possible? Thanks

timofischer

I totally agree with Nathan, @Chris Coleman. A user expects to get the full object using the diff filter. I was trying to find the error for an hour until I stumbled over this old thread.