How can I compare and differentiate complex arrays in n8n using the Compare Datasets node?

Below is a concise answer:

Can I use Compare Datasets for Your Use Case?

The Compare Datasets node is intended to compare two data streams by matching specific field values you define (for example, by an “id” field). It works best when the compared items share the same structure with simple values or flattened properties.

What to Do With Complex Arrays

If your API responses are complex (nested arrays or objects), the Compare Datasets node won’t automatically work like your Python set difference (i.e. using set operators). In other words, if the IDs you want to compare aren’t already at the top level of each item, you’ll need to transform your data first.

How to Transform Your Data

1. Extract/Flatten the IDs:
   Use a Code node (or transformation nodes like the Split Out node) to iterate over your complex arrays and output items in a simple format such as:

   JAVASCRIPTCopied
   // Assuming each item has a nested "data.id" property:
   return $input.all().map(item => {
     return { json: { id: item.json.data.id } };
   });

   After this, each item will have a clear "id" field that the Compare Datasets node can use.

2. Use the Compare Datasets Node:
   Once your inputs are flattened, you can set the Input A Field and Input B Field values to “id”. This will let the node compare the two datasets and produce outputs for:

   • Items that are only in A,
   • Items that are only in B,
   • Items that match,
   • And items that are different.

What If You Must Write Code?

If you prefer to avoid the extra transformation nodes or need more custom logic (like the Python set difference), the Code node is the right tool.
For example, in JavaScript you can implement a set difference like so:

JAVASCRIPTCopied
// Assume the incoming items are already simplified and that each node returns an array of IDs
// For instance, from two previous nodes:
const array1 = $input.all()[0].json.ids; // e.g. [1,2,3]
const array2 = $input.all()[1].json.ids; // e.g. [2,3];

// Compute the difference: Items in array1 that aren’t in array2:
const diff = array1.filter(id => !array2.includes(id));

// Return the difference as an array of items:
return diff.map(id => ({ json: { id } }));

Note: n8n’s Code node supports JavaScript (or Python if configured), which serves the same purpose as your Python set operations.

Summary

• Use Compare Datasets:
  Yes—but you’ll need to ensure items are formatted correctly (e.g. by extracting the IDs into top-level properties).

• Or Use Code:
  If the data is too complex or you want full control (like applying set-difference logic), process your data in a Code node.

This way, you can either leverage the native Compare Datasets node after proper transformation or write custom code in the Code node to achieve your desired result.