How can I remove duplicate records in an n8n Set node using the filter() function?

Below is one likely explanation for the issue along with a recommended solution.

Possible Cause: No Match Found

If the regular expression is unable to find a match in $json["message"] then the match() function returns null. In your code, you immediately grab index [1] from the result:

JAVASCRIPTCopied
$json["message"].match(/host=([^ ]+)/)[1]

If match() returns null then trying to access index [1] will cause the entire expression to return null. This behavior can often be masked in contexts where the same code runs as expected because the input data always includes a match. In the Set node where you observed the issue, the input data might not meet that expectation.

Recommended Fix: Validate the Match Result

To ensure that your code behaves as expected, add a check for a valid match before processing it further. For example, you can modify your expression like this:

JAVASCRIPTCopied
{{ 
  (() => {
    const message = $json["message"];
    const m = message.match(/host=([^ ]+)/);
    // If no match, return an empty array (or any fallback value)
    if (!m) return [];
    return m[1]
      .replace(/[<>]/g, '')
      .split('|')
      .filter((value, index, self) => self.indexOf(value) === index);
  })() 
}}

Explanation

1. Immediate Function: Wrapping your code in an immediately invoked function expression (IIFE) allows you to write multiple lines of logic inside the expression field.
2. Match Check: The variable m holds the result of the regex match. If no match is found (m is null), an empty array is returned.
3. Processing: If a valid match is present, the code continues with your existing operations: replacing characters, splitting the string, and filtering duplicates.

Using this approach ensures that the Set node doesn’t receive a null value when the regex doesn’t match at all, which should resolve the issue you’re encountering.