How can I clear intermediate step data in n8n workflows to reduce memory load?

Below is some guidance based on n8n’s documentation and best practices:

Removing Unnecessary Data

n8n does not offer an explicit “clear memory” function during a workflow. Instead, you control memory usage by ensuring that you aren’t passing along large, unnecessary data between nodes. In your case, once the PDF’s text has been processed and split, you can use nodes (such as an Edit Fields or Set node) to remove or override the original large data—from the execution payload—that is no longer necessary for subsequent processing.

Why This Helps

Reducing the size of the payload passed from node to node is one of the recommended strategies to lower memory consumption. Even though the underlying Node.js process manages memory automatically, keeping the payload lean helps prevent excessive memory usage, particularly in workflows processing large or binary data.

Alternative Approaches

Another approach is to split your workflow into sub-workflows. The idea is that the sub-workflow does the heavy lifting with minimal final output, thereby releasing memory once the sub-workflow execution is complete. This technique is very useful for isolating resource-intensive operations from the main workflow where only a small result is returned.

In Summary

• Yes, it’s feasible and beneficial to “clear” data by editing the payload: Use an Edit Fields or Set node to remove or empty fields that are no longer needed.
• Ensure only necessary data is passed forward: This helps reduce overall memory usage.
• Consider splitting workflows: Offload heavy processing into sub-workflows that only return minimal data.

Implementing these strategies should assist in managing memory usage, especially in workflows that process large datasets or binary data like PDFs.