HTTP Request Node Crashes on Third Page Request in n8n Workflow

Based on your description, it seems the workflow “hangs” (or even crashes) when processing the 3rd page response. Since you’re not getting any error output and the same simplified code works in isolation, the issue might not be with the HTTP requests themselves but rather with how the workflow (or n8n instance) is handling memory or resource allocation as more pages are processed.

Here are a few things to consider:

1. Memory-Related Issues

n8n does not restrict how much data each node may fetch or process. If later pages return larger data or if accumulation of the results pushes the memory usage over the available limits, it could cause the instance to hang or become unresponsive. When self-hosting, you might see related issues in your logs (for example, a JavaScript heap out of memory error) even if they aren’t directly surfaced in the workflow’s output. On n8n Cloud, the instance might restart automatically to recover.

What to try:

• Increase available memory: For self-hosted instances, you can allocate more memory to the n8n process using Node’s --max-old-space-size option (or via the NODE_OPTIONS environment variable).
• Reduce memory consumption: If increasing memory is not an option, consider re-building the workflow to process data in smaller batches—for example, by using the Split In Batches node and a Wait node between batches.

2. Workflow Design Considerations

Since your code loops through pages one after the other, it’s possible that cumulative data (or even a single large response) is being held in memory until the entire process completes. If one of these pages returns significantly more data, that could explain why the workflow crashes only on the 3rd request.

What to try:

• Inspect the payload size: Test the payload of each page independently to check if there is an unusual spike in data size.
• Process and discard data promptly: If you don’t need to keep all data in memory, make sure that each page’s data is processed and discarded rather than accumulated throughout the entire loop.

3. Testing in Isolation Versus Full Workflow

Your simplified workflow might work because it does not replicate the full behavior of the original workflow (e.g., other nodes, additional data handling, repeated executions, or manual copying of execution data for the frontend). Self-hosted and cloud setups might behave differently depending on memory provisioning and how multiple executions interact.

Summary

While the sample code you provided appears simple, the underlying issue likely comes down to memory exhaustion during workflow execution. You can mitigate this by:

• Increasing the memory allocated to your n8n instance.
• Re-building the workflow to reduce memory consumption (e.g., processing pages in smaller, separate batches).
• Investigating if one of the pages returns a larger payload than expected.

Review your instance’s resource usage (including any logs even if they don’t show explicit errors) and consider modifying the workflow design accordingly. This approach should help you avoid the crashes between page 2 and 3.

Below is some guidance on what might be happening and steps to troubleshoot the issue:

Possible Causes

• Memory Consumption Issues:
  n8n doesn’t limit the data each node can process. If the response from page 3 is significantly larger (or more complex) than earlier pages, it might cause the workflow to consume more memory than available. This aligns with scenarios described in our memory-related errors documentation, where workflows requiring more memory may hang or crash without an explicit error message.

• Rate Limiting or API Delays:
  While your simplified example works, the real API could be introducing delays, throttling (429 errors), or other network issues. These issues might not always yield a clear error message but could cause the workflow to hang if a response isn’t received in time.

• Loop or Data Buildup in Workflow:
  If additional nodes or processing steps accumulate data (for example, copying responses for frontend display during manual executions), the cumulative data might overload available resources.

Troubleshooting Steps

1. Check Your Logs:
   Look in the container or server logs for memory-related warnings like “Allocation failed - JavaScript heap out of memory” or other signs pointing toward resource exhaustion. This can help confirm whether the issue is memory-related.

2. Test the API Outside n8n:
   Use a tool like cURL or Postman to request the third page directly. This can confirm whether the API is returning an unusually large payload or if there are network delays, which might cause an execution hang.

3. Monitor Resource Usage:
   During workflow execution, monitor memory usage on your instance. This can reveal if the available memory is being exhausted when fetching page 3.

4. Add Batching or Retry Options:
   If you suspect rate limiting or intermittent delays, consider using the HTTP Request node’s built-in options to add batching or retries (as documented in the HTTP Request node common issues). This may help manage request pacing.

5. Simplify Data Handling:
   If possible, reduce the amount of data processed per page (e.g., by using filtering in your get request) so that each loop iteration consumes less memory.

Next Steps

• Increase Memory:
  If resource exhaustion is the culprit, for self-hosted instances you might consider increasing the memory available to n8n (or adjusting V8 memory limits via the --max-old-space-size option).

• Workflow Restructuring:
  Consider redesigning the workflow to process data in smaller chunks. For example, you could split the loop into multiple executions or use pagination features that allow processing one batch of results before triggering a separate batch.

By following these steps, you should be able to isolate whether it’s truly a memory constraint (or another cause, such as API throttling) and implement the appropriate workarounds.