Why does n8n's Extract to CSV node omit final rows in a malformed CSV file?

Below is an example of a markdown answer that describes the issue:

---

Problem Summary

You are receiving a CSV file via an HTTP GET request from the provided Google Drive URL. The file itself has 106 rows and 7 columns (with only 6 columns having proper headers and the 7th column filled only in a few rows). When you pass this file to the "Extract to CSV" node, it returns only 93 items (or 94 rows including the headers).

Observations

• Malformed Data: The CSV file contains voluntary malformed data. Because one column is missing headers and only has data in some rows, the CSV parser may be misinterpreting some rows.
• No Error Message: The node does not return an explicit error, it simply outputs fewer rows than expected.
• Workflow Expectation: You expect to see all 106 rows (or 105 data rows if not counting the header row).

Potential Cause

The "Extract to CSV" node might be using a parser that expects a consistent number of columns per row. When rows have an irregular number of fields (as is the case with the malformed 7th column), some rows might be skipped or merged, leading to 93 items in the output.

Next Steps to Troubleshoot

1. CSV Parsing Options:
   Check if the node or an upstream node allows you to configure CSV parsing options (e.g., delimiter settings, handling of missing fields, or strict mode). Adjusting these settings may help the node recognize all rows, even if some are malformed.

2. Data Preprocessing:
   Consider adding a Function node before the CSV extraction to manually inspect and sanitize the CSV data. Using a custom script to correct the row structure can help ensure the CSV is well-formatted before it is parsed.

3. Alternative CSV Extraction Methods:
   Try using a different node or external library that offers more robust CSV parsing, capable of dealing with irregular rows, then bring the data back into your workflow for further processing.

Summary

Even though there is no error message, the discrepancy in row count is likely due to the node not handling malformed rows as expected. Adjusting the parsing configuration or pre-processing the CSV file should help in retrieving the full 106 rows as expected.

---