How can I persist n8n community nodes across updates when self-hosting on Railway?

Below is an example of how to retrieve all records using the Node.js Airtable library. By default, Airtable returns records in pages (with a default page size of 100 records). Instead of getting just one page (which may be what you're experiencing), you can use the "eachPage" method to cycle through all pages and aggregate the records.

Example Code

JAVASCRIPTCopied
const Airtable = require('airtable');
const base = new Airtable({ apiKey: 'YOUR_API_KEY' }).base('YOUR_BASE_ID');

let allRecords = [];

// Adjust any parameters inside select() as needed (e.g., view, filterByFormula, etc.)
base('YourTableName').select({
  // view: 'Grid view',
})
.eachPage(
  // This function will be called for each page of records.
  function page(records, fetchNextPage) {
    allRecords.push(...records);
    // Fetch the next page of records
    fetchNextPage();
  },
  // This function will be called after all pages are fetched.
  function done(err) {
    if (err) {
      console.error('Error fetching records:', err);
      return;
    }
    // Now "allRecords" contains every record from the table.
    console.log('Retrieved all records:', allRecords);
  }
);

Explanation

• select(): This method initiates a query on your table. You can pass parameters such as the view name or filtering options.

• eachPage(): This method retrieves records one page at a time. The callback provided as the first argument processes each page. Once a page is processed, fetchNextPage() is called to load the next page.

• Aggregation: All records from different pages are combined into the allRecords array.

• Error Handling: The error callback handles any issues that occur during the process.

This approach ensures that you retrieve all data regardless of the number of records in your table.

Below is an explanation on how to persist your community nodes between updates.

Use a Persistent Volume for Community Nodes

n8n installs community nodes directly onto the file system and expects them to be available at startup. When you update your container without a persistent volume, any manually installed nodes (typically found in the ~/.n8n/nodes directory) are removed. To avoid this, you should attach a volume that maps to that directory.

Which Path to Mount

You need to mount a volume to the following directory:

~/.n8n/nodes

In most Docker setups, this path refers to the user home directory (often /root/.n8n/nodes if running as root). Make sure your persistent storage is mapped to this directory so that installed community nodes remain available between container updates.

Primary vs. Worker Instances

If you are running n8n in queue mode with separate primary and worker instances, it’s important that all instances have access to your community nodes. That means:

• Primary Instance: Mount the volume at ~/.n8n/nodes so that when the primary starts up, it can load the community nodes.
• Worker Instances: Also mount the same volume at ~/.n8n/nodes so that execution workers have access as well.

By doing this, the nodes you’ve installed will be available across your entire n8n setup.

Summary

• Mount a persistent volume at ~/.n8n/nodes in your primary Docker container.
• Do the same for worker containers in a queue mode setup so that all instances share the community nodes.

This setup will ensure that your community nodes aren’t removed upon container updates.