Why Do Node.js WebSocket Connections Fail Randomly Under Heavy Load?

Below is a list of potential causes and suggestions to help you scale your Socket.IO server beyond 10,000 connections:

---

1. NGINX Configuration Tuning

• Timeouts & Buffering:
  The default NGINX timeouts and buffer settings may not be optimal for a high number of persistent connections. Consider increasing some of these timeouts and disabling buffering for WebSockets. For example:

  NGINXCopied
  server {
      listen 80;
      server_name mydomain.com;
  
      location /socket.io/ {
          proxy_pass http://localhost:3000;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "Upgrade";
          proxy_read_timeout 3600s;
          proxy_send_timeout 3600s;
          proxy_buffering off;
      }
  }

• Connection Limits:
  Validate that NGINX and your operating system are configured to handle a large number of concurrent open file descriptors. Make sure settings such as worker_connections in NGINX and ulimit values on your server are increased as needed.

---

2. Node.js and Socket.IO Considerations

• Event Loop Blocking:
  Since Node.js runs on a single thread (per process), heavy operations or extensive logging (e.g., console.log on every connection or message) can block the event loop.

  • Tip: Remove or reduce synchronous logging in production or consider using non-blocking logging libraries.

• Engine.io Ping/Pong:
  Tuning the ping interval and timeout can help, but if set too high it might let lag accumulate. Revisit these settings and ensure they match the network conditions and client behavior.

• Scaling Out:
  Use the built-in clustering or a process manager (like PM2) to distribute the load across multiple CPU cores. Make sure your clustering strategy handles sticky sessions, as WebSockets need the same server process to maintain the connection. You mentioned trying clustering, so verify that:

  • Clients are consistently routed (sticky sessions).
  • The Redis adapter is properly managing state between clusters.

---

3. System-Level Tuning

• File Descriptors and Network:
  With 10,000+ WebSocket connections, you’ll need to ensure your operating system is configured to support such high numbers:

  • Increase ulimit for file descriptors.
  • Tune TCP stack parameters (e.g., net.core.somaxconn, net.ipv4.tcp_tw_reuse, etc.).

• Hardware and Resource Limits:
  High CPU usage can result from the node process being overloaded. Profiling your application will help – use tools like Node’s built-in profiler or external APMs to identify hotspots.

---

4. Architectural Adjustments

• Load Balancer:
  Consider using a load balancer that is built to support long-lived connections or using multiple reverse proxies. This can help distribute not just HTTP requests but also WebSocket traffic more evenly.

• Dedicated WebSocket Solutions:
  Depending on your needs, you might assess whether a dedicated WebSocket server or an event-driven framework (such as uWebSockets.js) might offer improved performance over Socket.IO in high-throughput scenarios.

• Stateless vs. State:
  Review how much state you’re keeping per connection. Offload state to external stores like Redis if needed, but also ensure that this external scaling doesn’t introduce latency.

---

5. Monitoring and Profiling

• Profiling:

  • Monitor CPU and memory usage using Node.js profiling tools.
  • Check system metrics (CPU, network, disk I/O) during peak loads.

• Error and Log Analysis:

  • Investigate “Connection reset by peer” errors to see if there’s a pattern that can point to misrouted or timed-out connections.
  • Log events carefully to avoid heavy I/O that may add to the load.

---

Conclusion

While a single Node.js process with Socket.IO can be efficient in low-load scenarios, scaling to 10,000+ connections often requires tuning at multiple levels—from NGINX to Node.js configuration and the underlying OS settings. Adjusting timeouts, reducing blocking operations (like extensive logging), increasing file descriptor limits, and ensuring proper clustering with sticky sessions will all contribute to a more robust high-load architecture.

Implement these changes gradually and monitor their effect on connection stability and CPU usage.