Why are webhook requests mixing up query parameters and XML payloads in a multi-worker setup?

Below is a list of things you might check and adjust in your setup to avoid mix-ups between webhook payload data during high-concurrency queue mode operations:

Verify Consistent Configuration Across Instances

• Shared Encryption Key:
  All main, worker, and webhook processor nodes need to share the same encryption key. If this isn’t set consistently—either via the environment variable or in the configuration file—there’s a chance sensitive data (including payload details) might be decrypted or processed wrongly. Ensure you’re setting it as follows on every node:

  BASHCopied
  export N8N_ENCRYPTION_KEY=<main_instance_encryption_key>

• Execution Mode and Queue Mode Settings:
  Confirm that all nodes (main and workers) are running with the environment variable EXECUTIONS_MODE=queue or have the corresponding configuration in the file. This avoids unexpected behavior when passing execution IDs and their associated data around in a distributed setup.

Check the Load Balancer (Nginx) and Webhook Routing

• Sticky Sessions & Header Forwarding:
  When using multiple webhook processes behind a reverse proxy like Nginx, be sure that your load balancer is correctly configured to preserve session information and ensure that the appropriate instance consistently handles related requests. Misrouted requests (for example, if some requests end up being routed to a worker that isn’t expecting that particular connection) might result in data from one agent being processed along with another’s query data.

• WebSocket and TCP Configurations:
  Although this issue is with HTTP payloads, if your setup also involves websockets (for worker notifications) confirm that your reverse proxy is configured to support them.

Review the Queue and Worker Setup

• Redis as a Message Broker:
  Since queue mode uses Redis to pass execution IDs from the main instance to the workers, verify that Redis is properly configured and accessible by all processes. Under high concurrency, even a slight misconfiguration (for example, timeout or incorrect Redis connection settings) might lead to race conditions when workers fetch execution details from the database.

• Worker Concurrency:
  With concurrency set to 20, multiple workflows may be picked up simultaneously. Double-check if the workers are processing the full HTTP payload correctly and not accidentally mixing parameters from concurrent requests. You might consider testing with a lower concurrency value temporarily to see if the issue persists.

Additional Considerations

• Payload Integrity:
  In situations where both query parameters and XML payload data include an agent ID, ensure that your webhook (and any middleware on Nginx) is not manipulating, caching, or inadvertently merging parts of the payload between requests. Logging incoming requests at the Nginx level might help pinpoint if the issue originates there or during n8n’s processing.

• Webhook Process Isolation:
  If you find that the webhook node handling (which can be processed by both the main instance and dedicated webhook processor nodes) is a factor, consider disabling webhook processing in the main process while using dedicated webhook processors. This can be done by setting:

  BASHCopied
  export N8N_DISABLE_PRODUCTION_MAIN_PROCESS=true

By carefully reviewing each of these points, you should be able to narrow down whether the data mix-ups come from a configuration issue, load balancing misrouting, or unintended behavior in the distributed processing of queued webhook executions.
If the problem continues after these checks, you may want to further log the exact payloads received at each stage (from Nginx, Redis, and within n8n workers) to pinpoint where the mix-up occurs.