Why does HttpContext disappear in a .NET Core SignalR cluster behind Cloudflare?

I've been struggling with an issue where the HttpContext accessed through IHttpContextAccessor becomes null when using SignalR over WebSockets in our .NET Core cluster. This happens when our NextJS clients connect through Cloudflare, which is used as a load balancer without session affinity.

Here's our configuration:

We run three identical Windows servers hosting various C# APIs with .NET 8 and IIS. Each server connects to a local RabbitMQ instance via Rebus, and we use a RabbitMQ cluster for message distribution. Cloudflare balances the load, but without session stickiness. The problem manifests when the API is accessed via a SignalR hub from a NextJS app. After a few refreshes (even if the target host remains the same), the HttpContext in IHttpContextAccessor unexpectedly becomes null.

Our SignalR setup is configured as follows:

CSHARPCopied
services
  .AddSignalR(_ => {
    _.EnableDetailedErrors = true;
  })
  .AddMessagePackProtocol(_ => {
    _.SerializerOptions = MessagePackSerializerOptions.Standard
      .WithSecurity(MessagePackSecurity.UntrustedData);
  })
  .AddRebusBackplane<myhub>(); // changed for post

And the RabbitMQ connection setup looks like:

CSHARPCopied
string rabbitMqConnectionString = $"amqp://rabbit:carrot@localhost:5672/myvhost"; // changed for post

services.AddRebus(configure => configure
  .Transport(x => {
    x.UseRabbitMq(rabbitMqConnectionString, GenerateTransientQueueName("queuename"))
      .InputQueueOptions(o => {
        o.SetAutoDelete(true);
        o.SetDurable(false);
      });
  });

services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();

Endpoints are mapped as follows:

CSHARPCopied
app.UseEndpoints(endpoints => {
  endpoints.MapControllers();
  endpoints.MapHub<myhub>("/myhub"); // changed for post
});

Within our API code, we attempt to access the HttpContext like this:

CSHARPCopied
_httpContextAccessor = httpContextAccessor;
_context = context;
_securityService = securityService;

Claim[]? claims = _httpContextAccessor.HttpContext?.User.Claims.ToArray();

if (_httpContextAccessor == null) throw new Exception("Httpcontextaccessor is null");
if (_httpContextAccessor.HttpContext == null) throw new Exception("Httpcontextaccessor => Httpcontext is null"); // <-- This exception throws
if (_httpContextAccessor.HttpContext.User == null) throw new Exception("Httpcontextaccessor => Httpcontext => User is null");

if (claims == null) {
  throw new Exception("Cant access Http Context!");
}

I suspect the issue is tied to how SignalR's negotiate request behaves behind a load balancer. When the load balancer redirects requests (due to lack of sticky sessions), the initial negotiate response (containing a server-specific hub ID) is not consistent across servers and causes the HttpContext to be lost. Indeed, when we bypass the negotiate step by setting skipNegotiation to true on the client, the issue disappears:

JAVASCRIPTCopied
const connection = new HubConnectionBuilder()
  .withUrl(`${process.env.NEXT_PUBLIC_API_BASEURI}/${global.endpoints.myhub}`, {
    accessTokenFactory: () => mytoken,
    skipNegotiation: true, // <-- added
    transport: HttpTransportType.WebSockets, // <-- added
  })
  .withHubProtocol(new MessagePackHubProtocol())
  .withAutomaticReconnect()
  .build();

However, as noted in Microsoft's security guidance (https://learn.microsoft.com/en-us/azure/azure-signalr/signalr-concept-client-negotiation), turning off negotiation can introduce security concerns. I would appreciate any insights or alternative solutions to maintain session consistency without compromising security.