Using Async in Django for Concurrent HTTP Calls

Yes, it's safe and reliable to use asynchronous programming to make your S3 calls concurrent in this scenario. Since your operations don't involve shared state, writes, or database interactions, this is an ideal use case for async programming.

Option 1: Using Async Views

You can convert your view to an async view and use asyncio to run the calls concurrently:

PYTHONCopied
async def view(request):
    import asyncio
    
    # If s3.get_object is a synchronous function, wrap it with sync_to_async
    from asgiref.sync import sync_to_async
    
    async_get_object = sync_to_async(s3.get_object, thread_sensitive=True)
    
    # Run both calls concurrently
    response1, response2 = await asyncio.gather(
        async_get_object(...),
        async_get_object(...)
    )
    
    # Process results and return
    return JsonResponse({"result": ...})

Even though you're running Django with synchronous workers on gunicorn (WSGI), Django will create a one-off event loop for your async view. This means you can still use async features, though you won't get the full benefits of an async stack.

Option 2: Using ThreadPoolExecutor

If you prefer to keep your view synchronous, you can use ThreadPoolExecutor:

PYTHONCopied
from concurrent.futures import ThreadPoolExecutor

def view(request):
    with ThreadPoolExecutor(max_workers=2) as executor:
        future1 = executor.submit(s3.get_object, ...)
        future2 = executor.submit(s3.get_object, ...)
        
        response1 = future1.result()
        response2 = future2.result()
    
    # Process results and return
    return JsonResponse({"result": ...})

Which Approach to Choose?

Both approaches will work for your use case:

1. ThreadPoolExecutor: Simpler if you're already using synchronous code and don't need to change much.
2. Async View: More modern approach that aligns with Django's direction, but requires more changes.

Since you're only making HTTP calls and not interacting with Django's ORM or other async-unsafe parts, either approach should reliably reduce your latency from 200ms to approximately 100ms (plus a small overhead).