What causes the performance difference between my two multithreaded implementations using requests.post?

The unexpected timing differences turned out not to be a threading issue at all but rather a difference in how arguments are passed to requests.post. In your two implementations:

1. parallel_group2

   You call a wrapper function:

   PYTHONCopied
   def process_payload(payload, url, headers):
       response = requests.post(url, headers=headers, json=payload)
       return response

   Here the keyword arguments are explicitly set (i.e. “headers” is passed as the headers of the HTTP request and “payload” is passed via the json keyword). This is the proper way to use the API and it sends the intended headers.

2. parallel_group

   You pass requests.post directly into executor.map:

   PYTHONCopied
   def parallel_group(payloads, url, headers):
       with ThreadPoolExecutor() as executor:
           results = executor.map(requests.post, [url]*len(payloads), [headers]*len(payloads), payloads)
       return list(results)

   Because executor.map passes the arguments positionally one by one, the call becomes equivalent to:

   PYTHONCopied
   requests.post(url, headers, payload)

   In the requests library the signature of post is roughly:

   PYTHONCopied
   def post(url, data=None, json=None, **kwargs):

   So what happens is:

   • The first argument is correctly assigned to url.
   • The second argument (your headers) is assigned to data (which normally represents form data), not to the headers.
   • The third argument (the payload) lands in the json parameter.

Because of this, the actual HTTP request issued in parallel_group does not include your headers at all. With no authorization header or content-type forcing JSON encoding, the OpenAI API (or any server in a similar situation) may behave very differently. In your tests it seems that the “misconfigured” requests respond significantly faster (or at least with a different timing profile) than the calls that include the proper headers and JSON encoding.

Summary

• parallel_group2 sends a proper request (with headers and payload as JSON) via process_payload.
• parallel_group inadvertently sends headers as form data (and omits the intended header information) because of positional argument binding.

This difference in how the requests are constructed leads to different network behavior and response times. It isn’t a threading bottleneck but how the function arguments are interpreted by requests.post.

Recommendation

If you meant to use the proper keyword arguments, you can use a lambda or functools.partial to “fix” the argument names when passing to executor.map. For example:

PYTHONCopied
from functools import partial

def parallel_group_fixed(payloads, url, headers):
    post_with_kwargs = partial(requests.post, url, headers=headers)
    with ThreadPoolExecutor() as executor:
        results = executor.map(post_with_kwargs, payloads, )
    return list(results)

Now, each call is equivalent to:

PYTHONCopied
requests.post(url, headers=headers, json=payload)

which should yield the behavior of parallel_group2 (and, accordingly, the corresponding timings).

This clarifies that the key difference stems from how arguments are passed to requests.post and not from multithreading per se.