How can I avoid resending the entire conversation context with each openai.ChatCompletion.create call?

The API is stateless by design. Every call to openai.ChatCompletion.create must include all of the context (instructions and previous conversation) that you want the model to consider. This means that even though the chat UI appears to "remember" previous interactions, it actually manages the conversation history on the client side and sends it with each request.

Options & Workarounds

1. Client-Side Conversation Management
   You can manage the conversation history on your end (e.g., in a list of messages) and pass only what you need with each request. For example:

   PYTHONCopied
   import openai
   
   conversation = [
       {"role": "system", "content": "Very long set of instructions..."}
   ]
   
   # Process first input
   conversation.append({"role": "user", "content": "Input 1"})
   response = openai.ChatCompletion.create(
       model="gpt-4",
       messages=conversation
   )
   conversation.append({"role": "assistant", "content": response['choices'][0]['message']['content']})
   
   # Process second input
   conversation.append({"role": "user", "content": "Input 2"})
   response = openai.ChatCompletion.create(
       model="gpt-4",
       messages=conversation
   )
   conversation.append({"role": "assistant", "content": response['choices'][0]['message']['content']})
   
   # And so on...

2. Instruction Summarization
   If the initial instructions are long, you might consider summarizing or compressing them into a shorter version that still conveys the core requirements. This way, each request uses fewer tokens.

3. Fine-Tuning or Custom Models
   Another approach could be to fine-tune a model on your specific instructions. By doing so, you can incorporate your detailed guidance into the model itself. This eliminates the need to send the long instruction prompt repeatedly. However, fine-tuning comes with its own set of considerations and costs.

Conclusion

There isn’t a built-in feature in the API that works exactly like the UI, where previous context is automatically persisted without the need for resending it. The responsibility for context management lies with you when using the API, making it necessary to include all desired context in every request.