How Can I Correctly Format Assistant Messages After a Tool Function Call in GPT Conversations?

Below is how you can pass the previous tool call information back to the model in a structured way rather than a custom, markdown‑formatted string. The idea is to replicate the internal structure that the API uses when returning function calls rather than doing a manual json.dumps output.

What You Should Do

When you get back an assistant message containing one or more tool calls (each with a unique id and the structured function information), you need to add new messages to your conversation history that preserve that structure. In other words, instead of manually formatting the tool call as a JSON string, you should rebuild the assistant message (or add new messages with role "tool") using the same fields as the original message. For example, if the original assistant message looked like this:

PYTHONCopied
{
  "role": "assistant",
  "content": null,
  "tool_calls": [
      {
          "id": "call_123",
          "type": "function",
          "function": {
              "name": "ask_user_clarifying_question",
              "arguments": "{\"question\": \"...\", \"choices\": [\"A\", \"B\"]}"
          }
      }
  ]
}

Then when passing along the result back to the model, you shouldn’t convert that into a string via json.dumps. Instead, you include this structured object in your conversation list. For example, if you want to attach a result to that function call, you’d add a message like:

PYTHONCopied
{
  "role": "tool",
  "content": "{\"answer\": \"User selected choice A\"}",
  "tool_call_id": "call_123"  # same id as provided in the tool_calls array
}

Key Points

• Preserve Structure:
  Do not wrap the function call in markdown or as a plain JSON string for the assistant message. Instead, the assistant message should have content: null, role: "assistant" and a tool_calls field that contains a list of structured call objects.

• Tool Call Result Message:
  When reporting back the result of the function call, the message should have role "tool" and include a tool_call_id that matches the original call. This way the model can correlate the result with the earlier tool call.

• Rebuild the Object Instead of Serializing to a String:
  If you reconstruct the object using the helper types (for example, ChatCompletionMessage and ChatCompletionMessageToolCall in your code), make sure that you convert it into the exact same JSON structure that the API expects. Essentially, you need to pass the message history exactly as structured data with arrays for tool_calls rather than preformatted strings.

Example

Below is a simplified example that shows how you might rebuild and pass along the assistant’s tool calls in a subsequent message:

PYTHONCopied
# Simulated previous assistant message with tool call information:
previous_assistant_message = {
    "role": "assistant",
    "content": None,
    "tool_calls": [
        {
            "id": "call_123",
            "type": "function",
            "function": {
                "name": "ask_user_clarifying_question",
                "arguments": "{\"question\": \"What type of device?\", \"choices\": [\"Phone\", \"Laptop\"]}"
            }
        }
    ]
}

# Later on, suppose we have processed that function call and now want to send a response back.
tool_result_message = {
    "role": "tool",
    "content": "{\"answer\": \"Laptop\"}",
    "tool_call_id": "call_123"
}

# When you build your conversation payload, simply include these structured messages:
conversation = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "I am experiencing connectivity issues..."},
    previous_assistant_message,
    tool_result_message
]

# Now pass 'conversation' to your chat completions endpoint.
response = openai.ChatCompletion.create(
    model="gpt-4o",  # or another model supporting tool calls
    messages=conversation
)

Summary

To avoid confusing the model:

1. Do not use markdown or manual JSON strings to represent the function call.
2. Pass the structured object exactly as provided in the initial response.
3. Use role "tool" with a matching tool_call_id for sending back results.

Following this documented pattern ensures the model continues to call tools as expected in subsequent messages.