Why isn’t the Assistant API following its configured instructions?

Understanding the Behavior

The key difference here is that the beta Assistant API you’re using handles the provided instructions differently from how system messages work in the Chat Completions API. Here are the main points:

• Instructions vs. System Messages:
  In the Chat Completions API, a system message is explicitly used to set the behavior for the conversation. It’s a clear directive that the model follows throughout the interaction. With the Assistant API, the instructions provided (e.g., "When I say 'red', write a short poem.") are used as part of the model’s internal configuration, but they aren’t mapped directly as system messages in the conversation.

• Interpretative Nature:
  The Assistant API may incorporate the instructions in a way that leaves room for interpretation. In your case, the model responded based on the word “red” without strictly executing the instruction, suggesting that the context or intended behavior wasn’t as tightly enforced as a system message would be.

• Recommendation:
  If you need behavior that is guaranteed to follow specific instructions in a conversational context, the Chat Completions API might be a better fit. It lets you define a clear system role that guides the entire interaction.

Example for Chat Completions

Using the Chat Completions API, your instruction would be provided as a system message like this:

JAVASCRIPTCopied
const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [
    {
      role: "system",
      content: "When I say 'red', write a short poem."
    },
    {
      role: "user",
      content: "red"
    }
  ]
});

This approach should ensure that the model directly follows the behavior outlined in the system message.

Summary

The Assistant API’s instructions parameter is not equivalent to a system message in Chat Completions. Instead, it serves as an internal signal that might not be strictly enforced in every context. For tighter control over the model’s behavior, especially in conversational setups, the Chat Completions API is recommended.