Does the Assistants API Token Count Include Instructions and Message History?

Below is an explanation in detail:

Token Counting in the Assistants API

When you send a new message via the assistants API, the total token count is computed based on the entire prompt you send. This typically means that the following parts are included:

1. Instructions:
   Any system-level instructions or fixed context provided to the model are part of the input tokens. These tokens help guide the assistant’s behavior and are therefore included in the token consumption.

2. Previous Messages in the Thread:
   The conversation history (i.e., the messages that have already been exchanged in the thread) is also included as part of the context. Each previous message’s tokens get counted, ensuring that the assistant has the full context of the conversation when generating a response.

3. New Message for the Thread:
   The new message you send is naturally part of the input token count.

How It Affects Pricing

Since pricing is directly proportional to the number of tokens processed (both input and output), the total tokens you pay for in a given API call will include all of the tokens from the instructions, previous messages, and the new message. For example, if your API call results in counting 100 tokens for these combined input elements and the API responds with an additional 150 tokens, you are billed for a total of 250 tokens.

Summary

• Yes, instructions are counted: They are included as part of the conversation context.
• Yes, previous messages are counted: All tokens from the conversation history are included to maintain context.
• New message tokens are counted as well: All input tokens combined form the total prompt tokens to which the pricing applies.

By including everything in the context, the assistant is able to generate coherent responses that take into account the full conversation history and any guiding instructions.

Feel free to ask if you need further clarification or technical examples!