How can I batch process classification tasks using a single pre-prompt for multiple strings?

Below is a summary of the approach we’d recommend based on best practices:

Use Batch Requests with the Completions Endpoint

Instead of using the Chat API where you repeat the entire prompt for every string or try to jam many examples into one conversation, you can use batch requests with the completions endpoint. This endpoint lets you supply a list of prompts (each built as “pre‑prompt + specific string”) in one RPC call so that the completion for each string is returned in a list. For example:

PYTHONCopied
import openai

# Define your pre-prompt and examples (as a string)
pre_prompt = "Label the following string based on this list of labels and their definitions: ..."
example = """
Example: "A quick brown fox" -> Label: ShortStory
"""

# Your list of strings to classify
strings_to_classify = [
    "Some text to classify 1.",
    "Some text to classify 2.",
    # ...
]

# Create a prompt for each item by combining the same pre prompt, examples and the specific string.
prompts = [f"{pre_prompt}\n{example}\nString: {s}\nLabel:" for s in strings_to_classify]

response = openai.Completion.create(
    model="curie",  # or whichever model fits your task best
    prompt=prompts,
    max_tokens=10,  # adjust to ensure you get complete labels
    temperature=0,
)

# Process each completion
labels = [choice["text"].strip() for choice in response["choices"]]
print(labels)

Why This Works Best

• Single Overhead for Each Prompt: Each string gets its own full prompt, but by sending them as a batch, you’re only paying the network overhead once.
• No Hallucination on Echoing: The model will only produce the label (or the small output following the prompt) rather than echoing the entire text.
• Clear Mapping: Each prompt is self-contained, so there’s no confusion about the ordering or association between an input string and its output.

Alternative Considerations

• Fine-Tuning: If you find that even with clear prompts the model struggles with your classification task, fine-tuning a model specifically for your task might help. This can reduce token overhead since your classification instructions can then be “baked in” to the model behavior.
• Deprecated Classification API: The earlier classification API is deprecated. The current recommendation is to use either prompt-based approaches (like the one above) or fine-tuning.

Final Thoughts

There is no built‑in API that allows you to provide one pre‑prompt and then a series of independent “follow‑up” prompts in a single call for the Chat API. Using the completions endpoint with batch input is the recommended pattern. This way, you construct each prompt (by combining your instructions with the specific string) but submit them all at once to get back N completions, each corresponding to your N inputs.

This approach minimizes repetition while ensuring the model context is clear—providing efficiency both in terms of cost and processing time.