How can I prevent hallucinations in Whisper transcriptions?

Below are some techniques to help reduce extra words (hallucinations) in your transcription:

1. Lower the Temperature

Setting a low temperature can reduce randomness during generation. A lower temperature (for example, 0) makes the model more deterministic. For instance, when using the Python client:

PYTHONCopied
transcription = client.audio.transcriptions.create(
    model="whisper-1",
    file=audio_file,
    response_format="text",
    temperature=0
)
print(transcription.text)

2. Use a Clear Prompt

If your audio contains filler words or other nuances, consider using the prompt parameter to guide the model. A prompt that mirrors the expected style can keep the transcription closer to the original content.

3. Ensure High-Quality Audio Input

Good audio quality helps reduce misinterpretations. Background noise or unclear speech can sometimes lead to extra words being added. Using clean, high-quality audio minimizes such issues.

4. Post-Processing

Sometimes, minor discrepancies remain. Using word timestamps (available in the transcription object) lets you review the alignment of words in your transcription, making it easier to identify and fix extra words if they occur.

By combining these techniques, you can improve transcription accuracy and minimize hallucinations in the output.