Why is the Retriever Assistant unable to process scanned PDFs?

Below is some clarification regarding your experience with scanned PDFs and GPT-4's image capabilities:

Understanding GPT-4 Vision Limitations

• Image Quality Matters:
  GPT-4’s image input is designed to work best with clear and unambiguous images. If a scanned PDF produces an image that’s blurry, low-resolution, or otherwise unclear, the model may struggle to extract the information accurately. The general guidance is that if an average human cannot read details at the provided resolution, neither can the model.

• File Type and Processing:
  The API is built to process image files (e.g., JPEG or PNG) rather than multi-page document formats like PDFs. If you’re trying to feed a scanned PDF directly, problems can arise unless the content is converted to a supported image format with sufficient clarity.

• No Fine-Tuning for Vision Aspects:
  It’s important to note that currently, there’s no support for fine-tuning the image processing part of GPT-4. This means that its performance relies on the input quality and inherent capabilities, leaving little room for adjustments when dealing with non-ideal inputs like scanned documents.

Next Steps

• Try Converting PDFs:
  Consider converting your scanned PDF pages into high-quality image files (ensuring the text is clear) before sending them to the API. This might lead to improved performance if the clarity issue is the primary challenge.

• Check Input Resolution:
  Ensure that your images meet a resolution where the details (like text in scans) are legible. As a rule of thumb, if a human finds the scan hard to read, the model might have similar difficulties.

If you continue to experience issues, it might be helpful to compare with other users’ experiences or reach out to official support channels to see if there’s an ongoing discussion or a known limitation regarding scanned PDFs.