Incorrect UTF-8 Encoding for Function Call Parameters in GPT-4-1106-Preview

Overview

It sounds like you’re encountering an issue with the preview version of GPT-4 (gpt-4-1106-preview) where function calls result in parameters that aren’t correctly UTF-8 encoded for non-ASCII characters. This behavior isn’t observed with the stable models (gpt-3.5-turbo and gpt-4). Since gpt-4-1106-preview is a preview model, it might still have some rough edges or unresolved encoding bugs.

Recommendations

• Report Through Official Channels:
  Since you have already reported this issue, consider following up with additional context (if possible) using OpenAI’s official support channels or developer forums. This helps ensure the issue is prioritized for resolution.

• Model Choice for Production Use:
  If correct encoding is critical for your application, it may be advisable to continue using the stable models (gpt-4 or gpt-3.5-turbo) until the preview model receives further updates.

• Workaround Consideration:
  In scenarios where you must use gpt-4-1106-preview, you might implement post-processing on the output parameters to check and correct potential UTF-8 encoding issues—but this depends on your workflow and how critical immediate accuracy is.

Additional Information

GPT-4-1106-preview is designed with features like improved instruction following, JSON mode, reproducibility for function call outputs, and parallel function calling. However, since it is still a preview model, it’s not uncommon to encounter edge-case issues such as this with non-ASCII data handling.

Please keep an eye on official release notes and documentation for updates addressing this issue. Your feedback is valuable for improving these models, and continued reporting from users often helps prioritize fixes in subsequent updates.