Asked 1 month ago by StarlitKeeper922
Why does Azure OpenAI Completion Request Fail with the gpt-4o Model?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Asked 1 month ago by StarlitKeeper922
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
I built a RAG web service that worked about 6 months ago, but now it fails completely.
I discovered that the SDK has changed; deprecated models were replaced with new ones. However, the new API calls are not working correctly.
Below is the code that uses the new API. When I use the new model: gpt-4o, the call to CompleteChatAsync never returns unless a CancellationToken is provided—in which case it times out after 30 sec. I get exceptions such as "The task canceled" or "Resource not found". The same code works fine with the model gpt-35-turbo-16k.
The Resource Group and the Resource are in 'eastus'.
The gpt-4o model with Deployment Name/Model ID 'gpt-4o-2024-08-06' was deployed successfully a while ago.
Note: The model deployment name mentioned is gpt-40-24-08-06 and the target URI endpoint is:
https://azureopenaiexperiment.openai.azure.com/openai/deployments/gpt-4o-2024-08-06/chat/completions?api-version=2024-08-01-preview
My question is: Why is it not working? What am I missing?
Additional Environment Details:
Windows 11, Visual Studio 2022.
.NET 9
Microsoft.NETCore.APP 9.0.0
Microsoft.Windows.SDK.NET.Ref.Windows 10.0.26100.54
Code:
CSHARPpublic async Task<string?> GenerateResponseAsync(CompletionModel model, string prompt, List<string> contextList, CancellationToken cancellationToken) { _logger.LogInformation("GenerateResponseAsync entered"); ArgumentNullException.ThrowIfNull(model, nameof(model)); ArgumentNullException.ThrowIfNullOrWhiteSpace(prompt, nameof(prompt)); string respMessage = ""; try { Uri azureOpenAIResourceUri = new Uri(_azureSettings.AzureOpenAiEndpoint); AzureKeyCredential azureOpenAiApiKey = new AzureKeyCredential(_azureSettings.AzureOpenAiApiKey); AzureOpenAIClient azureClient = new(azureOpenAIResourceUri, azureOpenAiApiKey); ChatClient chatClient = azureClient.GetChatClient(model.DeploymentName); if (chatClient == null) { throw new Exception("Failed to create AzureOpenAIClient"); } List<ChatMessage>? chatMessages = ConstructMessages(model, prompt, contextList); if (chatMessages == null) { throw new Exception("Failed to construct messages"); } _logger.LogInformation("Creating ChatCompletionOptions..."); ChatCompletionOptions? chatCompletionOptions = new ChatCompletionOptions() { FrequencyPenalty = 0f, PresencePenalty = 0f, MaxOutputTokenCount = model.TokenLimit - 1000, TopP = 0, Temperature = 1.0f, }; using var timeoutTokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(30)); var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeoutTokenSource.Token); _logger.LogInformation("Calling CompleteChatAsync..."); ChatCompletion chatCompletion = await chatClient.CompleteChatAsync(chatMessages.ToArray(), chatCompletionOptions, linkedTokenSource.Token); if (chatCompletion != null) { ChatMessageContent msgContent = chatCompletion.Content; if (msgContent != null) { respMessage = (msgContent.Count > 0) ? msgContent[0].Text : ""; } } _logger.LogInformation($"CONTENT: {respMessage}"); } catch (Exception ex) { _logger.LogError(ex, ex.Message); throw; } finally { _logger.LogInformation("GenerateResponse exiting"); } return respMessage; } public List<ChatMessage>? ConstructMessages(CompletionModel model, string prompt, List<string> contextList) { _logger.LogInformation("ConstructMessages entered"); try { StringBuilder sb = new StringBuilder($"QUESTION: {prompt}"); sb.AppendLine(Environment.NewLine); sb.AppendLine("CONTEXT:"); sb.AppendLine(Environment.NewLine); if (contextList != null) { for (int i = 0; i < contextList.Count; i++) { sb.AppendLine("## " + contextList[i].Trim('\n').Trim('\r')); sb.AppendLine(Environment.NewLine); } } List<ChatMessage> chatMessages = new List<ChatMessage>() { // The system message represents instructions or other guidance about how the assistant should behave new SystemChatMessage(_azureSettings.AzureOpenAiCompletionSystemInstruction), new UserChatMessage(sb.ToString()), }; _logger.LogInformation($"SystemInstruction: {chatMessages[0]}"); _logger.LogInformation($"User Message: {chatMessages[1]}"); string totPayloadStr = _azureSettings.AzureOpenAiCompletionSystemInstruction + sb.ToString(); if (IsExceedsModelTokenLimit(totPayloadStr, model) == true) { throw new Exception("Total prompt + system + context message size exceeds the model limit"); } _logger.LogInformation("Exiting exiting..."); return chatMessages; } catch(Exception ex) { _logger.LogError($"ConstructMessages exception: {ex.Message}"); } return null; }
Please help me identify what I'm missing.
Comments 0
•Answers 2
•Views 150
Answer by TychoAI Agent • 1 month ago
Below is a checklist of items to verify:
Deployment Name Consistency
Make sure you’re using the exact deployment name that was created in the Azure portal. In your code you mention "gpt-40-24-08-06" in one place and "gpt-4o-2024-08-06" in your endpoint URL. That kind of mismatch (a “0” versus an “O”) will cause the client call to fail with a "Resource not found" error.
Correct API Version
The endpoint in your code includes the API version parameter (?api-version=2024-08-01-preview). The new GPT-4o models are still in preview so double-check that your subscription or resource supports this preview version.
Model Availability and Permissions
Ensure that your Azure OpenAI resource is provisioned for GPT‑4o. The fact that your call works with gpt-35-turbo-16k indicates that the overall service is functioning but that GPT‑4o might require additional configuration or permissions in your Azure resource.
SDK and Service Differences
The new SDK now automatically uses different endpoints and may treat certain models differently. If the call doesn’t return unless a cancellation token is provided, it might be that the request isn’t reaching a valid deployment or is waiting indefinitely because of a misconfiguration. Verifying (1) and (2) should address that issue.
In summary, check that the deployment name in your code exactly matches what’s configured on your Azure resource and that you’re using the correct API version for GPT‑4o. Once those details are aligned, your request should complete correctly rather than silently timing out or throwing "Resource not found."
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Answer by SupernovaSeeker681 • 2 months ago
As mentioned, you are encountering issues when using the gpt-4o
model with the CompleteChatAsync
method, where the call either times out or throws a "Resource not found" exception. This behavior contrasts with the successful execution for the gpt-35-turbo-16k
model.
Initially, I observed similar behavior when attempting to use the gpt-4o
model with an insufficient timeout or a mismatched deployment name and endpoint configuration.
For example, a call using this setup would fail due to the timeout issue:
CSHARPusing var timeoutTokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(30)); var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeoutTokenSource.Token); ChatCompletion chatCompletion = await chatClient.CompleteChatAsync(chatMessages.ToArray(), chatCompletionOptions, linkedTokenSource.Token);
To address this, increasing the timeout duration resolved the issue. Updating the timeout to 60 seconds or more ensures sufficient time for the gpt-4o
model to generate a response:
CSHARPusing var timeoutTokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(60)); var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeoutTokenSource.Token); ChatCompletion chatCompletion = await chatClient.CompleteChatAsync(chatMessages.ToArray(), chatCompletionOptions, linkedTokenSource.Token);
Additionally, the "Resource not found" error was due to a mismatch in the deployment name and endpoint configuration. Using the correct deployment name (gpt-4o-2024-08-06
) and verifying the API endpoint resolved this issue:
CSHARPUri azureOpenAIResourceUri = new Uri("https://azureopenaiexperiment.openai.azure.com/"); ChatClient chatClient = azureClient.GetChatClient("gpt-4o-2024-08-06");
If you need to use the gpt-4o
model consistently, check below. But before using refer this link.
gpt-4o
model.gpt-4o
model in the eastus
region.Finally, the same approach should work if using other tools like Azure SDK or PowerShell to interact with the Azure OpenAI resource. Adjusting the parameters and checking the configuration aligns with the resource setup should resolve the issue.
No comments yet.
No comments yet.