How Do I Configure AsyncAzureOpenAI Client to Use a Custom FastAPI Endpoint?

Below is an explanation and an example of how you can correctly set up the AsyncAzureOpenAI client to work with your custom endpoints.

When using Azure OpenAI via the OpenAI SDK, the client automatically appends paths such as "/openai/chat/completions" and also adds an API version query parameter (e.g. ?api-version=2024-06-01). That means you should pass the “base” Azure endpoint (i.e. your resource endpoint URL without the extra API paths or deployment parts) via the azure_endpoint (or base_url) parameter. In other words, instead of including your custom route (e.g. …/v1/genai/models/deployments/gpt-4/chat/completions) in the azure_endpoint, you should set:

• The azure_endpoint to the root endpoint for your Azure resource (for example, http://localhost:8080)
• The deployment/model via the deployment_id parameter (or model parameter as required)
• The API version with the api_version parameter.

That way the SDK knows to build the correct URL and won’t append extra paths to what you already provided.

Below is an example of how you might configure your client:

PYTHONCopied
from openai import AsyncAzureOpenAI

LATEST_URL = "http://localhost:8080"  # This is your base endpoint (without appended paths)
deployment_id = "gpt-4"
B2B_token = "xxxx"

client = AsyncAzureOpenAI(
    azure_endpoint=LATEST_URL,
    deployment_id=deployment_id,      # Specify your model deployment id here
    api_version="2024-06-01",          # Use the desired API version
    api_type="azure",                # Indicate that you are using Azure
    api_key=B2B_token,
    max_retries=0
)

response = await client.chat.completions.create(
    messages=[{"role": "user", "content": "Hi"}],
    deployment_id=deployment_id,      # Use deployment_id to target your custom model endpoint
    max_tokens=50,
    temperature=0.7,
    timeout=30
)

print(response)

Key Notes

• Avoid embedding route paths in azure_endpoint:
  Do not pass a URL like http://localhost:8080/v1/genai/models/deployments/gpt-4/chat/completions as azure_endpoint because the client will append its own fixed path (/openai/chat/completions) and include an API version query parameter.

• Utilize deployment_id:
  You can pass the specific deployment or model id via the deployment_id (or sometimes the model parameter) so that the SDK constructs the request appropriately.

• Set the API version:
  Specify the api_version explicitly if needed. The client will add that as a query string to the final URL.

By following this approach, your custom FastAPI endpoints (with authentication using B2B token) should be correctly targeted by the AsyncAzureOpenAI calls without unwanted URL modifications that lead to 404 errors.