How do I fix the 'None Connection Type' error for the OpenAI embedding API in ai-rag-chat-evaluator?

Below is an explanation of what’s happening and how to fix it.

What’s Happening

The error

  Not Support connection type None for embedding api. Connection type should be in [AzureOpenAI, OpenAI]

occurs because the evaluation tool is trying to perform an embedding call but can’t detect a valid connection configuration. In other words, while your completion calls are using your OpenAI API key from OPENAICOM_KEY, the embedding (or vector) API isn’t getting a connection type—it’s defaulting to None. This happens when the necessary environment settings for embedding aren’t provided.

How to Fix It

When using OpenAI (openai.com) for both chat and embeddings, you need to ensure that your configuration tells the evaluation tool which connection to use. To do that:

1. Provide a Valid Organization ID
   Even if your evaluation for completions might work with just the API key, the underlying configuration for embeddings often requires a valid organization setting. In your .env file, set the organization variable by replacing the empty string:

   BASHCopied
   # For openai.com only:
   OPENAICOM_KEY="your-openai-api-key"
   OPENAICOM_ORGANIZATION="your-openai-organization-id"

2. Confirm Embedding Configuration
   Since your JSON configuration has "vector_fields": ["embedding"], the tool expects to send embedding requests. By providing the required organization (along with the API key), the tool can set the connection type to "OpenAI". If you don’t require embeddings, you could remove or update the "vector_fields" property, but typically you want them for search or evaluation metrics.

3. Double-Check Your Environment Variables
   Make sure there are no typos and that the variables are correctly loaded into the project. Some libraries load your .env file automatically, but it may help to verify they are being picked up correctly.

Summary

To work around the error, update your .env file so that for an OpenAI instance both the key and the organization are set. For example:

BASHCopied
OPENAI_HOST="openai"
OPENAI_GPT_MODEL="gpt-4o"  # using gpt-4o as in your config

# For openai.com only:
OPENAICOM_KEY="your-openai-api-key"
OPENAICOM_ORGANIZATION="your-openai-organization-id"

# For generating QA based on search index:
AZURE_SEARCH_ENDPOINT="my search url here"
AZURE_SEARCH_INDEX="gptkbindex"
# ... (other AZURE_SEARCH settings if applicable)

With this change, the evaluation tool should correctly determine your connection type as "OpenAI" for embedding API calls, and the error should no longer occur.