How do I delete Pinecone vectors by filtering on metadata?

Below is one way to approach the issue:

---

What Might Be Happening

• Malformed Request Payload:
  Even if you’re not seeing a formal error message, returning an empty response can mean Pinecone correctly processed the request but didn’t find any vectors matching your filter criteria. This is often due to the payload not being in the exact structure Pinecone expects.

• Filter Structure in Metadata:
  Pinecone requires the filter to be a valid JSON object that matches exactly how you stored your metadata. For example, if your metadata key is "file," the filter should look like "filter": { "file": "your_filename" }.

• Endpoint or HTTP Method Issues:
  Make sure you're using the correct endpoint and method for deletion. Pinecone typically expects a POST (or sometimes a DELETE) request to a specific URL, such as /vectors/delete.

---

How to Fix It

To successfully filter and delete the vectors, ensure that:

1. Your Metadata Matches Exactly:
   Confirm that the metadata field (e.g., "file") you’re trying to filter on is indeed exactly named and formatted including data type (string, number, etc.).

2. JSON Request Payload is Correct:
   Your JSON should include the filter correctly nested. For instance:

   BASHCopied
   curl -X POST "https://<your-index-name>-<your-environment>.svc.pinecone.io/vectors/delete" \
     -H "Api-Key: <your-api-key>" \
     -H "Content-Type: application/json" \
     --data-raw '{
       "filter": {
         "file": "filename_to_delete"
       }
     }'

   A couple of things to note:

   • Replace <your-index-name>, <your-environment>, and <your-api-key> with your actual values.
   • Verify that "filename_to_delete" exactly matches what’s stored in your metadata.

3. Endpoint Usage:
   Double-check if Pinecone has recently updated its API. If they now require the deletion endpoint to be a DELETE HTTP method rather than POST, adjust your curl accordingly. For example:

   BASHCopied
   curl -X DELETE "https://<your-index-name>-<your-environment>.svc.pinecone.io/vectors/delete" \
     -H "Api-Key: <your-api-key>" \
     -H "Content-Type: application/json" \
     --data-raw '{
       "filter": {
         "file": "filename_to_delete"
       }
     }'

   (Note: Read the Pinecone docs for the current version to confirm the required HTTP method.)

---

Debugging Steps

• Log the Raw Request:
  If possible, log the complete outgoing curl request or HTTP request within your n8n workflow to double-check the structure.

• Check Stored Metadata Directly:
  Make sure the “file” key exists in the metadata for the documents and that the value matches what you’re passing in your filter.

• Use a Test File:
  Try inserting a test vector with known metadata and then attempt to delete it using the filter. This can help ensure that the issue isn’t with the stored data structure.

By ensuring your request structure is aligned exactly with Pinecone’s requirements and that the metadata you’re filtering against is correct, you should be able to successfully delete your vectors.

Below is one of the most common issues people run into when trying to delete vectors by metadata in Pinecone: the filter in the JSON body isn’t formatted exactly as the API expects. Even if the HTTP node returns an empty object, it can mean that no vectors were found to delete because the filter didn’t match any records.

What to Check

1. JSON Body Structure
   The deletion endpoint expects a JSON body that either contains an "ids" array (to delete specific vectors) or a "filter" object to target vectors by meta criteria. For example, if your vectors’ metadata includes a key called "file", the filter should be structured to match that value exactly.

2. Metadata Consistency
   Verify that when your vectors were added to Pinecone, you used the exact key name ("file") and that the value you are filtering on exactly matches what is stored in the index.

3. HTTP Headers & Endpoint URL
   Make sure your cURL request includes the proper headers (especially the API key header and Content-Type: application/json) and that your endpoint URL is correctly set to your Pinecone index.

Example cURL Request

Below is an example cURL command to delete vectors with a metadata filter based on the filename:

BASHCopied
curl --request POST 'https://YOUR_INDEX_NAME-your-project.svc.us-west1-gcp.pinecone.io/vectors/delete' \
  --header 'Api-Key: YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "filter": {
      "file": "your_file_name.txt"
    }
  }'

Additional Considerations

• Empty Response: If you still get an empty object, it might indicate that no vectors matched the filter. Double-check that the file name in your query exactly matches the file name stored in the metadata.
• Debugging: You can add logging or test the filter using Pinecone’s built-in query methods (or a separate tool) to ensure it returns expected matches.
• Documentation: Refer to Pinecone’s delete vectors documentation (if available within your preferred version) to ensure your cURL request meets the latest API requirements.

By confirming these details, you should be able to adjust your request so that Pinecone correctly identifies and deletes the vectors you intend to update.