How can I dynamically filter Qdrant searches using metadata from user chat prompts?

Below is one approach you can use to dynamically construct a metadata filter from the user’s chat prompt. The idea is to extract the relevant term (for example, "apples" or "strawberries") from the prompt and then inject that term into a Qdrant filter. This filter will make sure that the similarity search is only executed on those chunks that have a matching metadata value.

For example, if your metadata key is named "category" (or something similar), you could write code like the following:

PYTHONCopied
from qdrant_client import models

# Step 1: Extract metadata value from the user's chat prompt
# For instance, if the user prompt contains a keyword, set it to metadata_value
# You might have some NLP logic to extract the keyword.
user_prompt = "Can you tell me about apples?"  # example input
# A simple example: assume the keyword is the last word of the prompt
metadata_value = user_prompt.split()[-1].strip('?').lower()  # yields "apples"

# Step 2: Create the metadata filter using the extracted value
metadata_filter = models.Filter(
    must=[
        models.FieldCondition(
            key="metadata.category",  # adjust this key to match your metadata field name
            match=models.MatchValue(value=metadata_value)
        )
    ]
)

# Step 3: Use the metadata_filter in your Qdrant search
results = qdrant.search(
    query_vector,       # the vector you got for the page content
    top_k=4,            # adjust the number as needed
    metadata_filter=metadata_filter
)

Explanation

1. Extracting the Metadata:
   In this example, we take the user’s prompt and use simple string handling to extract the keyword. In your application you might need a more robust approach (for example, using regex or NLP) to correctly identify whether the user is asking about "apples", "strawberries", or another metadata value.

2. Constructing the Filter:
   The metadata_filter is built using Qdrant’s filtering capabilities. Here, a filter is defined on the metadata field (e.g., "metadata.category"). When the search is executed, only those chunks that include a matching value in their metadata will be considered.

3. Integrating the Filter with the Search:
   When calling the search function on Qdrant, the filter is passed in as metadata_filter. This tells Qdrant to first restrict the set of chunks based on the metadata criteria before applying the vector search on the remaining content.

This way, the similar documents are only retrieved from the parts of the collection that match the metadata extracted from the user’s query. It keeps the metadata separate from the content of the page while still ensuring that the query is limited to the relevant subset of data.