How can I dynamically update the Chroma vector database without PM2 reloads in a live Python environment?

Example Code

PYTHONCopied
# ----------- code to store data in vectordb ----------------
ext_to_loader = {
    '.csv': CSVLoader,
    '.json': JSONLoader,
    '.txt': TextLoader,
    '.pdf': PDFPlumberLoader,
    '.docx': Docx2txtLoader,
    '.pptx': PPTXLoader,
    '.xlsx': ExcelLoader,
    '.xls': ExcelLoader,
    'single_page_url': WebBaseLoader,
    'all_urls_from_base_url': RecursiveUrlLoader,
    'directory': DirectoryLoader
}

def get_loader_for_extension(file_path):
    _, ext = os.path.splitext(file_path)
    loader_class = ext_to_loader.get(ext.lower())
    if loader_class:
        return loader_class(file_path)
    else:
        print(f"Unsupported file extension: {ext}")
        return None

def normalize_documents(docs):
    return [
        doc.page_content if isinstance(doc.page_content, str) else '\n'.join(doc.page_content) if isinstance(doc.page_content, list) else ''
        for doc in docs
    ]

def vectorestore_function(split_documents_with_metadata, user_vector_store_path):
    try:
        # Create vector store with metadata
        embeddings = OpenAIEmbeddings(
            model = "text-embedding-ada-002", 
            openai_api_key=OPENAI_API_KEY
        )

        vector_store = Chroma(
            embedding_function=embeddings, 
            persist_directory=user_vector_store_path
        )
        
        vector_store.add_documents(documents=split_documents_with_metadata)
        
        return vector_store
    except Exception as e:
        print(f'Error in vectorestore_function {str(e)}')

loader = get_loader_for_extension(saved_file_path)
docs = loader.load()
normalized_docs = normalize_documents(docs)
text_splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size)
split_docs = text_splitter.create_documents(normalized_docs)

split_documents_with_metadata = [
    Document(page_content=document.page_content, metadata={"user_id": user_id, "doc_id": document_id})
    for document in split_docs
]
vectorestore_function(
    split_documents_with_metadata, 
    user_vector_store_path
)
#Note: I use above (same) code to add or update new data


# ----------------------------------------------------------- code for interaction with AI -----------------------------------------------------------
def get_vector_store(user_vector_store_path):
    
    embeddings = OpenAIEmbeddings(
            model = "text-embedding-ada-002", 
            openai_api_key=OPENAI_API_KEY
        )
    vectorstore = Chroma(
            embedding_function=embeddings,
            persist_directory=user_vector_store_path
        )
    
    return vectorstore
document_id_list = [str(document_id) if isinstance(document_id, int) else document_id for document_id in document_id_list]

user_vector_store_path = os.path.join(VECTOR_STORE_PATH, user_id)
vectorstore = get_vector_store(user_vector_store_path)

retriever = vectorstore.as_retriever()

current_threshold = 0.25
try:
    # Configure filtering
    retriever.search_type = "similarity_score_threshold"
    retriever.search_kwargs = {
        "filter": {
            "$and": [
                {"user_id": user_id},
                {"doc_id": {"$in": document_id_list}}
            ]
        },
        "score_threshold": current_threshold,
        "k": 3
    }

    retrieved_docs = retriever.invoke(question)
except Exception as e:
    print(f'error: {str(e)}')

print(f"retrieved_docs : {retrieved_docs}")


if not retrieved_docs:
    return jsonify({'error': f'No relevant docs were retrieved.'}), 404

Error Message and Stack Trace (if applicable)

WARNING:langchain_core.vectorstores.base:No relevant docs were retrieved using the relevance score threshold 0.25

Description:

I’m encountering an issue on my live server where a new vector database is created successfully for a new user, and data is stored correctly. However, when new data is added, although it is saved in the vector database, it is not retrievable immediately.

This problem does not occur in the local environment; on the live server, the updated data only becomes available after executing pm2 reload "id".

Reloading PM2 forces a disconnection of active socket connections, which adversely affects users who may be in the middle of a conversation.

Tech Stack:

• Flutter – for the mobile application
• Node.js – for the back office
• Python – for data extraction, vector database creation, and AI conversations

Additional details:

• File downloads, embedding creation, and vector database updates are managed with Celery.
• The server uses Apache, and PM2 manages the application process.

Issue:

New data is added to the vector database but is not retrievable until a PM2 reload is triggered, which disrupts ongoing socket connections.

Desired Outcome:

I want the system to update the vector database dynamically so that new or updated data can be immediately accessed without requiring a PM2 reload and interrupting active socket connections.

PYTHONCopied
socketio.emit('status', {'message': {
    "user_id": user_id,
    "document_id": document_id,
    "status": 200,
    "message": f"Document ID {document_id} processed successfully."
}}, room=room)

Although status updates are correctly emitted and users can start conversations after receiving them, the retrieval issue for newly added data persists. Therefore, how can I ensure that the vector database is updated dynamically without reloading PM2, thereby preserving active socket connections?