How can I enforce strict adherence to the mail_tool rules in my LangChain agent?

Issue Description:

I am developing a LangChain agent that uses LangChain, langchain_openai, and a custom email-sending tool called mail_tool. This tool is designed to follow a strict set of rules and a specific input format for sending emails. However, the agent occasionally ignores these rules, invents email addresses, or fails to send the correct content.

Tool Rules and Input Format:

The mail_tool must adhere to the following:

1. Tool Usage: Use this tool only when the user explicitly requests to send an email and provides both their email address and the required content.
2. No Invented Emails: Only use the email provided by the user, without inventing any additional addresses.
3. Input Format: The tool expects a string that represents a dictionary (since the model outputs strings) formatted as follows:

JSONCopied
{
    'mail': 'user_email',  # The user's email address.
    'content': 'content_to_send'  # The information to be sent.
}

Code Implementation:

Below is the code implemented so far:

PYTHONCopied
import asyncio
from langchain_openai import ChatOpenAI
from langchain.tools import Tool
from langgraph.checkpoint.memory import MemorySaver
from langchain_core.messages import HumanMessage, AIMessage
from langgraph.graph import StateGraph, START, END, MessagesState
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import create_openai_functions_agent, AgentExecutor
import json

# Model and tools configuration
model = ChatOpenAI(model="gpt-4", openai_api_key="KEY_HERE")

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly assistant named Axel, and you will answer questions. You only provide answers of a maximum of 25 words. You respond in Spanish"),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad")
])

def send_mail_tool(data: str):
    try:
        parsed_data = json.loads(data)
        mail = parsed_data.get("mail")
        content = parsed_data.get("content")
        if mail and content:
            return "Email sent"
        else:
            return "Error: 'mail' or 'content' is missing"
    except Exception as e:
        return f"Error: {str(e)}"

mail_tool = Tool(
    name="send_mail_tool",
    func=send_mail_tool,
    description="""
        FOLLOW THESE RULES STRICTLY TO USE THIS TOOL:
        - Use this tool only when the user requests to send information via email and provides their email address and the required content.
        - Only use it for this purpose.
        - Do not invent email addresses; only use the one provided by the user.
        - Always, without exception, use the Input format.
        - this is the Input Format:
            The input must be a dictionary with the following keys:
            {
                'mail': 'user_email',  # The user's email address.
                'content': 'content_to_send'  # The information to be sent.
            }
    """
)

tools = [mail_tool]

agent = create_openai_functions_agent(llm=model, prompt=prompt, tools=tools)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=False)

workflow = StateGraph(state_schema=MessagesState)

async def call_agent(state: MessagesState):
    user_input = state["messages"][-1].content
    chat_history = state["messages"][:-1]
    response = await agent_executor.ainvoke({
        "input": user_input,
        "chat_history": chat_history
    })
    return {"messages": state["messages"] + [AIMessage(content=response["output"])]}

workflow.add_node("agent", call_agent)
workflow.add_edge(START, "agent")

memory = MemorySaver()
app = workflow.compile(checkpointer=memory)

config = {"configurable": {"thread_id": "user1234"}}

app.update_state(config, {"messages": [
    AIMessage(content="You are a friendly assistant named Axel, and you will answer questions. You only provide answers of a maximum of 25 words. You respond in Spanish")
]})

async def main():
    print("Assistant Online. Type 'exit' to quit.")
    while True:
        user_input = input("\nYou: ")
        if user_input.lower() == "exit":
            print("Goodbye!")
            break
        app.update_state(config, {"messages": [HumanMessage(content=user_input)]})
        state = app.get_state(config).values
        try:
            output = await app.ainvoke(state, config)
        except Exception as e:
            print("\nAssistant: Sorry, an error occurred while processing your request.")
            print(f"Details: {str(e)}")
            continue
        print("\nAssistant:")
        print(output["messages"][-1].content)

if __name__ == "__main__":
    asyncio.run(main())

Specific Issues Encountered:

• The agent sometimes invents email addresses instead of using the provided one.
• The agent occasionally fails to send the correct content or follow the specified input format.

Question:

How can I modify my approach to ensure the agent consistently adheres to the rules and input format for mail_tool? Are there alternative methods or improvements for a more robust enforcement?

Additional Information:

1. I am using GPT-4 as the language model.
2. The send_mail_tool function is a simulation and does not send real emails, but it must follow the defined format and rules.

I appreciate any guidance or suggestions. Thank you!