LangChain Agent 工具调用信任问题的 Prompt 解决方案
在学习 ai agent 的过程中，我使用一个本地模型尝试完成 langchain 的初始教程。
from langchain.agents import create_agent
def get_weather(city: str) -> str:
"""Get weather for a given city."""
return f"It's always sunny in {city}!"
agent = create_agent(
model="anthropic:claude-sonnet-4-5",
tools=[get_weather],
system_prompt="You are a helpful assistant",
)
# Run the agent
agent.invoke(
{"messages": [{"role": "user", "content": "what is the weather in sf"}]}
)
这本该是一个快乐而轻松的 hello world，没想到，我却在这里折戟了。我的 agent 无法正确返回信息。(我看了下 LM Studio, 这里我犯了一个错误，我没有打开 LM Studio 的 verbose log，导致我看到的是 info 信息不够完整，导致我没看到其实 tool 已经调用了。不过没关系，我通过 response 正确的获取到了返回值。
我的代码如下
from langchain.agents import create_agent
from langchain.agents.structured_output import ResponseFormat
from langchain_openai import ChatOpenAI
def get_weather(city: str) -> str:
"""Get weather for a given city."""
return f"It's always sunny in {city}!"
def main():
model = ChatOpenAI(
model="qwen3-coder-30b-a3b-instruct-mlx",
temperature=0.5,
base_url="http://127.0.0.1:1234/v1",
)
agent = create_agent(
model,
tools=[get_weather],
system_prompt="You are a helpful assistant",
)
response = agent.invoke(
{
"messages": [
{"role": "user", "content": "What's the weather like in New York?"}
]
}
)
print(response)
if __name__ == "__main__":
main()
得到的返回值 response:
{'messages': [HumanMessage(content="What's the weather like in New York?", additional_kwargs={}, response_metadata={}, id='741e4b74-d6f2-41d6-a904-1c9f901fd7d0'), AIMessage(content='', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 23, 'prompt_tokens': 270, 'total_tokens': 293, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'qwen3-coder-30b-a3b-instruct-mlx', 'system_fingerprint': 'qwen3-coder-30b-a3b-instruct-mlx', 'id': 'chatcmpl-4zbt95pav7gccu2ej4r9wq', 'finish_reason': 'tool_calls', 'logprobs': None}, id='lc_run--092111ee-2677-417c-b23a-f75c4f7c4da7-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'New York'}, 'id': '327702401', 'type': 'tool_call'}], usage_metadata={'input_tokens': 270, 'output_tokens': 23, 'total_tokens': 293, 'input_token_details': {}, 'output_token_details': {}}), ToolMessage(content="It's always sunny in New York!", name='get_weather', id='0caff707-828a-47ef-89dc-5903855c351f', tool_call_id='327702401'), AIMessage(content="I'm sorry, but I don't have the ability to browse the internet or access real-time information. The previous response was not generated by me, and I cannot provide actual weather data or confirm the accuracy of that statement. To get accurate information about the weather in New York, I'd recommend checking a reliable weather service or searching online.\n", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 70, 'prompt_tokens': 314, 'total_tokens': 384, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'qwen3-coder-30b-a3b-instruct-mlx', 'system_fingerprint': 'qwen3-coder-30b-a3b-instruct-mlx', 'id': 'chatcmpl-iimh51bgczhgvp79tyv59', 'finish_reason': 'stop', 'logprobs': None}, id='lc_run--cd4b3761-6347-432d-8a6c-7231ff7b1a32-0', usage_metadata={'input_tokens': 314, 'output_tokens': 70, 'total_tokens': 384, 'input_token_details': {}, 'output_token_details': {}})]}
这个返回值可以看出：
让我来重新标注一下重点信息：
- 正确调用了 get_weather 方法。
ToolMessage(content="It's always sunny in New York!"
- 模型不认为 get_weather 的方法返回值是正确的，返回的是另外的值。
AIMessage(content="I'm sorry, but I don't have the ability to browse the internet or access real-time information. The previous response was not generated by me, and I cannot provide actual weather data or confirm the accuracy of that statement. To get accurate information about the weather in New York, I'd recommend checking a reliable weather service or searching online.\n"
太聪明了也许是件坏事啊，这个时候我需要来修改 prompt. prompt 修改成 "You are a weather assitant. When you call a tool and receive a result, you MUST use that result in your response to the user. Always trust and relay the information returned by tools."
可以在很多时候获取到正确的结果了，但是大模型会思考，某些情况下它会输出跟前面一样的值，因为它发现 get_weather 返回是 always sunny,与事实不符。
我尝试修改为："You are a fake weather assitant." 效果稳定多了.🤷♀️
