|
近期，Qwen 发布了 QwQ-32B - 一个在许多基准测试中性能可与 DeepSeek-R1 相媲美的推理模型。QwQ在推理模型中集成了调用工具的能力，使其能够在使用工具的同时进行批判性思考，并根据反馈调整推理过程。这样的能力使得QwQ能够很好在Agentic System中使用。本文介绍如何通过vLLM和SgLang结合QwQ-32B，搭建OpenAI格式的聊天API，并与外部函数结合来拓展模型的更多功能。
tools是OpenAI的Chat Completion API中的一个可选参数，可用于提供函数调用规范（function specifications）。这样做的目的是使模型能够生成符合所提供的规范的函数参数格式。同时，API 实际上不会执行任何函数调用。开发人员需要使用模型输出来执行函数调用。
vLLM和SgLang均支持OpenAI-API的tool参数。通过tool参数以及其中的函数调用规范，QwQ将能决定何时调用什么样的函数，以及怎么调用函数。
注：本文测试用例参考OpenAI cookbook：https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models
本文主要包含以下两个个部分：
模型文件下载 modelscopedownload--model=Qwen/QwQ-32B--local_dir./QwQ-32B
环境安装 pipinstallvllmpipinstall"sglang[all]>=0.4.3.post2"
vLLM部署命令 vllmserve/ModelPath/QwQ-32B\--port8000\--reasoning-parserdeepseek_r1\--max_model_len4096\--enable-auto-tool-choice\--tool-call-parserhermes
sglang部署命令 python-msglang.launch_server--model-path/ModelPath/QwQ-32B--port3001--host0.0.0.0--tool-call-parserqwen25
模型调用使用OpenAI的API格式调用本地部署的QwQ模型 单轮对话 fromopenaiimportOpenAI#设置OpenAI的API密钥和API基础URL使用vLLM的API服务器。openai_api_key="EMPTY"openai_api_base="http://localhost:8000/v1"client=OpenAI(api_key=openai_api_key,base_url=openai_api_base,)#使用流式输出（stream=True）chat_response=client.chat.completions.create(model="path/to/QwQ-32B",messages=[{"role":"user","content":"你好"}],stream=True#启用流式响应)#处理流式输出contents=[]foreinchat_response:#print(e.choices[0].delta.content,end="")contents.append(e.choices[0].delta.content)print("".join(contents))
多轮对话 from openai import OpenAIimport os
# 初始化OpenAI客户端client = OpenAI(api_key = "empty",base_url="http://localhost:8000/v1")
reasoning_content = ""# 定义完整思考过程answer_content = "" # 定义完整回复is_answering = False # 判断是否结束思考过程并开始回复
messages = []conversation_idx = 1while True:print("="*20+f"第{conversation_idx}轮对话"+"="*20)conversation_idx += 1user_msg = {"role": "user", "content": input("请输入你的消息：")}messages.append(user_msg)# 创建聊天完成请求completion = client.chat.completions.create( model="path/to/QwQ-32B",# 此处以 qwq-32b 为例，可按需更换模型名称messages=messages,stream=True)print("\n" + "=" * 20 + "思考过程" + "=" * 20 + "\n")for chunk in completion:# 如果chunk.choices为空，则打印usageif not chunk.choices:print("\nUsage:")print(chunk.usage)else:delta = chunk.choices[0].delta# 打印思考过程if hasattr(delta, 'reasoning_content') and delta.reasoning_content != None:print(delta.reasoning_content, end='', flush=True)reasoning_content += delta.reasoning_contentelse:# 开始回复if delta.content != "" and is_answering is False:print("\n" + "=" * 20 + "完整回复" + "=" * 20 + "\n")is_answering = True# 打印回复过程print(delta.content, end='', flush=True)answer_content += delta.contentmessages.append({"role": "assistant", "content": answer_content})print("\n")# print("=" * 20 + "完整思考过程" + "=" * 20 + "\n")# print(reasoning_content)# print("=" * 20 + "完整回复" + "=" * 20 + "\n")# print(answer_content)
首先，定义模型调用函数 from openai import OpenAI # 设置 OpenAI 的 API 密钥和 API 基础 URL 使用 vLLM 的 API 服务器。openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"MODEL = "path/to/QwQ-32B" client = OpenAI(api_key=openai_api_key,base_url=openai_api_base,)
def chat_completion_request(messages, tools=None, tool_choice=None, model=MODEL):try:response = client.chat.completions.create(model=model,messages=messages,tools=tools,tool_choice="auto",)return responseexcept Exception as e:print("Unable to generate ChatCompletion response")print(f"Exception: {e}")raise
然后，我们定义一些实用工具，用于调用聊天完成 API 以及维护和跟踪对话状态。 defpretty_print_conversation(messages):role_to_color={"system":"red","user":"green","assistant":"blue","function":"magenta",}formessageinmessages:ifmessage["role"]=="system":print(colored(f"system:{message['content']}\n",role_to_color[message["role"]]))elifmessage["role"]=="user":print(colored(f"user:{message['content']}\n",role_to_color[message["role"]]))elifmessage["role"]=="assistant"andmessage.get("function_call"):print(colored(f"assistant:{message['function_call']}\n",role_to_color[message["role"]]))elifmessage["role"]=="assistant"andnotmessage.get("function_call"):print(colored(f"assistant:{message['content']}\n",role_to_color[message["role"]]))elifmessage["role"]=="function":print(colored(f"function({message['name']}):{message['content']}\n",role_to_color[message["role"]]))
这里假设了一个天气 API，并设置了一些函数规范和它进行交互。将这些函数规范传递给 Chat API，以便模型可以生成符合规范的函数参数。 tools=[{"type":"function","function":{"name":"get_current_weather","description":"Getthecurrentweather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"Thecityandstate,e.g.SanFrancisco,CA",},"format":{"type":"string","enum":["celsius","fahrenheit"],"description":"Thetemperatureunittouse.Inferthisfromtheuserslocation.",},},"required":["location","format"],},}},{"type":"function","function":{"name":"get_n_day_weather_forecast","description":"GetanN-dayweatherforecast","parameters":{"type":"object","properties":{"location":{"type":"string","description":"Thecityandstate,e.g.SanFrancisco,CA",},"format":{"type":"string","enum":["celsius","fahrenheit"],"description":"Thetemperatureunittouse.Inferthisfromtheuserslocation.",},"num_days":{"type":"integer","description":"Thenumberofdaystoforecast",}},"required":["location","format","num_days"]},}},]
如果我们向模型询问当前的天气情况，它将会反问，希望获取到进一步的更多的参数信息。 messages=[]messages.append({"role":"user","content":"hi，canyoutellmewhat'stheweatherliketoday"})chat_response=chat_completion_request(messages,tools=tools)print(chat_response)assistant_message=chat_response.choices[0].messagemessages.append(assistant_message)assistant_message
一旦我们通过对话提供缺失的参数信息，模型就会为我们生成适当的函数参数。 messages.append({"role":"user","content":"I'minGlasgow,Scotland."})chat_response=chat_completion_request(messages,tools=tools)assistant_message=chat_response.choices[0].messagemessages.append(assistant_message)assistant_message
通过不同的提示词，我们可以让它反问不同的问题以获取函数参数信息。 messages=[]messages.append({"role":"user","content":"canyoutellme,whatistheweathergoingtobelikeinGlasgow,Scotlandinnextxdays"})chat_response=chat_completion_request(messages,tools=tools)assistant_message=chat_response.choices[0].messagemessages.append(assistant_message)assistant_messagemessages.append({"role":"user","content":"5days"})chat_response=chat_completion_request(messages,tools=tools)chat_response.choices[0]
并行函数调用 支持一次提问中，并行调用多次函数 messages = []messages.append({"role": "user", "content": "what is the weather going to be like in San Francisco and Glasgow over the next 4 days"})chat_response = chat_completion_request(messages, tools=tools, model=MODEL)
assistant_message = chat_response.choices[0].message.tool_callsassistant_message
| 
