返回顶部
热门问答 更多热门问答
技术文章 更多技术文章

ollama 大版本0.9 发布,支持流式响应和推理模式

[复制链接]
链载Ai 显示全部楼层 发表于 8 小时前 |阅读模式 打印 上一主题 下一主题


ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;color: rgb(63, 63, 63);">
ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;color: rgb(63, 63, 63);">

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;display: table;padding: 0px 0.2em;color: rgb(255, 255, 255);background: rgb(250, 81, 81);">简介

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">Ollama 连发两个大版本,分别是ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">v0.8.0ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">v0.9.0,0.8 版本后直接发 0.9 大版本,中间没有小功能迭代。

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">在 0.8 版本中,ollama 支持ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">工具调用流式响应(stream response with tool calls)。

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">在 0.9 版本中,ollama 增强了对推理模型的支持,支持开启和关闭模型的推理模式。

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;display: table;padding: 0px 0.2em;color: rgb(255, 255, 255);background: rgb(250, 81, 81);">0.8.0

支持工具调用的开源模型比较多,有:

  • •Qwen 3[1]
  • •Devstral[2]
  • •Qwen2.5[3]andQwen2.5-coder[4]
  • •Llama 3.1[5]
  • •Llama 4[6]
  • •其它模型[7]

我们以官方仓库ollama-python/examples[8]中的例子修改得到获取天气的样例代码 weather.py:

fromollamaimportChatResponse, chat

defget_current_weather(location:str,format:str) ->str:
return"33"

# Tools can still be manually defined and passed into chat
get_current_weather_tool = {
'type':'function',
'function': {
'name':'get_current_weather',
'description':'Get weather of location',
'parameters': {
'type':'object',
'required': ['location','format'],
'properties': {
'location': {'location':'string','description':'The location'},
'format': {'format':'string','description':'The result format'},
},
},
},
}

messages = [{'role':'user','content':'What is the weather today in Shanghai?'}]
print('Prompt:', messages[0]['content'])

available_functions = {
'get_current_weather': get_current_weather,
}

response: ChatResponse = chat(
'qwen3:latest',
messages=messages,
tools=[get_current_weather, get_current_weather_tool],
stream=True,
)

forrespinresponse:
# print('resp {}'.format(resp))
ifnotresp.message.tool_calls:
print(resp['message']['content'], end='', flush=True)
else:
# There may be multiple tool calls in the response
fortoolinresp.message.tool_calls:
# Ensure the function is available, and then call it
iffunction_to_call := available_functions.get(tool.function.name):
print('Calling function:', tool.function.name)
print('Arguments:', tool.function.arguments)
output = function_to_call(**tool.function.arguments)
print('Function output:', output)
else:
print('Function', tool.function.name,'not found')

# Add the function response to messages for the model to use
messages.append(resp.message)
messages.append({'role':'tool','content':str(output),'name': tool.function.name})

# Get final response from model with function outputs
forfrinchat('qwen3:latest', messages=messages, stream=True):
print(fr['message']['content'], end='', flush=True)

上述代码是获取指定位置的温度,得到结果后将温度传给模型,生成最终的结果:

$python weather.py
Prompt: What is the weather today in Shanghai?
<think>
Okay, the user is asking for the weather in Shanghai today. Let me check the tools provided. There's a function called get_current_weather. The parameters required are location and format. The description mentions getting the weather for a location. Even though the parameters' descriptions are a bit unclear, location is a string, so I should provide "Shanghai" as the location. For format, maybe the user wants a simple response, so I'll use "json" as a common format. I'll call the function with these arguments.
</think>

Calling function: get_current_weather
Arguments: {'format': 'json', 'location': 'Shanghai'}
Function output: 33
<think>
Okay, the user asked about the weather in Shanghai today. I called the get_current_weather function with the location set to Shanghai and format as JSON. The response came back as "33". Hmm, that's not a typical weather description. Maybe the API returned a temperature in Celsius? 33°C is quite hot. Let me check if there's more data. Wait, the response was just "33", so maybe the API is broken or there was an error. Alternatively, maybe it's a placeholder. I should inform the user that the information is incomplete. I'll mention the temperature if I assume it's Celsius and note that other details aren't available. Let me make sure to ask them to check again later or try another source.
</think>

The current weather in Shanghai is not fully available. However, based on the data received, the temperature is approximately **33°C**. For more detailed information (e.g., humidity, wind speed, or conditions), please check again later or use a dedicated weather service. Let me know if you'd like further assistance! ?️

0.9.0

当前,只有DeepSeek R1[9]和Qwen3[10]支持推理能力,后续会有更多模型支持。

开启和关闭推理能力,可以在调用 Ollama API 时指定"think": true

curl http://localhost:11434/api/chat -d '{
"model": "deepseek-r1",
"messages": [
{
"role": "user",
"content": "Why is the sky blue?"
},
],
"think": true
}'

也可以在 ollama 终端用/set think/set nothink来开启和关闭推理模式。

总结

工具调用的流式响应,可以带来更好的用户体验。推理模式开启和关闭,ollama 支持的有点晚,印象中最先支持的是 qwen3。

这两个功能都很酷,目前看一些开源 对话应用、支持库都还没支持 开启、关闭推理模式,希望开源社区尽快支持。

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

链载AI是专业的生成式人工智能教程平台。提供Stable Diffusion、Midjourney AI绘画教程,Suno AI音乐生成指南,以及Runway、Pika等AI视频制作与动画生成实战案例。从提示词编写到参数调整,手把手助您从入门到精通。
  • 官方手机版

  • 微信公众号

  • 商务合作

  • Powered by Discuz! X3.5 | Copyright © 2025-2025. | 链载Ai
  • 桂ICP备2024021734号 | 营业执照 | |广西笔趣文化传媒有限公司|| QQ