低显存福音！OpenAvatarChat开源！实测仅需4G显存即可畅玩多模态数字人！windows保姆级本地部署教程！ - 链载Ai

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">Open Avatar Chat 是由阿里开源的数字人对话系统，让数字人具备视觉感知+语音交互+情感表达多模态能力，能达到类似真人的对话体验。

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">本教程将采用硬件需求最低的部署方案：ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">SenseVoice+ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">LLM API+ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">CosyVoice API的方式运行 LAM 或 LiteAvatar 数字人。显存占用最低仅有3G！

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;display: table;padding: 0.3em 1.2em;color: rgb(255, 255, 255);background: rgb(250, 81, 81);border-radius: 8px 24px;box-shadow: rgba(0, 0, 0, 0.06) 0px 2px 6px;">实测显存占用情况

ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-size: 16px;letter-spacing: 0.1em;color: rgb(63, 63, 63);">在搭载ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">RTX3060 6G的笔记本电脑上运行测试
运行ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">LAM数字人，仅占用ingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-feature-settings: normal;font-variation-settings: normal;font-size: 14.4px;text-align: left;line-height: 1.75;color: rgb(221, 17, 68);background: rgba(27, 31, 35, 0.05);padding: 3px 5px;border-radius: 4px;">3.1G显存！对话流畅，性能相当强悍！
实测图如下：

运行LiteAvatar数字人，也仅需5.3G显存！实测图如下：

系统依赖

• N卡4G以上显存
• N卡支持 CUDA >= 12.4
• ffmpeg
• miniconda 或 Anaconda
• 阿里云百炼API KEY(可白嫖)

安装 miniconda 、ffmpeg 可查看往期文章
•安装miniconda
•安装ffmpeg

拉取仓库

# 拉取主仓库 --depth=1 仅拉取最新的提交，减少空间占用
gitclone--depth=1 https://github.com/HumanAIGC-Engineering/OpenAvatarChat.git
cdOpenAvatarChat

# 拉取子仓库
git submodule update --init --recursive

逐个拉取仓库(推荐)

针对网络环境不好的同学，推荐逐个拉取仓库。在拉取失败时能快速定位到异常仓库。
所有git clone命令均带有路径，可直接在主仓库根目录下执行。

注：由于是最小化组件安装，没有拉取MiniCPM-o仓库

所有子仓库可在.gitmodules文件中查看

主仓库 OpenAvatarChat

gitclone--depth=1 https://github.com/HumanAIGC-Engineering/OpenAvatarChat.git
cdOpenAvatarChat

liteavatar

# 跳过lfs下载大型文件，后续把大型文件单独下载后放入指定位置
# set GIT_LFS_SKIP_SMUDGE=1
gitclone--depth=1 https://github.com/HumanAIGC/lite-avatar.git ./src/handlers/avatar/liteavatar/algo/liteavatar

liteavatar仓库中共有三个大型文件，把它们单独下载到本地再放到对应位置：

• weights\model_1.onnx (175 MB)
• weights\speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch\model.pb (839 MB)
• weights\speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch\lm\lm.pb (226 MB)

silero_vad

gitclone--depth=1https://github.com/snakers4/silero-vad.git./src/handlers/vad/silerovad/silero_vad

gradio_webrtc_videochat

gitclone--depth=1https://github.com/HumanAIGC-Engineering/gradio-webrtc.git./src/third_party/gradio_webrtc_videochat

CosyVoice

# CosyVoice 仓库中包含子仓库：Matcha-TTS。添加参数 --recursive 将自动拉取
gitclone--depth=1 --recursive https://github.com/FunAudioLLM/CosyVoice.git ./src/handlers/tts/cosyvoice/CosyVoice

LAM_Audio2Expression

gitclone--depth=1https://github.com/aigc3d/LAM_Audio2Expression.git./src/handlers/avatar/lam/LAM_Audio2Expression

安装依赖

conda新建python 3.10环境

# 新建名为 openavatarchat 的 python 3.10 环境
conda create -n openavatarchat python=3.10
# 激活环境
conda activate openavatarchat

conda安装pynini

condainstall-cconda-forgepynini==2.1.6

安装pytorch

pipinstalltorchtorchvisiontorchaudio--index-urlhttps://download.pytorch.org/whl/cu124

安装fastrtc

这个fastrtc是经过OpenAvatarChat团队修改的版本，且打包成了whl包，跟随OpenAvatarChat仓库发布。千万不要安装gradio版的fastrtc

pipinstall-ihttps://mirrors.aliyun.com/pypi/simple./src/third_party/gradio_webrtc_videochat/dist/fastrtc-0.0.19.dev0-py3-none-any.whl

安装其它依赖

请严格按照以下顺序安装依赖包。已经过多轮测试，安装后可正常运行。

pip install -i https://mirrors.aliyun.com/pypi/simple aiohttp==3.11.16
pip install -i https://mirrors.aliyun.com/pypi/simple modelscope==1.25.0
pip install -i https://mirrors.aliyun.com/pypi/simple openai==1.74.0
pip install -i https://mirrors.aliyun.com/pypi/simple dynaconf==3.2.10
pip install -i https://mirrors.aliyun.com/pypi/simple email-validator==2.2.0
pip install -i https://mirrors.aliyun.com/pypi/simple fastapi-cli==0.0.7
pip install -i https://mirrors.aliyun.com/pypi/simple loguru==0.7.3
pip install -i https://mirrors.aliyun.com/pypi/simple opencv-python-headless==4.11.0.86
pip install -i https://mirrors.aliyun.com/pypi/simple pyaml==25.1.0
pip install -i https://mirrors.aliyun.com/pypi/simple funasr==1.2.6
pip install -i https://mirrors.aliyun.com/pypi/simple vocos==0.1.0
pip install -i https://mirrors.aliyun.com/pypi/simple vector-quantize-pytorch==1.21.9
pip install -i https://mirrors.aliyun.com/pypi/simple transformers==4.44.1
pip install -i https://mirrors.aliyun.com/pypi/simple h5py==3.12.1
pip install -i https://mirrors.aliyun.com/pypi/simple pypinyin==0.53.0
pip install -i https://mirrors.aliyun.com/pypi/simple typeguard==2.13.3
pip install -i https://mirrors.aliyun.com/pypi/simple onnxruntime-gpu~=1.20.1
pip install -i https://mirrors.aliyun.com/pypi/simple onnxruntime~=1.20.1
pip install -i https://mirrors.aliyun.com/pypi/simple dashscope==1.23.1
pip install -i https://mirrors.aliyun.com/pypi/simple edge-tts==7.0.0
pip install -i https://mirrors.aliyun.com/pypi/simple addict==2.4.0
pip install -i https://mirrors.aliyun.com/pypi/simple yapf==0.40.1
pip install -i https://mirrors.aliyun.com/pypi/simple spleeter==2.4.0
pip install -i https://mirrors.aliyun.com/pypi/simple gradio==5.9.1

下载模型文件

模型文件均来自modelscope魔塔社区，无需担心网络问题。
在OpenAvatarChat根目录下执行，模型文件会下载到对应的文件夹中

SenseVoice语音识别模型

modelscopedownload--modeliic/SenseVoiceSmall--local_dir./models/iic/SenseVoiceSmall

LiteAvatar提供的预训练数字人形象(100个)

modelscopedownload--modelHumanAIGC-Engineering/LiteAvatarGallery--local_dir./resource/avatar/liteavatar

lam 数字人依赖模型

modelscope download --model AI-ModelScope/wav2vec2-base-960h --local_dir ./models/wav2vec2-base-960h

modelscope download --model Damo_XR_Lab/LAM_audio2exp LAM_audio2exp_streaming.tar --local_dir ./models/LAM_audio2exp

注意：这里的LAM_audio2exp_streaming.tar压缩包需要解压，否则在运行lam数字人时报错：No checkpoint found

使用WinRAR解压有可能会遇到只解压出一个文件的问题，正常解压出来的应当是一个文件夹。猜测可能是解压软件的问题。遇到问题的同学可以修改LAM_audio2exp_streaming.tar文件后缀名为.tar.gz即可使用WinRAR解压。
如下图：

解后的目录结构为:
./models/LAM_audio2exp/pretrained_models/LAM_audio2exp_streaming.tar

白嫖大模型API

白嫖 CosyVoice 语音合成

• 阿里云百炼 CosyVoice-v1 链接:

https://bailian.console.aliyun.com/?tab=model#/model-market/detail/cosyvoice-v1

当白嫖额度消耗殆尽时，可在配置文件中替换为EDGE-TTS语言合成，但速度稍慢

白嫖LLM大语言模型

• 阿里云百炼 qwen-vl-plus 链接:

https://bailian.console.aliyun.com/?tab=model#/model-market/detail/qwen-vl-plus

• OpenRouter qwen2.5-vl-72b-instruct 链接:

https://openrouter.ai/qwen/qwen2.5-vl-72b-instruct:free

注意

在登录阿里云百炼后，所使用模型的白嫖额度会显示在页面中，注意查看：

修改配置文件

配置文件位于：config文件夹内

一、配置LiteAvatar数字人

1. 复制配置文件chat_with_openai_compatible_bailian_cosyvoice.yaml到同目录内，重命名为liteavatar.yaml
2. 编辑liteavatar.yaml文件，把CosyVoice apikey和LLM大语言模型apikey填进去。如下图示例：

若是使用OpenRouter的 LLM大模型，则要修改model_name、api_url、api_key。如下图示例：

二、配置LAM数字人

1. 复制配置文件chat_with_openai_compatible_bailian_cosyvoice.yaml到同目录内，重命名为lam.yaml
2. 编辑lam.yaml文件，填写CosyVoice apikey和LLM大语言模型apikey，与上文配置LiteAvatar数字人相同
3. 把RtcClient删除，替换为LamClient

LamClient:
   module:client/h5_rendering_client/client_handler_lam
   asset_path:"lam_samples/image.zip"
   concurrent_limit:5

参考下图：

4.把LiteAvatar删除，替换为LAM_Driver

LAM_Driver:
   module:avatar/lam/avatar_handler_lam_audio2expression

参考下图：

使用EDGE-TTS语言合成

1. 编辑yaml配置文件
2. 把CosyVoice配置项删除，或者把CosyVoice配置项的enabled改为False
3. 把下面的Edge_TTS配置项添加到配置文件中去

Edge_TTS:
    enabled: True
    module: tts/edgetts/tts_handler_edgetts
    voice: "zh-CN-XiaoxiaoNeural"

运行数字人

运行 LiteAvatar版本数字人

pythonsrc/demo.py--configconfig/liteavatar.yaml

运行 LAM 版数字人

pythonsrc/demo.py--configconfig/lam.yaml