链载Ai

标题: 树莓派AI套件：如何将混乱的数据变成有序的魔法 [打印本页]

作者: 链载Ai 时间: 昨天 11:50
标题: 树莓派AI套件：如何将混乱的数据变成有序的魔法

使用 Raspberry PiAI 套件进行非结构化数据处理——Hailo边缘AI

非结构化数据处理、Raspberry Pi 5、Raspberry Pi AI套件、Milvus、Zilliz、数据、图像、计算机视觉、深度学习、Python

在边缘实时相机流中检测、显示和存储检测到的图像

即使预算有限，你也可以利用像配备8GB内存的Raspberry Pi 5和NVIDIA Jetson Orin Nano这样的强大设备，开始构建一些边缘AI用例。最近，针对RPI5平台的Raspberry Pi AI套件发布了，所以我必须入手一个并尝试一下。

AI套件增加了一个神经网络推理加速器，其性能可达每秒13万亿次操作（TOPS），对于70美元的价格来说，这相当不错。连接到这个M.2 Hat的是Hailo-8L M.2入门级加速模块，它将为我们提供AI能力。

在第一次演示中，我修改了提供的RPI5 Hailo AI Python示例之一，以对网络摄像头进行实时图像检测，然后将检测结果发送到Slack频道，更重要的是，将检测结果与元数据一起向量化并存储到Milvus中。

在Raspberry Pi 5上实时运行

我们使用了Hailo提供的RPI5对象检测程序示例，并对其进行了增强，以便将结果发送到Slack、MiNio和Milvus。

因此，我们使用了示例对象检测程序，但首先我为Slack、Milvus、S3、TIMM、Sci-Kit Learn、Pytorch和UUID库添加了一些导入语句。我还设置了一些稍后要使用的常量。然后，我们连接到我们的Milvus服务器和Slack频道，并开始GStreamer循环。我设置了一个时间检查，如果检测到内容，我会将相机帧保存到文件中，然后上传到S3并发送到我的Slack频道。最后，我添加了向量化的图像以及S3路径、文件名、标签和置信度等重要元数据。我们的集合中的每个条目都会获得一个自动生成的ID。

我们的图像已经上传到MinIO：

它们也已随我们的文本消息发送到我们的#reports Slack频道。

最重要的是，我们的元数据和向量已经上传，并且已经可以用于超快速搜索。

现在我们可以开始查询我们的向量，我会通过Jupyter笔记本向你展示如何操作

查询数据库并显示图像

加载

In [1]:

!pipinstallboto3

Requirementalreadysatisfied:boto3in./milvusvenv/lib/python3.12/site-packages(1.34.129)
Requirementalreadysatisfied:botocore<1.35.0,>=1.34.129in./milvusvenv/lib/python3.12/site-packages(fromboto3)(1.34.129)
Requirementalreadysatisfied:jmespath<2.0.0,>=0.7.1in./milvusvenv/lib/python3.12/site-packages(fromboto3)(1.0.1)
Requirementalreadysatisfied:s3transfer<0.11.0,>=0.10.0in./milvusvenv/lib/python3.12/site-packages(fromboto3)(0.10.1)
Requirementalreadysatisfied:python-dateutil<3.0.0,>=2.1in./milvusvenv/lib/python3.12/site-packages(frombotocore<1.35.0,>=1.34.129->boto3)(2.9.0.post0)
Requirementalreadysatisfied:urllib3!=2.2.0,<3,>=1.25.4in./milvusvenv/lib/python3.12/site-packages(frombotocore<1.35.0,>=1.34.129->boto3)(2.2.1)
Requirementalreadysatisfied:six>=1.5in./milvusvenv/lib/python3.12/site-packages(frompython-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.129->boto3)(1.16.0)

In [5]:

from__future__importprint_functionimportrequestsimportsysimportioimportjsonimportshutilimportsysimportdatetimeimportsubprocessimportsysimportosimportmathimportbase64fromtimeimportgmtime,strftimeimportrandom,stringimporttimeimportpsutilimportbase64importuuidimportsocketimportosfrompymilvusimportconnectionsfrompymilvusimportutilityfrompymilvusimportFieldSchema,CollectionSchema,DataType,CollectionimporttorchfromtorchvisionimporttransformsfromPILimportImageimporttimmfromsklearn.preprocessingimportnormalizefromtimm.dataimportresolve_data_configfromtimm.data.transforms_factoryimportcreate_transformfrompymilvusimportMilvusClientimportosfromIPython.displayimportdisplay

In [6]:

from__future__importprint_functionimportrequestsimportsysimportioimportjsonimportshutilimportsysimportdatetimeimportsubprocessimportsysimportosimportmathimportbase64fromtimeimportgmtime,strftimeimportrandom,stringimporttimeimportpsutilimportbase64importuuidimportsocketimportosfrompymilvusimportconnectionsfrompymilvusimportutilityfrompymilvusimportFieldSchema,CollectionSchema,DataType,CollectionimporttorchfromtorchvisionimporttransformsfromPILimportImageimporttimmfromsklearn.preprocessingimportnormalizefromtimm.dataimportresolve_data_configfromtimm.data.transforms_factoryimportcreate_transformfrompymilvusimportMilvusClientimportosfromIPython.displayimportdisplay

In [8]:

# -----------------------------------------------------------------------------
class FeatureExtractor:def __init__(self, modelname):# Load the pre-trained modelself.model = timm.create_model(modelname, pretrained=True, num_classes=0, global_pool="avg")self.model.eval()
# Get the input size required by the modelself.input_size = self.model.default_cfg["input_size"]
config = resolve_data_config({}, model=modelname)# Get the preprocessing function provided by TIMM for the modelself.preprocess = create_transform(**config)
def __call__(self, imagepath):# Preprocess the input imageinput_image = Image.open(imagepath).convert("RGB")# Convert to RGB if neededinput_image = self.preprocess(input_image)
# Convert the image to a PyTorch tensor and add a batch dimensioninput_tensor = input_image.unsqueeze(0)
# Perform inferencewith torch.no_grad():output = self.model(input_tensor)
# Extract the feature vectorfeature_vector = output.squeeze().numpy()
return normalize(feature_vector.reshape(1, -1), norm="l2").flatten()

In [9]:

extractor = FeatureExtractor("resnet34")
# -----------------------------------------------------------------------------# Constants - should be environment variables# -----------------------------------------------------------------------------DIMENSION = 512 MILVUS_URL = "http://192.168.1.163:19530" COLLECTION_NAME = "pidetections"BUCKET_NAME = "images"DOWNLOAD_DIR = "/Users/timothyspann/Downloads/code/images/"AWS_RESOURCE = "s3"S3_ENDPOINT_URL = "http://192.168.1.163:9000"AWS_ACCESS_KEY = "minioadmin" AWS_SECRET_ACCESS_KEY = "minioadmin"S3_SIGNATURE_VERSION = "s3v4"AWS_REGION_NAME = "us-east-1"S3_ERROR_MESSAGE = "Download failed"# -----------------------------------------------------------------------------

In [10]:

# -----------------------------------------------------------------------------# Connect to Milvus
# Local Docker Servermilvus_client = MilvusClient( uri=MILVUS_URL)# -----------------------------------------------------------------------------

In [12]:

import osimport boto3from botocore.client import Config
# -----------------------------------------------------------------------------# Access Images on S3 Compatible Store - AWS S3 or Minio or ...# -----------------------------------------------------------------------------s3 = boto3.resource(AWS_RESOURCE,endpoint_url=S3_ENDPOINT_URL,aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_SECRET_ACCESS_KEY,config=Config(signature_version=S3_SIGNATURE_VERSION),region_name=AWS_REGION_NAME)
bucket = s3.Bucket(BUCKET_NAME)
# -----------------------------------------------------------------------------# Get last modified image# -----------------------------------------------------------------------------files = bucket.objects.filter()files = [obj.key for obj in sorted(files, key=lambda x: x.last_modified, reverse=True)]
for imagename in files:query_image = imagenamebreak
search_image_name = DOWNLOAD_DIR + query_image
try:s3.Bucket(BUCKET_NAME).download_file(query_image, search_image_name)except botocore.exceptions.ClientError as e:print(S3_ERROR_MESSAGE)
# -----------------------------------------------------------------------------# Search Milvus for that vector and filter by a label# -----------------------------------------------------------------------------results = milvus_client.search(COLLECTION_NAME,data=[extractor(search_image_name)],filter='label in ["keyboard"]',output_fields=["label", "confidence", "id", "s3path", "filename"],search_params={"metric_type": "COSINE"},limit=5)
# -----------------------------------------------------------------------------# Iterate through last five results and display metadata and image# -----------------------------------------------------------------------------for result in results:for hit in result[:5]:label = hit["entity"]["label"]confidence = hit["entity"]["confidence"]filename = hit["entity"]["filename"]s3path = hit["entity"]["s3path"]try:s3.Bucket(BUCKET_NAME).download_file(filename, DOWNLOAD_DIR + filename)except botocore.exceptions.ClientError as e:print(S3_ERROR_MESSAGE)print(f"Detection: {label} {confidence:.2f} for {filename} from {s3path}" )img = Image.open(DOWNLOAD_DIR + filename)display(img) # Enhancement:we could also post this to slack or discord

详细代码请查看GitHub：

https://gist.github.com/tspannhw/8e2ec1293c1cff1edaefbf7fde54f47a#file-edgeaifind-ipynb

我已经录制了这个演示的运行过程，所以你可以看到实时的操作情况。

如果你购买了一个并想设置它以复制我的演示，请参阅本文末尾的步骤。

演示打包清单

MinIO/S3、Milvus、Slack、Python、Boto3、OpenCV2、Pytorch、Sci-Kit Learn、TIMM、Hailo、YOLOv6n、对象检测、Raspberry PiAI套件、配备8GB内存的Raspberry Pi5、logi网络摄像头、resnet34、Torchvision、PyMilvus、Hailo8L M.2模块、M.2 M-Key Hat、散热片。

入门指南

添加硬件（请参阅下面的视频和链接）后，安装库，重启，然后你就应该准备好了。

tspann@five:/opt/demo $ hailortcli fw-control identify
Executing on device: 0000:01:00.0Identifying boardControl Protocol Version: 2Firmware Version: 4.17.0 (release,app,extended context switch buffer)Logger Version: 0Board Name: Hailo-8Device Architecture: HAILO8LSerial Number: HLDDLBB241601635Part Number: HM21LB1C2LAEProduct Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP
tspann@five:/opt/demo $ dmesg | grep -i hailo
[3.155152] hailo: Init module. driver version 4.17.0[3.155295] hailo 0000:01:00.0: Probing on: 1e60:2864...[3.155301] hailo 0000:01:00.0: Probing: Allocate memory for device extension, 11600[3.155321] hailo 0000:01:00.0: enabling device (0000 -> 0002)[3.155327] hailo 0000:01:00.0: Probing: Device enabled[3.155350] hailo 0000:01:00.0: Probing: mapped bar 0 - 0000000095e362ea 16384[3.155357] hailo 0000:01:00.0: Probing: mapped bar 2 - 000000005e2b2b7e 4096[3.155362] hailo 0000:01:00.0: Probing: mapped bar 4 - 000000008db50d03 16384[3.155365] hailo 0000:01:00.0: Probing: Force setting max_desc_page_size to 4096 (recommended value is 16384)[3.155375] hailo 0000:01:00.0: Probing: Enabled 64 bit dma[3.155378] hailo 0000:01:00.0: Probing: Using userspace allocated vdma buffers[3.155382] hailo 0000:01:00.0: Disabling ASPM L0s[3.155385] hailo 0000:01:00.0: Successfully disabled ASPM L0s[3.417111] hailo 0000:01:00.0: Firmware was loaded successfully[3.427885] hailo 0000:01:00.0: Probing: Added board 1e60-2864, /dev/hailo0

示例代码

https://github.com/hailo-ai/hailo-rpi5-examples?source=post_page-----c959dd7fff47--------------------------------

动物园模型

https://github.com/hailo-ai/hailo_model_zoo/tree/master/docs/public_models/HAILO8L?source=post_page-----c959dd7fff47--------------------------------

额外指令

gst-inspect-1.0hailotoolslspci|grepHailouname-av4l2-ctl--list-formats-ext-d/dev/video0ls/dev/video*ffplay-fv4l2/dev/video0

欢迎光临链载Ai (https://www.lianzai.com/)