YOLOv5 Deployment

Introduction

YOLOv5 is a single-stage object detection network framework. Its main structure consists of four parts: a network backbone composed of a modified CSPNet, a high-resolution feature fusion module composed of an FPN (Feature Pyramid Network), a pooling module composed of SPP (Spatial Pyramid Pooling), and three different detection heads used to detect objects of various sizes.

This chapter demonstrates how to complete the deployment, loading, and recognition process of YOLOv5s on edge devices. Two deployment methods are provided:

• AidLite Python API
• AidLite C++ API

In this case, model inference runs on the device-side NPU computing unit. The code calls the relevant interfaces to receive user input and return results.

• Device: IQ9075
• System: Ubuntu 24.04
• Source Model: YOLOv5s
• Quantized Model Precision: INT8 quantization
• Model Farm Model Reference: YOLOv5s-INT8

Supported Platforms

Platform	Running Method
IQ9075	Ubuntu 24.04

Prerequisites

1. IQ9075 hardware

2. Ubuntu 24.04 system

System Dependency Configuration

Configure the AidLux Repository

# Download the correct public key
sudo wget -O- https://archive.aidlux.com/ubuntu24/public.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/private-aidlux.gpg > /dev/null

# Edit the source file
sudo vim /etc/apt/sources.list.d/private-aidlux.list

# Add the private key provided by AidLux to the source file
deb [arch=arm64 signed-by=/etc/apt/trusted.gpg.d/private-aidlux.gpg] https://archive.aidlux.com/ubuntu24 noble main

# Update the cache
sudo apt update

After the update, you can obtain the official AidLux SDK dependencies with the following command:

sudo apt list | grep aid | grep unknown

# Install software
# Must be installed first; not included with the system
sudo apt install python3 python3-pip libopencv-dev python3-opencv  net-tools
# Required before installing aidlite
sudo apt install aidlux-aistack-base aidrtcm

# Install aidlite and its dependencies
sudo apt install aid-lms aidlms-sdk aidlite-sdk cmake
sudo apt-get install libfmt-dev nlohmann-json3-dev
sudo apt install aidlite-*

# Enable DSP support
sudo apt-get install qcom-fastrpc1
sudo apt-get install qcom-fastrpc-dev

# Install the aidgen SDK
sudo apt install aidgen-qnn240-sdk

# Install the mms service
sudo apt install aid-mms

# Enable GPU support
sudo apt-add-repository -s ppa:ubuntu-qcom-iot/qcom-ppa
sudo apt install qcom-adreno-cl1
sudo ln -s /usr/lib/aarch64-linux-gnu/libOpenCL.so.1 /usr/lib/aarch64-linux-gnu/libOpenCL.so

After installation, check that the aidlite and aidgen directories have been added under /usr/local/share.

Device Authorization

Obtain the Device SN Code

cat  /sys/devices/soc0/serial_number

Obtain the License File

Provide the SN code to the Aplux technical staff to generate a device-specific License file, and place it under /etc/opt/aidlux/license/AidLuxLics.

Activate Authorization

sudo /opt/aidlux/cpf/aid-lms/manager.sh restart

Download YOLOv5s-INT8 Model Resources

mms list yolov5s

#------------------------ YOLOv5s models available ------------------------
Model        Precision  Chipset           Backend
-----        ---------  -------           -------
YOLOv5s      INT8       Qualcomm QCS6490  QNN2.31
YOLOv5s      INT8       Qualcomm QCS8550  QNN2.31
YOLOv5s      FP16       Qualcomm QCS8550  QNN2.31
YOLOv5s      W8A16      Qualcomm QCS6490  QNN2.31
YOLOv5s      W8A16      Qualcomm QCS8550  QNN2.31
YOLOv5s-seg  INT8       Qualcomm QCS8550  QNN2.16
YOLOv5s-seg  INT8       Qualcomm QCS6490  QNN2.31

# Download YOLOv5s-int8
mms get -m YOLOv5s -p int8 -c qcs8550 -b qnn2.31 -d /home/ubuntu/yolov5s
cd /home/ubuntu/yolov5s
# Unzip
unzip YOLOv5s_qcs8550_w8a8.zip

💡Note

Developers can also search for and download models on the Model Farm website.

AidLite SDK Installation

Developers can also refer to the README.md in the model folder to install the SDK.

• Ensure the QNN backend version is ≥ 2.31
• Ensure the versions of aidlite-sdk and aidlite-qnnxxx are 2.3.x

# Check AidLite & QNN versions
dpkg -l | grep aidlite
#------------------------ You should see output similar to the following ------------------------
ii  aidlite-qnn236       2.3.0.230         arm64        aidlux aidlite qnn236 backend plugin
ii  aidlite-sdk          2.3.0.230         arm64        aidlux inference module sdk

QNN & AidLite Version Update

# Install AidLite SDK
sudo apt update
sudo apt install aidlite-sdk
sudo apt install aidlite-qnn236

# AidLite SDK C++ version check
python3 -c "import aidlite; print(aidlite.get_library_version())"

# AidLite SDK Python version check
python3 -c "import aidlite; print(aidlite.get_py_library_version())"

AidLite Python API Deployment

Run the Python API example

cd /home/ubuntu/yolov5s/code

# --target_model: Path to the model file
# --imgs: Input image
# --invoke_nums: Number of loops
python3  python/run_test.py --target_model /home/ubuntu/yolov5s/models/QCS8550/W8A8/cutoff_yolov5s_qcs8550_w8a8.qnn231.ctx.bin --imgs ./python/bus.jpg --invoke_nums 10

You can see the model inference time (in ms) and the detection results in the terminal:

=======================================
QNN inference 10 times :
 --mean_invoke_time is 1.5697240829467773 
 --max_invoke_time is 2.2764205932617188 
 --min_invoke_time is 1.477956771850586 
 --var_invoketime is 0.05570707855895307
=======================================
Detected 5 regions
1 [668, 385, 141, 500] 0.86635846 person
2 [219, 407, 125, 465] 0.86257815 person
3 [55, 393, 178, 515] 0.845101 person
4 [3, 207, 812, 601] 0.8401416 bus
5 [0, 551, 73, 325] 0.5058796 person
Image saved at ./python/yolov5s_result.jpg
=======================================

AidLite C++ API Deployment

Run the C++ API example

cd /home/ubuntu/yolov5s/code/cpp

mkdir build && cd build
cmake ..
make
./run_yolov5

You can see the model inference time (in ms) and the detection results in the terminal:

The C++ code performs 10 inference loops by default.

current thread_idx[1] [9] get_output_tensor cost time : 0.911757
repeat [10] time , input[21.061591] --- invoke[16.763899] --- output[16.023339] --- sum[53.848829]ms
postprocess cost time : 0.158227 ms
Result id[0]-x1[209.905304]-y1[242.500031]-x2[284.438965]-y2[518.384033]
Verify result : idx[0] id[0] coverage_ratio[0.983873]
Result id[0]-x1[105.769806]-y1[228.641464]-x2[234.684357]-y2[546.474487]
Verify result : idx[0] id[0] coverage_ratio[0.129211]
Verify result : idx[1] id[0] coverage_ratio[0.000000]
Verify result : idx[2] id[0] coverage_ratio[0.934018]
Result id[0]-x1[474.906281]-y1[228.192184]-x2[558.729797]-y2[524.893982]
Verify result : idx[0] id[0] coverage_ratio[0.000000]
Verify result : idx[1] id[0] coverage_ratio[0.952345]
Result id[5]-x1[81.684174]-y1[122.895874]-x2[562.999390]-y2[479.283447]
Verify result : idx[0] id[5] coverage_ratio[0.893258]

The result image result.jpg is saved in the build folder.