Model Farm User Guide

Introduction

To accelerate the performance evaluation of AI models on target edge hardware platforms, Aplux has built the Model Farm platform. Model Farm contains hundreds of mainstream open-source models across various functionalities, optimized and adapted for different hardware platforms, and provides measured benchmark performance metrics for reference. Developers can quickly complete evaluations based on actual requirements without investing significant costs or enduring long waiting periods.

At the same time, Model Farm also provides ready-to-run model inference example code, which greatly reduces the difficulty and workload for developers testing model performance and developing AI functional code on existing hardware devices, shortens the overall process time, and accelerates solution deployment.

Features

Specifically, Model Farm helps developers accomplish the following:

• Query the AI performance of a specific AI model on specific Qualcomm hardware.
• Download adapted and optimized AI models (capable of invoking NPU hardware compute units for inference acceleration).
• Download the corresponding pre/post-processing and model inference example code.
• View the model adaptation and optimization steps, serving as a reference for developers to adapt and optimize their own fine-tuned models.

Model Farm Model Testing Workflow

User Fine-tuned Model Testing Workflow

Model Management Service (MMS)

Aplux provides the Model Management Service (MMS) component, allowing developers to access Model Farm without using a web browser. Through this service, model information and related file/code packages can be obtained directly on edge devices.

System Dependency Configuration

Configure the AidLux Repository

# Download the correct public key
sudo wget -O- https://archive.aidlux.com/ubuntu24/public.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/private-aidlux.gpg > /dev/null

# Edit the source file
sudo vim /etc/apt/sources.list.d/private-aidlux.list

# Add the private key provided by AidLux to the source file
deb [arch=arm64 signed-by=/etc/apt/trusted.gpg.d/private-aidlux.gpg] https://archive.aidlux.com/ubuntu24 noble main

# Update the cache
sudo apt update

After the update, you can obtain the official AidLux SDK dependencies with the following command:

sudo apt list | grep aid | grep unknown

# Install software
# Must be installed first; not included with the system
sudo apt install python3 python3-pip libopencv-dev python3-opencv  net-tools
# Required before installing aidlite
sudo apt install aidlux-aistack-base aidrtcm

# Install aidlite and its dependencies
sudo apt install aid-lms aidlms-sdk aidlite-sdk cmake
sudo apt-get install libfmt-dev nlohmann-json3-dev
sudo apt install aidlite-*

# Enable DSP support
sudo apt-get install qcom-fastrpc1
sudo apt-get install qcom-fastrpc-dev

# Install the aidgen SDK
sudo apt install aidgen-qnn240-sdk

# Install the mms service
sudo apt install aid-mms

# Enable GPU support
sudo add-apt-repository ppa:ubuntu-qcom-iot/qcom-noble-ppa
sudo apt install qcom-adreno-cl1
sudo ln -s /usr/lib/aarch64-linux-gnu/libOpenCL.so.1 /usr/lib/aarch64-linux-gnu/libOpenCL.so

After installation, check that the aidlite and aidgen directories have been added under /usr/local/share.

Device Authorization

Obtain the Device SN Code

cat  /sys/devices/soc0/serial_number

Obtain the License File

Provide the SN code to the Aplux technical staff to generate a device-specific License file, and place it under /etc/opt/aidlux/license/AidLuxLics.

Quick Start

The process for a developer to complete AI model inference on an edge device via Model Farm involves the following steps:

In addition, we provide a concrete example to help developers get started with Model Farm more quickly: Deploying the YOLOv5s model to a Qualcomm QCS8550 device via Model Farm

Prerequisites

Prepare the Development Board

The models provided by Model Farm are adapted and optimized for the Qualcomm Dragonwing IoT chip platform and have been performance-tested on development boards using Qualcomm chips. Model Farm currently supports the following Qualcomm chip models:

• Qualcomm QCS6490
• Qualcomm QCS8550
• IQ9075

Prepare a Developer Account

Developers can browse model information and performance metrics on Model Farm without logging in.

When developers download models and example code, they need to log in with a developer account to download.

• Register as an Aplux developer
  • Please visit: Developer Account Registration
  • Follow the registration form prompts and requirements to fill in the relevant developer information
  • After ensuring the information is correct, submit the account creation request

Log in to Model Farm

Web Login

Visit Model Farm

MMS Login

Aplux board users can also log in to Model Farm through the Model Management Service (MMS) component built into the AidLux SDK, as follows:

# Log in to Model Farm
$ mms login -u <developer-account-username> -p <developer-account-password>

A successful login will display:

Login successfully.

View Models

Developers can search for models on Model Farm according to their needs, understand detailed model information, and make quick evaluations.

Method 1: View via the Web

Developers can browse and view model details by visiting Model Farm in a browser.

Model Farm provides multiple ways to filter and search for models:

• Filter by model type
• Filter by model data precision
• Filter by chip platform
• Keyword search

Method 2: View via MMS

Aplux board users can also view model information on Model Farm through the Model Management Service (MMS) component built into the AidLux SDK.

Example of using MMS to search for models:

# List all models
$ mms list

# Search for a model by name
$ mms list yolo
Model        Precision  Chipset           Backend
-----        ---------  -------           -------
YOLO-NAS-l   FP16       Qualcomm QCS8550  QNN2.29
YOLO-NAS-l   INT8       Qualcomm QCS6490  QNN2.29
YOLO-NAS-l   INT8       Qualcomm QCS8550  QNN2.29
YOLO-NAS-l   W8A16      Qualcomm QCS6490  QNN2.29
YOLO-NAS-l   W8A16      Qualcomm QCS8550  QNN2.29
YOLO-NAS-m   FP16       Qualcomm QCS8550  QNN2.29
YOLO-NAS-m   INT8       Qualcomm QCS6490  QNN2.29
YOLO-NAS-m   INT8       Qualcomm QCS8550  QNN2.29

Download Models

Method 1: Download via the Web

On the model details page, click the corresponding button under Performance Reference to download the model and code package.

Method 2: Download via MMS

After developers successfully log in to Model Farm using the mms login command, they can use the mms get command to download models and example code:

# Download the yolov6l model with INT8 precision, optimized for the QCS8550 chipset, using the QNN2.23 inference framework.
$ mms get -m yolov6l -p int8 -c qcs8550 -b qnn2.23
Downloading model from https://aiot.aidlux.com to directory: /var/opt/modelfarm_models

Downloading [yolov6l_qcs8550_qnn2.23_int8_aidlite.zip] ... done! [40.45MB in 375ms; 81.51MB/s]

Download complete!

For detailed usage, refer to the output of the mms get -h command.

Model Performance Reference

The model details page of Model Farm provides the measured performance of AI models on the corresponding hardware.

• Device: The development board model and corresponding chip model used for the measurement.
• AI Framework: The framework and version used for model conversion and inference.
• Data Precision: The data precision of the converted model.
• Inference Latency: The measured execution time of the model, excluding pre- and post-processing.
• Accuracy Loss: The cosine similarity of the output matrices between the source model (FP32) and the converted model.
• Model Size: The file size of the converted model.

💡Note

For the same SoC, model performance may vary across devices with different hardware specifications; these figures serve as a reference only.

Take YOLOv5s on MeiG SNM972 (QCS8550) as an example:

Test Models

Models downloaded from Model Farm can be tested for inference in the following two ways:

Use APLUX AidLite for Model Inference

Aplux provides the AI inference framework AidLite for invoking the Qualcomm NPU to run AI model inference on edge devices.

All models supported by Model Farm can be inferred through the AidLite SDK. In addition, Model Farm provides the pre/post-processing code for the corresponding models, ensuring developers can quickly preview the model's effect.

Through the Download Models step, developers can obtain a complete model file + inference code package, whose file structure is shown below (using YOLOv5s as an example):

/model_farm_yolov5s
    
    |__ models # Model files

    |__ python # Python-based model inference code

    |__ cpp # C++-based model inference code

    |__ README.md # Model information & installation guide for related software dependencies

Running the model test case involves the following steps:

• Activate the development board environment: refer to Prerequisites for the activation method.
• Install AidLite and other software environment dependencies: refer to README.md.
• Run the example: refer to README.md.

Use Qualcomm QNN for Model Inference

Please refer to the Qualcomm QNN documentation

Convert Fine-tuned Models

Aplux provides the Model Optimization Platform AIMO for converting models to a format specific to the Qualcomm platform.

Almost all models supported by Model Farm can be converted using AIMO. Based on this, Model Farm not only provides converted and optimized model files, but also provides reference steps for converting the model with AIMO.

The AIMO model conversion reference steps can be found in the following two places:

• In the Performance Reference module on the right side of the model details page; click Model Conversion Reference to access it.
• In the Model Conversion Reference module of the README.md file in the code package; click the link to access it.

For an introduction and usage of AIMO, please refer to: Model Optimization Platform AIMO User Guide

Complete Example

YOLOv5s Model Deployment

This example demonstrates how to deploy the YOLOv5s model to a Qualcomm QCS8550 device via Model Farm.

Step 1: Visit the Website and Search for the Model

Developers can access Model Farm directly through a web browser: https://aiot.aidlux.com/zh

Search for the YOLOv5s model directly in the search bar to find YOLOv5s versions with different quantization precisions. In this case, we select the INT8 quantization precision and click the YOLOv5s-INT8 model card to view details.

Step 2: View Model Performance

The left side of the model details page shows the model introduction and related tool descriptions, while the floating window on the right shows the reference performance of the model on the corresponding hardware.

We select the MeiG SNM972 device (QCS8550) as the reference hardware and can see the performance metrics of the YOLOv5s INT8 precision on this hardware, as shown below:

💡Note

The inference latency metric refers to the model execution time, excluding the pre- and post-processing code.

Step 3: Download the Model and Code Package

To obtain the model files and inference code, developers need to register a Model Farm developer account and log in. For specific operations, please refer to: Prepare a Developer Account and Log in to Model Farm

After logging in with the developer account, click the Model & Code button to download the model files and code package.

At the same time, we can view the file structure and description of the code package in the Inference Example Code module on the model details page, as shown below:

Step 4: Import the Model & Code Package and Install Software Dependencies

Once the hardware development board is ready, developers can import the downloaded code package into the development board environment. For environment preparation, please refer to the Prerequisites section.

Developers need to install the software dependencies for model inference by referring to the README.md file in the code package. In this case, we only need to install the AidLite SDK (QNN2.16 version). For an introduction and usage of the AidLite SDK, please refer to: AI inference framework AidLite

Note

The AidLite SDK has multiple versions for QNN. You can check the QNN version in the Performance Reference of the model details, or check the QNN version in the README.

sudo aid-pkg update
sudo aid-pkg install aidlite-sdk

# Install AidLite QNN2.16 version
sudo aid-pkg install aidlite-qnn216

After installation, verify whether the AidLite SDK was installed successfully:

# AidLite SDK C++ version check
python3 -c "import aidlite; print(aidlite.get_library_version())"

# AidLite SDK Python version check
python3 -c "import aidlite; print(aidlite.get_py_library_version())"

Step 5: Run the Model Test Case

After the software dependencies are installed, refer to the running steps in the README.md file. Developers can run the YOLOv5s case in the python and cpp folders of the code package using different programming languages.

Step 6: Convert Your Custom Fine-tuned Model and Run Inference

When developers want to test the performance of their own fine-tuned YOLOv5s model on the QCS8550, Model Farm provides reference steps for converting the YOLOv5s model using AIMO.

The AIMO model conversion reference steps can be found in the following two places:

• In the Performance Reference module on the right side of the model details page; click Model Conversion Reference to access it.
• In the Model Conversion Reference module of the README.md file in the code package; click the link to access it.

Developers can use these steps to convert their own YOLOv5s model with AIMO. For an introduction and usage of AIMO, please refer to:

Model Optimization Platform AIMO User Guide

After the model conversion is complete, developers can replace the YOLOv5s model file in the code package and run the test case again to see the results.

💡Note

There may be differences between models. The test case cannot guarantee successful inference by only replacing the model. Developers need to adjust the pre/post-processing code for their own model.

Accessing Preview Model Resources

Models in the Preview section of the Model Farm do not support direct web downloads. Developers can download preview models on Aplux boards using the mms command. The following example demonstrates how to retrieve the MobileClip2-S3 model.

1. Log in via mms

mms login

# Enter your username: 
# Enter your password:

# A message will appear after entering the correct credentials:
# Login successfully.

2. Search for the model

mms list mobileclip # Supports keyword search

# Expected output:
Model           Precision  Chipset           Backend
-----           ---------  -------           -------
MobileClip-S2   FP16       Qualcomm QCS8550  QNN2.31
MobileClip2-S3  FP16       Qualcomm QCS8550  QNN2.36

3. Download MobileClip2-S3

# -m: Model name
# -p: Precision
# -c: Chipset
# -b: QNN version
mms get -m MobileClip2-S3 -p fp16 -c qcs8550 -b qnn2.36

# Model resources are downloaded to /var/opt/modelfarm_models by default.