Edge Deployment of Qwen3.5 Series

Introduction

In recent months, Qwen3.5 has stepped up its investment in developing highly practical and high-performance foundation models. Qwen3.5 represents a major leap forward, combining breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility, giving developers and enterprises unprecedented capabilities and efficiency.

This chapter demonstrates how to perform deployment, loading, and conversation with the Qwen3.5 series models on edge devices. The following deployment method is provided:

• AidGen C++ API

In this case, the large language model inference runs on the device side, and the relevant interfaces are called through code to receive user input and return conversation results in real time.

• Device: Rhino Pi-X1
• System: Ubuntu 22.04
• Model: Qwen3.5-4B

Supported Platforms

Platform	Running Method
Rhino Pi-X1	Ubuntu 22.04, AidLux

Preparation

1. Rhino Pi-X1 hardware

2. Ubuntu 22.04 system or AidLux system

AidGen Case Deployment

Step 1: Install the AidGen SDK

# Install AidGen SDK
sudo aid-pkg update
sudo aid-pkg -i aidgen-sdk
sudo aid-pkg -i aidgen-qnn240
sudo aid-pkg -i aidgen-qnn248

# Copy test code
cd /home/aidlux/aidmlm

cp -r /usr/local/share/aidgen/examples/ ./

💡Note

Make sure that the versions of aidgen-sdk and aidgen-qnn248 match, and that the aidgen-sdk version is >= 2.4.4.*

Step 2: Download Model Resources

Since Qwen3.5-4B is currently in the Model Farm preview section, it must be retrieved via the mms command.

# Log in
mms login

# Search for the model
mms list Qwen3.5

# Download the model
mms get -m "Qwen3.5-4B" -p w4a16 -c qcs8550 -b qnn2.48 -d /home/aidlux/aidmlm/

cd /home/aidlux/aidmlm/
unzip qnn248_qcs8550_cl8192.zip

Step 3: Confirm Resource Files

The file distribution is as follows:

/home/aidlux/aidmlm/qnn248_qcs8550_cl8192
├── aidgen_config.json
├── examples
├── position_ids_cos.rawposition_ids sin.raw
├── ingredients.png
├── qwen3.5-4b_qnn248_qcs8550_cl18192_1_of_3.serialized.bin.aidem
├── qwen3.5-4b_qnn248_qcs8550_c18192_2_of_3.serialized.bin.aidem
├── embedding_fp32.bin
├── qwen3.5-4b_qnn248_qcs8550_c18192_3_of_3.serialized.bin.aidem
├── veg.serialized.bin.aidem

Step 4: Compile and Run

cd /home/aidlux/aidmlm/examples

# Replace the example files
cp ../qnn248_qcs8550_cl8192/examples/* ./

# Compile
mkdir build && cd build
cmake .. && make

# Copy the compiled example program to the model directory
cp ./test_qwen35vl ../../qnn248_qcs8550_cl8192
cd /home/aidlux/aidmlm/qnn248_qcs8550_cl8192

# Run the example program
./test_qwen35vl qnn248

• After the model runs successfully, the following log output is displayed:

Image-text embedding input
This image shows a fresh, colorful assortment of ingredients arranged on a dark background.

- **Center**: Two slices of raw sea bream (or similar white-fleshed fish) sit on a round wooden cutting board.
- **Surrounded by various vegetables and fruits**:
  - The upper-left corner shows fresh green cilantro and a few pieces of ginger.
  - On the left there are two dark-green cucumbers, three green tomatoes, and four orange carrots.
  - The upper-right corner has a cluster of small red tomatoes.
  - The lower-right corner contains an avocado and a pomegranate, along with a bunch of leafy greens (likely cabbage or lettuce).
  - In the bottom center there are two yellow lemons.

The overall composition is full and layered, with bright, vivid colors, giving a healthy, natural feel and is often used to showcase ingredient variety or healthy-eating themes.<|im_end|>