AidGenSE Manual Model Upload and Configuration

Introduction

This document details how to upload models and use AidGenSE to start OpenAI HTTP services.

Prerequisites

Before uploading and loading models, please ensure the following conditions are met:

• Install AidGenSE related dependencies, please refer to AidGenSE Installation
• Complete model folder (including .gguf, .bin, or .aidem model files and configuration files)
• Understand the model types and limitations supported by the current platform, please refer to AidGenSE Support Status

Model Resource Preparation

GGUF Format Models

Recommended file structure:

qwen2.5-1.5b-instruct-q4_k_m/

|__ qwen2.5-1.5b-instruct-q4_k_m.gguf

|__ qwen2.5-1.5b-instruct-q4_k_m.json   ← Model configuration file (needs manual creation)

Configuration File Template:

LLM Model:

{
  "backend_type": "llamacpp",
  "model": {
    "path": "<absolute path to model file>"
  }
}

VLM Model (adds an mmproj field compared to the LLM):

{
  "backend_type": "llamacpp",
  "model": {
    "path": "<absolute path to model file>",
    "mmproj": "<absolute path to mmproj file>"
  }
}

💡Note

The mmproj field is optional and specifies the multimodal projector file path for VLM models.

Qualcomm QNN Bin Format Models

Recommended structure:

qwen2.5-1.5b-instruct-8550-bin/

|__ *.serialized.bin (multiple files)

|__ qwen2.5-1.5b-instruct-htp.json    ← Main configuration file

|__ qwen2.5-1.5b-instruct-tokenizer.json

|__ htp_backend_ext_config.json

💡Note

The large model resources on Model Farm are organized according to standard structure. Information about Qualcomm large model configuration files can be found in Qualcomm Genie.

Special Notes

qwen2.5-1.5b-instruct-htp.json is the main configuration file, where related file paths need to be set as absolute paths. Example:

"tokenizer": {
  "path": "/opt/aidlux/app/aid-openai-api/res/models/qwen2.5-1.5B-instruct-8550-bin/qwen2.5-1.5b-instruct-tokenizer.json"
},
"ctx-bins": [
  "/opt/aidlux/app/aid-openai-api/res/models/qwen2.5-1.5B-instruct-8550-bin/qwen2.5-1.5b-instruct_qnn229_qcs8550_4096_1_of_3.serialized.bin",
  "/opt/aidlux/app/aid-openai-api/res/models/qwen2.5-1.5B-instruct-8550-bin/qwen2.5-1.5b-instruct_qnn229_qcs8550_4096_2_of_3.serialized.bin",
  "/opt/aidlux/app/aid-openai-api/res/models/qwen2.5-1.5B-instruct-8550-bin/qwen2.5-1.5b-instruct_qnn229_qcs8550_4096_3_of_3.serialized.bin"
],
"extensions": "/opt/aidlux/app/aid-openai-api/res/models/qwen2.5-1.5B-instruct-8550-bin/htp_backend_ext_config.json"

APLUX aidem Encrypted Format

Recommended file structure:

qwen2.5-1.5b-instruct-q4_k_m/

|__ *.aidem (multiple files)

|__ qwen2.5-1.5b-instruct.json   ← Model configuration file (needs manual creation)

Configuration File Template:

LLM Model:

{
  "backend_type": "genie",
  "model_type": "llm",
  "config_version": 5,
  "htp_config_path": "<absolute path to HTP config file, or empty>",
  "model": {
    "path": ["<absolute path to model file>"]
  }
}

VLM Model:

{
  "backend_type": "genie",
  "model_type": "vlm",
  "config_version": 5,
  "htp_config_path": "<absolute path to HTP config file, or empty>",
  "model": {
    "path": []
  },
  "vlm_model": {
    "type": "qwen2.5",
    "vision_model_path": "<absolute path to vision model file>",
    "pos_embed_cos_path": "<absolute path to position_ids_cos.raw>",
    "pos_embed_sin_path": "<absolute path to position_ids_sin.raw>",
    "vocab_embed_path": "<absolute path to embedding_weights.raw>",
    "window_attention_mask_path": "<absolute path to window_attention_mask.raw>",
    "full_attention_mask_path": "<absolute path to full_attention_mask.raw>",
    "llm_path_list": ["<absolute path to llm shard files>"],
    "qwen2.5": {
      "img_h": 336,
      "img_w": 504,
      "img_c": 3,
      "T": 2,
      "patch_size": 14,
      "temporal_patch_size": 2,
      "merge_size": 2
    }
  }
}

💡Note

The type field currently supports the following 4 types: qwen2.5, qwen3, locateanything, magevl.

Embedding Model:

{
  "backend_type": "genie",
  "model_type": "embedding",
  "config_version": 5,
  "htp_config_path": "<absolute path to HTP config file, or empty>",
  "embedding_model": {
    "path": ["<absolute path to model file>"]
  }
}

Configuration File Setup

Configure model resources by editing api_cfg.json

vi /opt/aidlux/app/aid-openai-api/api_cfg.json

Add model:

{
...
"prompt_template_list": [
    ...
    "<conversation template type>": "conversation template format"
],
"model_cfg_list": [
    ...
    {
        "model_id": "<model ID>",
        "model_create": "<timestamp>",
        "model_owner": "<creator>",
        "cfg_path": "<absolute path to model config file>",
        "prompt_template_type": "<conversation template type>"
    }
]
}

Field Descriptions:

• model_id: Model identifier, unique ID used to specify the model at runtime, e.g., qwen2.5-1.5b-instruct-q4_k_m
• model_create: Model registration timestamp, can be generated using command printf "%s%03d\n" "$(date +%s)" "$((10#$(date +%N) / 1000000))", in milliseconds
• model_owner: Model owner name, customizable, e.g., "aplux"
• cfg_path: Absolute path to main model configuration file
• prompt_template_type: Conversation prompt template type, common values like "qwen1.5", "qwen2", "deepseek", should match the model training format

View Added Models Using Commands

# Use aidllm list api command to check if the model has been added to aidllm
aidllm list api

qwen2.5-7b-instruct
aplux_qwen2-7B
qwen2.5-7b-instruct-q4_k_m
qwen2.5-7B-8550
qwen2.5-1.5b-instruct-q4_k_m
qwen2.5-1.5B-instruct-8550-bin

# Use aidllm start api -m qwen2.5-1.5b-instruct-q4_k_m to specify and run the model, check if the model can run
aidllm start api -m qwen2.5-1.5b-instruct-q4_k_m

This document applies to aidgense V3.3.1