AidLite Python API Reference

Use AutoModel

These steps apply to the Linux Python SDK. The current maintained source includes AutoModel. Use a package with matching core, Python extension, and backend components. An older published package may not include this interface. AutoModel provides one entry point for local backends and the Android inference service. An enum member or a recognized model format does not establish runtime support. Install a matching backend plugin and SDK. The backend must provide the required tensor APIs. A common entry point does not establish device compatibility for every model. AutoModel and InferenceSession let you select LOCAL or REMOTE. The REMOTE examples below require matching source builds with this feature. An older published package may not include this feature. Read REMOTE troubleshooting before use.

Prepare the model

1. Use a converted local model.
2. Make sure that the model backend matches the installed SDK version.
3. Prepare the input as specified by the original model.

New AIMO models can store input and output metadata in .aidem files. Raw QNN context files use a matching <model filename>.io.json file. Keep these files together. Do not convert all inputs to FP32. Metadata requires matching updated AIMO converter and model-packaging components. Existing model files do not update automatically.

The original model is the ONNX model supplied to the AIMO conversion stage. Metadata cannot restore an interface changed by an earlier exporter. The caller must still normalize images, tokenize text, and complete other required input preparation.

Call a model with one input

1. Read the original input array from a file.
2. Load the model.
3. Call the model.

import numpy as np
from aidlite import AutoModel

inputs = np.load("input.npy", allow_pickle=False)
with AutoModel.from_pretrained("model.aidem") as model:
    outputs = model(inputs)

Keep the dtype and shape required by the original model. For example, use numpy.float16 for an FP16 input. Use each input's declared dtype, including IDs and masks. A model with one output returns an array. A model with several outputs returns a tuple in original output order. Outputs remain valid after the model closes.

Call a model with several inputs

Save the input arrays in one .npz file. Use the original input names as keys. Keep each array's own dtype.

import numpy as np
from aidlite import AutoModel

with np.load("inputs.npz", allow_pickle=False) as archive:
    inputs = {name: archive[name] for name in archive.files}

with AutoModel.from_pretrained("model.aidem", return_dict=True) as model:
    outputs = model(inputs)

The result dictionary uses the original output names. Positional arguments also use the original input order. Do not change the input order to match the converted tensors.

Check the interface

After loading, check model.inputs and model.outputs. Each TensorSpec provides a name, shape, and dtype. model_shape and runtime_dtype describe the adapted runtime interface.

Option	Behavior
interface="auto"	Prefer usable AIMO metadata. Fall back to backend tensor APIs if metadata is absent or unusable.
interface="original"	Require metadata that verifies the original interface.
interface="runtime"	Use the internal interface and manage conversions in caller code.

New AIMO QNN models use verified tensor names, order, and axis permutations. The wrapper performs dtype conversions required by the ordinary SDK interface. It does not repeat backend quantization or dequantization. Restoring a dtype cannot restore precision lost during quantization.

A call fails if an interface conversion changes integer or Boolean values. Strict original mode fails if names, shapes, or layouts cannot be verified. Auto mode emits ModelMetadataWarning when metadata is unusable. It then uses backend names, shapes, and dtypes. Check model.metadata_source, model.metadata_warning, model.inputs, and model.outputs after loading. Correct the metadata if the original interface is required. Do not use arbitrary dtype conversions to bypass these errors.

Supported scope

Original I/O adaptation currently supports graph 0 of a QNN model with one artifact. It requires aplux.model_io metadata, version 1. A QNN context still uses the fixed shapes selected during conversion. Metadata does not enable dynamic inputs for a fixed model.

A .bin file or older .aidem without metadata may require an explicit framework. from_pretrained() reads converted local models only. It does not download models or directly convert arbitrary Transformers weights.

ONNX input types

Install the matching updated ONNX plugin. The plugin can bind float32, float16, bool, int8, int16, int32, int64, uint8, uint16, uint32, and uint64 inputs. Prepare arrays with the dtype in model.inputs. For example, use numpy.float16 for a declared FP16 input. Do not convert it to FP32 first. The current interface does not support float64, strings, or packed types. If a type is unsupported, correct the model or select a backend that supports it.

Select LOCAL or REMOTE

The default is LOCAL. Set implementation="remote" to call the Android inference service through IPC. Keep the same model(x) call. device selects the execution device. implementation selects the execution location. You can use device="dsp" with implementation="remote".

import numpy as np
from aidlite import AutoModel

with AutoModel.from_pretrained(
    "model.qnn240.ctx.bin", implementation="remote"
) as model:
    print(model.implementation)  # remote
    # This example uses one fixed-shape input. Prepare real data for your model.
    x = np.zeros(model.inputs[0].shape, dtype=model.inputs[0].dtype)
    y = model(x)

For several inputs, supply a dictionary of tensor names and arrays. You can also select return_dict=True, select outputs, and reuse output arrays. With valid AIMO metadata, AutoModel adapts the original dtype and layout. Without that metadata, it uses the runtime interface from the service. Check model.inputs and model.outputs. Do not assume that each model uses FP32.

InferenceSession accepts the same execution option:

from aidlite import InferenceSession

with InferenceSession(
    "model.qnn240.ctx.bin", framework="qnn240", device="dsp",
    implementation="remote",
) as session:
    # x must match the runtime interface. This session does not adapt original I/O.
    y = session.run(x)

Existing code can set config.implement_type = ImplementType.TYPE_REMOTE. An explicit implementation overrides the copied config. It does not change your config. None uses the config value. TYPE_DEFAULT selects LOCAL. Read the session's implementation property to inspect the selected location.

Check these conditions:

1. Install matching Python extension, core, REMOTE plugin, and Android service components.
2. Check the service backend and model versions. A client update does not update QNN.
3. Check that the service returns actual tensor information after loading. Manual model shapes are not required.
4. Use close() or with to release the session. The output-view lifetime rules still apply.

If the service or model information is unavailable, the call reports an error. It does not fall back to LOCAL. The legacy 2.3.2 adapter does not support high-level REMOTE. Use matching maintained builds. The current dynamic ONNX input capability requires LOCAL CPU execution. This option does not add dynamic-shape support to the service. IPC latency depends on the service, model, and device. LOCAL latency is not guaranteed.

QNN REMOTE call optimization

Static, single-graph QNN REMOTE models can use the complete-run path. AutoModel selects this path automatically. Keep the same model(x) call. This path combines the inputs, execution, and selected outputs in one native call. With a compatible service, it can reduce IPC round trips. The tensor names, original dtype, and layout rules stay the same. You can select outputs, supply output arrays, or use copy=False.

Install matching Python extension, core, and REMOTE plugin components. Before execution, the plugin checks service capabilities and control-message size. It uses the original step-by-step path for older services or oversized control messages. After an execution error or timeout, it does not retry inference automatically. Follow the error message to create a new session.

Other backends, dynamic models, and multi-graph models keep the original high-level path. InferenceSession.run() keeps its step-by-step input-binding behavior. For example, you can bind input B, then supply input X and execute.

The REMOTE plugin maps QNN version identifiers to the QNN backend identifier in the service protocol. This mapping does not install or update QNN on Android. The service must still support the QNN version used by the model.

The default output is an independent copy. copy=False returns read-only views that retain the Python session owner. Do not use an old view after the next execution or an explicit session close. Measure latency with your model. This optimization does not guarantee LOCAL latency.

Reuse remote input storage

The matching service reuses length headers, validation flags, and byte storage in the complete-run path. Use the same model(x) and InferenceSession.run() calls. Each input has separate staging and submitted storage, with up to two byte arrays. Preparation does not rewrite an array that the backend currently holds. A failed payload read does not set partial inputs or execute inference. After an SDK or output-write failure, subsequent requests use fresh cache storage.

The buffers belong to the current model context. A changed input byte count replaces the corresponding storage. Java can collect the buffers after the model owner is released. Input data still copies from shared memory into Java arrays. The JNI and backend I/O calls stay the same. Native input wrappers also reuse storage. This change primarily reduces temporary object allocation.

A limited QCS8550, Android 13, and matching QNN 2.48 test reduced ART process allocation by about 29% to 39% for native UINT8 inputs. Large-input P50 latency fell by about 12% to 17%. These figures apply only to the tested models and environment. Small FP32 inputs with default outputs had P50 changes of +3.47%, +4.34%, and +1.54% across three rounds. Tail latency also varied. These results do not establish equal latency. Each byte input can retain up to two payload arrays. Check memory use and latency with your model before deployment.

See the service input storage guide for build and check steps.

Check QNN JNI buffers

The matching JNI reads and caches actual input and output specifications after model load. It uses SDK metadata. Caller-declared shapes do not relax validation. Inference does not query these specifications again. This interface uses the default graph. It has no graph-selection operation.

Ordinary input and output routes still use the FP32 representation. Native mode uses the model's runtime representation. A native FloatArray requires an FP32 model tensor. A byte array contains complete native bytes, including FP16 or wider types. Existing byte entry names and JNI method signatures stay the same. These entry points do not infer numeric types from byte contents or add backend capabilities.

An input array must have exactly the required length. A supplied size must also equal the array's byte count. Null arrays, wrong lengths, negative indices, and out-of-range indices produce IllegalArgumentException. JNI checks the contract before it obtains array data or calls the SDK. Invalid input does not replace previously submitted input. You can continue with correct data in the loaded session.

Missing, invalid, or overflowing metadata makes model load fail. Release the model owner. Then correct the model or matched SDK components. JNI rejects null output data and wrong output lengths before copying to Java arrays. It also releases acquired input arrays when an SDK call throws. Use each owner sequentially and release it after use.

Python callers continue to use AutoModel or InferenceSession. These Android QNN JNI checks add no Python option. With valid AIMO metadata, AutoModel still adapts the original model dtype and layout. See the QNN JNI buffer guide for build and check steps.

Use a QNN 2.48 remote service

A QNN 2.48 model requires matching Android service, JNI, and QNN runtime components. AutoModel and InferenceSession use the same implementation="remote" option. For example, load a model converted for QNN 2.48:

from aidlite import AutoModel

with AutoModel.from_pretrained(
    "model.qnn248.ctx.bin", implementation="remote"
) as model:
    outputs = model(inputs)

Prepare inputs as specified by the original model. Do not convert all inputs to FP32. With valid AIMO metadata, AutoModel still adapts the original dtype and layout. The service must support the model version and data mode. A QNN version identifier in the client does not update the Android runtime.

For a custom service build, use the header and archive from one SDK build. Maintained SDK builds export sdk-package/Release/manifest.json. This SDK build manifest is separate from model input/output metadata. The newer SDK changes the parameter structure and cleanup API. Supply the complete SDK package through QNN_JNI_SDK_MANIFEST when you build JNI. The build checks file hashes, application binary interface (ABI), Android API level, and C++ runtime. Without this option, the original SDK pair remains selected. See the SDK build guide and JNI build guide.

To diagnose a custom Android service, call getInferenceFrameworkVersion() after AidQNN.init() succeeds. The method returns the loaded backend's QNN core API version and caches the result. An older JNI or SDK without the query returns unknown, instead of a fixed 2.36.0. The core API version differs from the QAIRT package release. The qualified QAIRT 2.40.0.251030 package reports 2.30.0; 2.48.0.260626 reports 2.37.0. Keep the SDK build manifest and library hashes to identify the exact package. The query runs outside inference. It does not update the remote service.

An isolated QCS8550 service tested matching QNN 2.40 and 2.48 runtimes. Checks include two inputs, three outputs, named feeds, output reuse, and read-only views. Two sessions with different data modes also passed concurrent checks. Controlled metadata tested original input types and layout adaptation. This does not qualify a fresh AIMO conversion, encrypted models, or all platforms. A source update does not update an installed service package.

Measure latency with the actual model. In this paired fixture, six P50 changes for 2.48 ranged from -4.96% to +3.18%. FP32 default-output P99 increased by about 8.8%. These results do not establish tail-latency equivalence or native latency.

Optional inference controls

The default model(inputs) call remains available. Use predict() when you need output selection or reusable output arrays.

Select outputs

Use output names from model.output_names. The result follows the selected order.

import numpy as np
from aidlite import AutoModel

inputs = np.load("input.npy", allow_pickle=False)
with AutoModel.from_pretrained("model.aidem") as model:
    name = model.output_names[0]
    result = model.predict(inputs, outputs=[name])

Only selected outputs are read and adapted. This can reduce output copying and conversion. It does not promise less computation inside the model. return_dict=True uses the selected output names as dictionary keys.

Reuse an output array

Allocate a writable, C-contiguous NumPy array. Use the output's declared dtype and concrete shape. For a dynamic output, obtain the actual shape before allocating the array.

import numpy as np
from aidlite import AutoModel

inputs = np.load("input.npy", allow_pickle=False)
with AutoModel.from_pretrained("model.aidem") as model:
    name = model.output_names[0]
    first = model.predict(inputs, outputs=[name])
    buffer = np.empty_like(first)
    result = model.predict(inputs, outputs=[name], out=buffer)

The returned array is the supplied buffer. Reusing the buffer overwrites its previous result. For several outputs, pass out={name: array} with exactly the selected names. Output buffers must not overlap inputs or each other. Output buffers require return_tensors="np". Backend-to-buffer copying can still occur. Required dtype or layout conversions can still allocate temporary arrays.

Select native input and output modes

These options are for QNN and SNPE pipelines that already manage backend tensor formats. Set native_input and native_output independently when loading the model. Both options default to False.

from aidlite import AutoModel

with AutoModel.from_pretrained(
    "model.qnn248.ctx.bin",
    native_input=True,
    native_output=False,
) as model:
    print(model.inputs, model.outputs)

Enabling either option selects the internal interface for the whole model. Prepare every tensor according to model.inputs and model.outputs. A native side uses physical tensor types. An ordinary side uses the backend's ordinary interface. The caller must verify quantization parameters and tensor meaning. The wrapper does not add quantization. Native modes stay fixed for the model's lifetime. They cannot be combined with interface="original".

Set graph priority

Pass backend settings through config. The framework copies the settings at load time and does not change the caller's object.

from aidlite import AutoModel, Config

config = Config.create_instance()
if config is None:
    raise RuntimeError("Cannot create AidLite config")
config.graph_priority = "high"
with AutoModel.from_pretrained("model.qnn248.ctx.bin", config=config) as model:
    print(model.inputs)

graph_priority defaults to an empty string, which keeps the backend default priority. QNN DSP supports low, normal_low, normal, normal_high, high, high_plus, critical, and critical_plus. QNN 2.40 provides only four native priority levels. It maps normal_low to low. It maps high_plus, critical, and critical_plus to high. QNN 2.48 uses the corresponding native levels. The effect depends on backend and device support. Changing the original config after loading does not change the loaded model.

Control Python concurrency

release_gil defaults to False. Default calls keep the existing Python scheduling behavior. Set release_gil=True in AutoModel.from_pretrained() or InferenceSession() to let other Python threads run during native inference. This option needs the matching updated native extension and an owned session. The legacy borrowed Model/Context adapter does not support this option.

Use a separate session for each worker thread. Call one session in sequence. Do not change inputs, switch graphs, or close the session during inference. AutoModel serializes calls on one model. InferenceSession rejects other native operations on that session during inference.

Measure the actual workload before you enable this option. Short CPU models can become slower with thread contention. Releasing the GIL does not guarantee parallel execution on one NPU. Set config.number_of_threads for the device and the workload. Several sessions with large CPU thread pools can increase latency.

Inspect quantization parameters

TensorSpec.dtype is the dtype required by the calling interface. TensorSpec.native_dtype is the physical dtype reported by the backend. These dtypes can differ. For example, ordinary QNN I/O can use floating arrays while physical tensors use uint8.

from aidlite import AutoModel

with AutoModel.from_pretrained("model.qnn248.ctx.bin") as model:
    for spec in model.inputs + model.outputs:
        print(spec.name, spec.dtype, spec.native_dtype)
        print(spec.quantization, spec.quantize_scale, spec.quantize_offset)

Available QNN per-tensor parameters are returned in spec.quantization. The parameter object provides scale, zero_point, dtype, and offset. offset follows the QNN convention: zero_point = -offset. spec.quantize_scale and spec.quantize_offset are shortcuts for the same parameters. The parameters come from tensor information for the loaded backend model. AIMO metadata continues to supply original-interface mappings and verified shapes. Missing, invalid, or known unsupported quantization encodings return None. Do not treat None as scale 1 or zero point 0. The scalar parameter object does not represent per-axis, block, or packed low-bit encodings. When an older backend API cannot report the full encoding type, the caller must still verify the model's quantization scheme. Other backends also return None when their quantization convention is not verified.

Quantize and dequantize explicitly

quantize() and dequantize() are optional utilities. model(inputs) does not call them automatically. The default original-interface dtype rules do not change.

The following example requires a per-tensor quantized QNN model with one input and one output. The input file must use the layout required by the backend.

import numpy as np
from aidlite import AutoModel, quantize, dequantize

inputs = np.load("input-runtime-layout.npy", allow_pickle=False)
with AutoModel.from_pretrained(
    "model.qnn248.ctx.bin", native_input=True, native_output=True,
) as model:
    input_params = model.inputs[0].quantization
    output_params = model.outputs[0].quantization
    if input_params is None or output_params is None:
        raise ValueError("This model needs explicit supported quantization parameters")
    quantized = quantize(inputs, input_params)
    result = model.predict(quantized, copy=False)
    restored = dequantize(result, output_params, dtype=inputs.dtype)

quantize() calculates round_to_even(x / scale) + zero_point, then clamps to the integer dtype range. dequantize() calculates (q - zero_point) * scale. QNN offset has the opposite sign to zero point. Do not pass a QNN offset directly as zero_point. The utilities do not change layout, infer parameters, or restore precision lost during quantization. Integer dtypes are native-endian int8, uint8, int16, uint16, int32, and uint32. Scale must be positive and finite. Non-finite inputs, invalid zero points, conflicting parameters, and mismatched dtypes raise errors. Dequantization defaults to float32; you can select float16 or float64 explicitly. This utility default does not change the original model's interface dtype.

You can also supply known parameters explicitly.

import numpy as np
from aidlite import QuantizationParams, quantize, dequantize

params = QuantizationParams(scale=0.5, zero_point=10, dtype=np.uint8)
quantized = quantize(np.array([1.0], dtype=np.float16), params)
restored = dequantize(quantized, params, dtype=np.float16)
assert quantized[0] == 12
assert restored[0] == 1.0

same = quantize([1.0], scale=0.5, zero_point=10, dtype=np.uint8)

When you use a parameter object, do not also supply scale, zero_point, or the quantized target dtype. For multiple inputs or outputs, use each tensor's own parameters. Do not quantize or dequantize manually when ordinary QNN or SNPE I/O already performs the conversion.

Borrow a backend output view

The default is copy=True; results remain independent of later inference and model closure. Use predict(copy=False) explicitly to reduce output copying.

import numpy as np
from aidlite import AutoModel

inputs = np.load("input-runtime-layout.npy", allow_pickle=False)
with AutoModel.from_pretrained("model.onnx", interface="runtime") as model:
    view = model.predict(inputs, copy=False)
    saved = view.copy()

A borrowed result is a read-only NumPy view. The framework does not copy that output's data. The backend itself can still copy or convert data. A layout permutation can return a non-contiguous view. If the output requires a dtype conversion, copy=False raises an error before inference. Use the default copy, or explicitly select an internal interface with the required dtype. copy=False cannot be combined with out or return_tensors="pt". Output selection and return_dict=True remain available.

Keep the model alive and finish using the view before the next call or model closure. The next call can overwrite or reallocate backend output memory. Do not access a borrowed array after closing the model. Other threads must not call or close the same model while you use the view. To retain a result, call view.copy() within its valid lifetime.

Use the QNN 2.40 and 2.48 backends

These behaviors require the QNN 2.40 or 2.48 plugin with the native tensor fixes. A source merge does not publish an installation package. QNN 2.40 and 2.48 use the same high-level calls. The example uses a QNN 2.48 model. For QNN 2.40, select a model converted for that version. For example, load model.qnn240.ctx.bin. The model must match the plugin and runtime versions. Before upgrading, check that the plugin contains these fixes.

import numpy as np
from aidlite import AutoModel, Config

config = Config.create_instance()
if config is None:
    raise RuntimeError("Cannot create AidLite config")
config.qnn_shared_buffer = True

with AutoModel.from_pretrained(
    "model.qnn248.ctx.bin", device="npu", config=config,
) as model:
    inputs = np.load("input.npy", allow_pickle=False)
    result = model(inputs)

Check the model input requirements before preparing input.npy. Valid AIMO metadata still defines the original model dtype and layout. This section does not require FP32 for every original model.

qnn_shared_buffer=True selects RPC shared buffers owned by the plugin. It does not register an arbitrary NumPy array as NPU memory. Input can still require copying, quantization, or layout conversion. Shared buffers do not imply an end-to-end path without copying.

Use native_input=True and native_output=True for internal data. The dtype must match native_dtype. FP16, boolean, integer, and fixed-point data use their actual byte widths. Native output borrows backend storage directly. Ordinary FP32 conversion storage is allocated on demand and reused. Scalar quantization conversion requires valid scale/offset parameters. For other encodings, use native mode and perform the required conversion yourself.

A quantized scalar input contains one element. A failed conversion reports an error. Ordinary input conversion reuses the tensor size check. It no longer allocates temporary storage for a dimension list. The quantization algorithm and rounding rules stay the same.

If loading a .so model or a QNN 2.48 .dlc model fails, read the backend log. QNN model buffer must contain data means that the model data is empty. Create QNN model or Write QNN model file identifies a file preparation error. Check the system error in that message before you retry. The model object owns the internal model file. Releasing that object also releases the file, even if initialization failed.

For an optional backend extension, set its JSON path before loading the model:

config.backend_extension_config = "/absolute/path/backend-config.json"

Pass this configuration as config=config to AutoModel.from_pretrained. Use the extension library and JSON settings for the same QNN SDK version. Ordinary JSON configuration does not require a DLC resource manager. If loading fails, check the backend log for a library, interface, or configuration error. The SDK releases extension objects and libraries when construction fails or the model closes.

Use graph to select a graph in a model with multiple graphs. For example, graph=1, interface="runtime" selects the second graph. Each call executes only the selected graph. After failed inference, do not read the previous output. Correct the error and run inference again.

If closing fails, retain the model and its input buffers. Check the error log and correct the runtime error. Then retry model.close(). Power configurations that were released are not released again. Wait for closing to complete before further inference.

If you set a runtime library path, retain the matching DSP file path. AIDLUX_QNN_LIB_DIR_PATH selects the runtime directory. ADSP_LIBRARY_PATH can provide additional DSP search directories. Use host libraries, stubs, and skels from the same SDK release. Do not mix QNN runtime versions in one process.

Use optional QNN 2.48 memory binding

The maintained QNN 2.48 backend provides the QNN_BIND_MEM_HANDLES build option. The default is off. A matching build with this option checks the binding capability. The backend must report support when config.qnn_shared_buffer is enabled. An unsupported capability keeps the normal shared-buffer path. This option does not change AutoModel or model(x) calls. It does not provide zero-copy access to arbitrary input arrays.

Each graph retains its own binding information. Model loading prepares each binding. Inference does not prepare it again. Model cleanup releases the binding before it frees that graph's memory. If cleanup fails, keep the model object. Resolve the cause, then retry close(). Use destroy() with the C++ SDK. Do not run inference or replace the model while cleanup is pending.

The QNN 2.48 runtime in the tested QCS8550 environment does not support this capability. Real NPU tests verify the normal fallback path. Controlled host tests verify binding and failure handling. Binding on a supported NPU remains unverified. Measure latency with the actual model. This option does not guarantee a speedup. See the QNN build guide for build steps and test tools.

Select another backend

At model creation, the SDK asks the current core about local backend support. Importing the Python package does not scan installed backend plugins. Python and core use the same model-format rules. Automatic discovery uses format hints from the current native core. A filename provides a hint only. An explicit framework argument or aidlite.json declaration takes priority over the filename. A .dlc file defaults to generic QNN, not SNPE2. An explicit framework or valid AIMO metadata can select the backend. A versioned filename such as .qnn248.dlc can supply a version hint when no concrete version is selected explicitly. The SDK recognizes .tnnmodel, .mnn, .nb, and .ms formats. Format recognition does not mean that the current core implements the backend. The maintained core uses QNN 2.40 and QNN 2.48. It disables QNN 2.36 and earlier versions, and the SNPE backends. The related enums and old examples remain for legacy migration. A .rknn file selects RKNN and defaults to NPU. For QNN and SNPE, device="npu" maps to DSP. Other backends retain the NPU selection. Use model_weight_path when the backend requires a separate weight file.

Separate model declarations from runtime support

The framework field in aidlite.json declares the model's backend. The SDK checks only the field's string type and identifier syntax. The SDK converts names to lowercase. After conversion, use letters, digits, and underscores, and start with a letter. For example, a model asset can retain a future backend name. This does not mean that the current SDK can run that backend. Keep an explicit backend declaration even when the filename ends in .onnx. The core and plugin still validate actual execution.

Backend declarations in explicit arguments, aidlite.json, and actual AIMO metadata must agree. Conflicts between these declarations still produce an error.

A generic qnn declaration can be refined to a concrete version such as qnn248. For example, set framework="qnn248" when matching metadata declares qnn. Two concrete versions in these declarations must agree. For example, the SDK reports an error when AIMO metadata declares qnn240 and the explicit argument selects qnn248. Metadata fallback does not ignore a real framework conflict.

Use the query functions in the matching updated SDK to inspect core dispatch support:

import aidlite

framework = aidlite.infer_framework_type("model.dlc")
backend = aidlite.get_local_backend_key(framework)
print(framework, backend)

infer_framework_type() uses the filename only. It does not read the model or inspect plugin files. It returns FrameworkType.TYPE_DEFAULT when the format is unknown. get_local_backend_key() returns the local backend key implemented by this core, or None if it is not implemented. Generic QNN returns "qnn"; model creation selects the concrete version. A nonempty result does not establish plugin installation, loadable dependencies, or device compatibility. Actual loading still requires a matching plugin, SDK, and device.

Model creation errors distinguish these cases:

Condition	Action
The native binding does not recognize the backend name	Check the name and use matching core and Python binding versions. Do not rewrite a correct model declaration only because the installed SDK is older.
The core does not implement the local backend	Select a supported backend or convert the model again. Retaining an enum does not enable a disabled backend.
Plugin loading, dependencies, or version checks fail	Use the backend path and reason in the initialization error to check the plugin, dependencies, and versions.

get_last_backend_error() returns the current thread's most recent backend initialization error. The next init() clears it. Read it immediately when checking low-level return codes manually.

Check metadata fallback

Valid AIMO metadata supplies the preferred input and output shapes. At load time, backend tensor APIs verify the contract once. This check protects native buffer sizes. Inference reuses the verified plan. If it cannot be used, auto mode queries get_input_tensor_info() and get_output_tensor_info(). Backend information describes the converted model. It cannot reconstruct missing original names, layouts, or dtypes. Use interface="original" to require the original interface without fallback. A backend query or model-loading failure still raises an error. SDK errors include available operation/status details and suggested checks.

REMOTE troubleshooting

This section applies to Android inference services called through interprocess communication (IPC). Check the backend version, input count, and dtypes supported by the Android service. The native backend and REMOTE mode can have different capabilities.

Check the service and backend versions

First, make sure that the Android inference service is running. Then check that its QNN SDK version supports the model. Updating AidLite on Linux does not update the Android QNN backend. If the model requires a newer QNN SDK, update the matching service or convert the model again.

Shared memory in a custom Android service

When you build an Android inference service, use matching AidLuxIPC Java and JNI components. Builds with this repair support Java try-with-resources for shared-memory objects. Existing destroy_ashmem() calls still work. Before you close an object, finish operations that use its borrowed file descriptor. See Java ownership and array bounds.

JNI builds with the region-write change copy only the requested input range. You can continue to reuse a larger Java input array. Byte writes and aligned float writes use one region copy. Unaligned float byte offsets keep the existing compatible path. If a copy fails, the exception is preserved. The write does not retry through another path. Data already written is not automatically restored. Measure latency with your model. The region-write change does not guarantee faster inference.

Python callers can continue to use with for AutoModel or InferenceSession. This releases the session. Manual Android shared-memory management is not required.

Check service crash logs

An Android service crash can cause an inference timeout. First, connect to the test device through Android Debug Bridge (ADB). Use adb devices to identify the device serial number. Replace DEVICE_SERIAL with that number.

adb -s DEVICE_SERIAL logcat -d -b crash

Check the process name and error message in the log. If the log contains both ai.onnxruntime and JNI DETECTED ERROR, supply the complete log. If it also contains mid == null, ask the device maintainer to check the ONNX release configuration. This issue can require a new build and release of the Android service. After the service update, check normal inference and error reporting again.

Low-level API reference

Use these interfaces to manage the model, configuration, and interpreter manually. The selected backend defines their dtype requirements. These requirements differ from AutoModel's original-interface adaptation.

Model Data Type class DataType

For AidliteSDK, it is used to handle different models from different frameworks, and each model may have different input and output data types. When calling certain interfaces, you need to set the input and output data types of the model, so this data type enumeration is required.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid data type
TYPE_UINT8	int	1	Unsigned char data
TYPE_INT8	int	2	Char data
TYPE_UINT32	int	3	Unsigned int32 data
TYPE_FLOAT32	int	4	Float data
TYPE_INT32	int	5	Int32 data
TYPE_INT64	int	6	Int64 data
TYPE_UINT64	int	7	Unsigned int64 data
TYPE_INT16	int	8	Int16 data
TYPE_UINT16	int	9	Unsigned int16 data
TYPE_FLOAT16	int	10	Float16 data
TYPE_BOOL	int	11	Bool data

Inference Implementation Type class ImplementType

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid ImplementType type
TYPE_MMKV	int	1	Reserved in the C++ API; currently not exported in Python bindings and cannot be used directly
TYPE_REMOTE	int	2	Remote inference implementation
TYPE_FAST	int	2	Compatibility alias of TYPE_REMOTE; deprecated and not recommended for continued use
TYPE_LOCAL	int	3	Local inference implementation

💡Note

Special notes:

The difference between TYPE_LOCAL and TYPE_REMOTE is that, for the Aplux integrated system, the former completes inference locally in the Linux environment without data transfer, while the latter exchanges data with the Android environment through IPC (inter-process communication), and the inference process runs in Android, so data transfer is involved. Therefore, TYPE_LOCAL is the preferred option for completing inference whenever possible.

Model Framework Type class FrameworkType

Because AidliteSDK integrates multiple deep learning inference frameworks, during use you need to specify which framework the current model belongs to, so this framework type enumeration is required.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid FrameworkType type
TYPE_SNPE	int	1	SNPE 1.x (DLC) model
TYPE_TFLITE	int	2	TFLite model
TYPE_RKNN	int	3	RKNN model
TYPE_QNN	int	4	QNN model
TYPE_SNPE2	int	5	SNPE 2.x (DLC) model
TYPE_NCNN	int	6	NCNN model
TYPE_MNN	int	7	MNN model
TYPE_TNN	int	8	TNN model
TYPE_PADDLE	int	9	Paddle model
TYPE_MS	int	10	MindSpore model
TYPE_ONNX	int	11	ONNX model
TYPE_QNN216	int	101	QNN2.16 model
TYPE_SNPE216	int	102	SNPE2.16 model
TYPE_QNN223	int	103	QNN2.23 model
TYPE_SNPE223	int	104	SNPE2.23 model
TYPE_QNN229	int	105	QNN2.29 model
TYPE_SNPE229	int	106	SNPE2.29 model
TYPE_QNN231	int	107	QNN2.31 model
TYPE_QNN236	int	108	QNN2.36 model
TYPE_QNN240	int	109	QNN2.40 model
TYPE_QNN248	int	110	QNN2.48 model

💡Note

Special notes:

• Historical enum values remain available. This does not mean that the core implements every listed backend. Query local dispatch support with get_local_backend_key().
• TYPE_QNN selects the newer installed plugin among concrete QNN backends implemented by the current core. The maintained versions are QNN 2.40 and QNN 2.48. Older QNN and SNPE backends remain disabled.
• Do not mix QNN runtime versions in one process.

Inference Acceleration Hardware Type class AccelerateType

Each deep learning inference framework may support running on different acceleration devices, such as SNPE/QNN models running on DSP devices and RKNN models running on NPU devices. Therefore, during use you need to specify which device the current model is expected to run on, and this acceleration hardware enumeration is required.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid AccelerateType type
TYPE_CPU	int	1	Run with CPU acceleration
TYPE_GPU	int	2	Run with GPU acceleration
TYPE_DSP	int	3	Run with DSP acceleration
TYPE_NPU	int	4	Run with NPU acceleration

Log Level class LogLevel

AidliteSDK provides interfaces for configuring log output, which will be introduced later. If you need to specify the log level currently used by AidliteSDK, this log level enumeration is required.

Member Variable Name	Type	Value	Description
INFO	int	0	Information
WARNING	int	1	Warning
ERROR	int	2	Error
FATAL	int	3	Fatal error

Device Information class DeviceInfo

Starting from 2.3.1, Python bindings also provide the DeviceInfo type, which is used to describe available underlying inference devices. It is currently mainly used for device querying and custom device configuration in QNN DSP scenarios.

Member Variable List

Member Variable Name	Type	Default Value	Description
id	int	0	Device ID
type	int	0	Device type number interpreted by the backend
cores_num	int	1	Number of cores included in the device
cores_id	list	Empty	List of device core IDs

💡Note

The version field in the C++ structure is currently not exported in Python bindings. On the Python side, use the other four public fields directly.

Input/Output Information class TensorInfo

Each model has its own inputs and outputs. For models that developers are not very familiar with, it may be necessary during development to query the model's input and output Tensor information, so that pre-processing and post-processing for model inference can be implemented more conveniently. Aidlite provides interfaces for querying detailed input and output information, as well as the TensorInfo data type used to store these details.

Member Variable List

The TensorInfo object is used to store detailed information about model input and output Tensors, including the following parameters:

Member	version
Type	int
Default	0X00020205
Description	This member identifies the TensorInfo version. If more information needs to be extended in the future, the version number will be updated

Member	index
Type	int
Default	No default value
Description	Index value of the Tensor

Member	name
Type	str
Default	No default value
Description	Name string of the Tensor

Member	element_count
Type	int
Default	0
Description	Number of data elements in the Tensor, not the number of bytes

Member	element_type
Type	DataType
Default	DataType::TYPE_DEFAULT
Description	Data type of elements in the Tensor

Member	dimensions
Type	int
Default	0
Description	Number of dimensions of data elements in the Tensor

Member	shape
Type	list[int]
Default	No default value
Description	Shape description of data elements in the Tensor

Model class Model

Before creating an inference interpreter, you need to configure detailed parameters related to the specific model. The Model class is mainly used to record model file information, structural information, and model-related content during runtime.

Create Empty Instance Object create_instance()

The current Python bindings also provide a no-argument overload, which allows an empty Model instance to be created first, and then the caller can supplement the model path or property information later in the workflow.

API	create_instance
Description	Construct a Model instance object that does not carry a model file path
Parameters	None
Return	Normal: Model instance object
	Exception: None

Create Instance Object create_instance()

To configure detailed model information, you first need a model instance object. This function is used to create a Model instance object. If the model framework, such as SNPE or TFLite, requires only one model file, call this interface.

API	create_instance
Description	Construct a Model type instance object by passing the path name of the model file
Parameters	model_path: Path name of the model file
Return	Normal: Model instance object
	Exception: None

# Use the inceptionv3_float32.dlc file in the current path to create a model object; report an error if the return value is empty
model = aidlite.Model.create_instance(model_path=r"./inceptionv3.dlc")
if model is None:
    print("Create model failed.")
    return False

Create Instance Object create_instance()

To configure detailed model information, you first need a model instance object. This function is used to create a Model instance object. If the model framework, such as NCNN or TNN, involves two model files, call this interface.

API	create_instance
Description	Construct a Model type instance object by passing the path names of model files
Parameters	model_struct_path: Path name of the model structure file
	model_weight_path: Path name of the model weight parameter file
Return	Normal: Model instance object; Exception: None

# Use the two NCNN model files in the current path to create a model object; report an error if the return value is empty
model = aidlite.Model.create_instance( 
        model_struct_path=r"./mobilenet_ssd_voc_ncnn.param",
        model_weight_path=r"./mobilenet_ssd_voc_ncnn.bin")
if model is None:
    print("Create model failed.")
    return False

Set Model Properties set_model_properties()

After the model instance object is created successfully, you need to set the input and output data types of the model and the shape information of input and output Tensor data.

API	set_model_properties
Description	Set model properties, including input and output data shapes and data types.
Parameters	input_shapes: Shape array of input Tensors, a two-dimensional array structure.
	input_data_type: Data type of input Tensors, of DataType enumeration type.
	output_shapes: Shape array of output Tensors, a two-dimensional array structure.
	output_data_type: Data type of output Tensors, of DataType enumeration type.
Return	Normal: 0; Exception: non-zero
Note	For REMOTE, set the input and output information required by the backend. The ordinary QNN and SNPE low-level interfaces use aidlite.DataType.TYPE_FLOAT32 by default. For native data modes, check tensor metadata and the capabilities of the current REMOTE service. For other backends, prepare arrays with the dtypes required by the selected interface.

# The data type uses the aforementioned DataType, and input/output shapes are two-dimensional arrays
input_shapes=[[1,640,640,3]]
output_shapes=[[1,10,10,255], [1,20,20,255], [1,40,40,255]]
model.set_model_properties( input_shapes=input_shapes, 
                            input_data_type=aidlite.DataType.TYPE_FLOAT32, 
                            output_shapes=output_shapes, 
                            output_data_type=aidlite.DataType.TYPE_FLOAT32)

Get Model Path get_model_absolute_path()

If the model framework, such as SNPE or TFLite, involves only one model file, then when the Model object is constructed successfully, the passed model file parameter will be converted to an absolute file path.

API	get_model_absolute_path
Description	Used to obtain the absolute path of the model file
Parameters	None
Return	Model file path string corresponding to the Model object

Get Model Path get_model_struct_absolute_path()

If the model framework, such as NCNN or TNN, involves two model files, then when the Model object is constructed successfully, the passed model structure file parameter will be converted to an absolute file path.

API	get_model_struct_absolute_path
Description	Used to obtain the absolute path of the model structure file
Parameters	None
Return	Model structure file path string corresponding to the Model object

Get Model Path get_model_weight_absolute_path()

If the model framework, such as NCNN or TNN, involves two model files, then when the Model object is constructed successfully, the passed model weight parameter file will be converted to an absolute file path.

API	get_model_weight_absolute_path
Description	Used to obtain the absolute path of the model weight parameter file
Parameters	None
Return	Model weight parameter file path string corresponding to the Model object

Configuration class Config

Before creating an inference interpreter, in addition to setting specific Model information, you also need to set some configuration information used during inference. The Config class records configuration options that need to be set in advance, and these options will be used at runtime.

Create Instance Object create_instance()

To configure runtime information, you naturally need a configuration instance object first. This function is used to create a Config instance object.

API	create_instance
Description	Used to construct an instance object of the Config class
Parameters	None
Return	Normal: Config instance object
	Exception: None

# Create a configuration instance object; report an error if the return value is None
config = aidlite.Config.create_instance()
if config is None:
    print("Create config failed.")
    return False

Member Variable List

The Config object is used to set runtime configuration information, which includes the following parameters:

Member	accelerate_type
Type	class AccelerateType
Default	AccelerateType.TYPE_CPU
Description	Type of acceleration hardware

Member	implement_type
Type	class ImplementType
Default	ImplementType.TYPE_DEFAULT
Description	Used to distinguish different environments of backend inference implementation
Note	Developers do not need to explicitly set this configuration item. AidliteSDK will derive a more suitable ImplementType according to factors such as the current system environment and model framework

Member	framework_type
Type	class FrameworkType
Default	FrameworkType.TYPE_DEFAULT
Description	Type of the underlying inference framework

Member	number_of_threads
Type	int
Default	-1
Description	Number of threads; valid when greater than 0
Note	This configuration item can be set optionally if and only if framework_type is TFLITE

Member	snpe_out_names
Type	list
Default	None
Description	List of model output node names
Note	This configuration item must be set if and only if framework_type is SNPE

Member	is_quantify_model
Type	int
Default	0
Description	Whether the model is quantized; 1 indicates a quantized model
Note	This configuration item must be set if and only if implement_type is REMOTE

Member	remote_timeout
Type	int
Default	-1
Description	Interface timeout in milliseconds; valid when greater than 0
Note	This configuration item can be set optionally if and only if implement_type is REMOTE

Member	fast_timeout
Type	int
Default	-1
Description	Historical compatibility field of `remote_timeout`; it is still accessible in current Python bindings but is not recommended for continued use

Member	qnn_shared_buffer
Type	int
Default	0
Description	Whether to enable QNN shared buffer; currently mainly used in QNN DSP scenarios

Member	batch_invoke_size
Type	int
Default	1
Description	Backend batch inference configuration parameter; currently mainly used in QNN DSP scenarios

Member	backend_extension_config
Type	str
Default	Empty string
Description	Extension configuration file of the backend inference framework; currently mainly used in QNN DSP scenarios

Member	custom_device_info
Type	list[DeviceInfo]
Default	Empty
Description	Custom device information list used to specify the device set actually used by the backend

Member	reserved1
Type	int
Default	0
Description	Reserved field. It currently has no business meaning, and callers do not need to set it explicitly

💡Note

remote_timeout and fast_timeout correspond to the same timeout configuration. fast_timeout is only a historical compatibility field, and new code should consistently use remote_timeout.

config.framework_type = aidlite.FrameworkType.TYPE_QNN
config.accelerate_type = aidlite.AccelerateType.TYPE_DSP
# Set other configuration items as needed
# ... ...

Context class Context

It is used to store the runtime context involved in execution, including the Model object and Config object. Runtime context data may be extended in the future.

Constructor Context()

API	Context
Description	Construct a Context instance object
Parameters	model: Model instance object
	config: Config instance object
Return	Normal: Context instance object
	Exception: None

context = aidlite.Context(model=model, config=config)

Get Model Member Variable get_model()

API	get_model
Description	Get the Model object managed by context
Parameters	None
Return	Model object

model = context.get_model()

Get Config Member Variable get_config()

API	get_config
Description	Get the Config object managed by context
Parameters	None
Return	Config object

config = context.get_config()

Interpreter class Interpreter

An instance of the Interpreter type is the entity responsible for performing inference operations, carrying out the specific inference process. After the interpreter object is created, all operations are based on the interpreter object, so it is the absolute core of AidliteSDK.

Create Instance Object create_instance

To perform inference-related operations, an inference interpreter is indispensable. This function is used to construct an instance object of the inference interpreter.

API	create_instance
Description	Use the various data managed by the Context object to construct an Interpreter type object
Parameters	context: Context instance object
Return	Normal: Interpreter instance object
	Exception: None

# Create an interpreter object; report an error if the return value is None
interpreter = aidlite.Interpreter.create_instance(context=context)
if interpreter is None:
    print("Create Interpreter failed.")
    return False

Initialize init()

After the interpreter object is created, some initialization operations need to be performed, such as environment checking and resource construction.

API	init
Description	Complete initialization work required for inference
Parameters	None
Return	Normal: 0
	Exception: non-zero

# Initialize the interpreter; report an error if the return value is non-zero
result = interpreter.init()
if result != 0:
    print("sample : interpreter.init() failed.")
    return False

Get Device Information get_device_info()

Python bindings now also provide get_device_info(), which is used to query device information actually visible to the backend. This interface is supported from 2.3.1 onward and is currently mainly used in QNN DSP scenarios.

API	get_device_info
Description	Get the backend device detail list
Return	Normal: List of DeviceInfo objects
	Exception: raises an exception

device_info_list = interpreter.get_device_info()
for device in device_info_list:
    print(device.id, device.type, device.cores_num, device.cores_id)

Load Model load_model()

After the interpreter object completes initialization, the required model file can be loaded into the interpreter. The subsequent inference process uses the loaded model resources.

API	load_model
Description	Complete model loading related work. Since the model file path has already been configured in the Model object, the model loading operation can be executed directly
Parameters	None
Return	Normal: 0
	Exception: non-zero

# Load the model with the interpreter; report an error if the return value is non-zero
result = interpreter.load_model()
if result != 0:
    print("sample : interpreter.load_model() failed.")
    return False

Get Input Tensor Details get_input_tensor_info()

If you want to obtain detailed information about the model's input Tensors, you can use this interface.

API	get_input_tensor_info
Description	Get detailed information about input Tensors in the model
Return	Normal: A two-dimensional array of TensorInfo type data
	Exception: None
Note	This interface is supported from AidliteSDK version 2.2.5 onward

# Get detailed information about the model's input Tensor
input_tensor_info = aidlite_interpreter.get_input_tensor_info()
if len(input_tensor_info) == 0 :
    print("interpreter get_input_tensor_info() failed.")
    return False
for gi, graph_tensor_info in enumerate(input_tensor_info):
    for ti, tensor_info in enumerate(graph_tensor_info):
        print(  f"Input  tensor : Graph[{gi}]-Tensor[{ti}]-name[{tensor_info.name}]"
                f"-element_count[{tensor_info.element_count}]-element_type[{tensor_info.element_type}]"
                f"-dimensions[{tensor_info.dimensions}]-shape{tensor_info.shape}")

Get Output Tensor Details get_output_tensor_info()

If you want to obtain detailed information about the model's output Tensors, you can use this interface.

API	get_output_tensor_info
Description	Get detailed information about output Tensors in the model
Return	Normal: A two-dimensional array of TensorInfo type data
	Exception: None
Note	This interface is supported from AidliteSDK version 2.2.5 onward

# Get detailed information about the model's output Tensor
output_tensor_info = aidlite_interpreter.get_output_tensor_info()
if len(output_tensor_info) == 0 :
    print("interpreter get_output_tensor_info() failed.")
    return False
for gi, graph_tensor_info in enumerate(output_tensor_info):
    for ti, tensor_info in enumerate(graph_tensor_info):
        print(  f"Output tensor : Graph[{gi}]-Tensor[{ti}]-name[{tensor_info.name}]"
                f"-element_count[{tensor_info.element_count}]-element_type[{tensor_info.element_type}]"
                f"-dimensions[{tensor_info.dimensions}]-shape{tensor_info.shape}")

Set Input Data set_input_tensor()

Before executing inference, the required input Tensor data must be provided. Before setting input data, different preprocessing steps are needed for different models in order to adapt to the specific model.

API	set_input_tensor
Description	Set the input Tensor data required for inference
Parameters	in_tensor_tag: Index value or name string of the input Tensor, of type int or str
	input_data: Input data supporting the buffer protocol, such as numpy.ndarray, memoryview, or bytes
Optional Parameter	is_native: Whether the passed data should be treated as native backend data format. When `false`, quantization or dequantization is handled according to SDK default logic. When `true`, the raw format is passed directly. The default value is False
Return	Normal: 0
	Exception: non-zero
Note	1. The parameter in_tensor_tag supports Tensor name strings from AidliteSDK version 2.2.5 onward 2. For numpy.ndarray and memoryview, a C-contiguous memory layout is required; Non-contiguous arrays/views will trigger a RuntimeWarning and be automatically converted into contiguous memory copies, which may cause additional data copying overhead 3. For SNPE and QNN frameworks, input data is typically processed as float32 by default. If the native-format interface is used, the user must ensure the data format matches backend requirements.

# Set inference input data; report an error if the return value is non-zero
result = interpreter.set_input_tensor(in_tensor_tag=1, input_data=obj)
# result = interpreter.set_input_tensor(in_tensor_tag="input_tensor_name", input_data=obj)
# result = interpreter.set_input_tensor(in_tensor_tag=1, input_data=obj, is_native=True)
if result != 0:
    print("interpreter->set_input_tensor() failed.")
    return False

Execute Inference invoke()

After input data is set, the next step is naturally to execute inference on that input data.

API	invoke
Description	Execute the inference computation process
Parameters	None
Return	Normal: 0
	Exception: non-zero

# Execute inference; report an error if the return value is non-zero
result = interpreter.invoke()
if result != 0:
    print("sample : interpreter->invoke() failed.")
    return False

Get Output Data get_output_tensor()

After inference is completed, the result data produced by inference needs to be obtained. Once the result data is retrieved, it can be processed further to determine whether the result is correct.

API	get_output_tensor
Description	Obtain inference result data after successful inference
Parameters	out_tensor_tag: Index value or name string of the result output Tensor, of type int or str
	output_type: Target data type of the result output. This is optional. If omitted, it defaults to aidlite.DataType.TYPE_FLOAT32
Optional Parameter	is_native: Whether to return the underlying output byte view directly in the backend's native data format. The default value is False
Return	Normal: numpy.ndarray result data.
	Exception: None.
Note	1. The parameter out_tensor_tag supports strings from AidliteSDK version 2.2.5 onward 2. `output_type` is used to reinterpret underlying `uint8` results as the target data type and does not perform an extra data copy 3. For SNPE and QNN, the default interface usually returns data with float32 semantics. If the native-format interface is used, the user must parse the actual backend output format manually.

# Get inference result data; the return value is a numpy.ndarray
out_data = interpreter.get_output_tensor(out_tensor_tag=1, output_type=aidlite.DataType.TYPE_INT32)
# out_data = interpreter.get_output_tensor(out_tensor_tag="output_tensor_name", output_type=aidlite.DataType.TYPE_INT32)
# out_data = interpreter.get_output_tensor(out_tensor_tag=1, is_native=True)
if out_data is None:
    print("interpreter->get_output_tensor() failed.")
    return False

Resource Release destroy()

As mentioned earlier, the interpreter object needs to perform init() initialization and load_model() model loading. Correspondingly, the interpreter also needs to perform cleanup operations to destroy the resources created earlier.

API	destroy
Description	Complete the necessary cleanup operations
Parameters	None
Return	Normal: 0
	Exception: non-zero

# Release the interpreter; report an error if the return value is non-zero
result = interpreter.destroy()
if result != 0:
    print("sample : interpreter-> destroy() failed.")
    return False

Interpreter Builder class InterpreterBuilder

This class provides unified creation functions for Interpreter objects and is used to create the required interpreter object.

Build Interpreter build_interpreter_from_path()

When building an inference interpreter object, different parameters can be provided. The simplest way is to provide only the path and name of the model file. If the model framework, such as SNPE or TFLite, requires only one model file, call this interface.

API	build_interpreter_from_path
Description	Directly create the corresponding interpreter object from the path name of the model file
Parameters	model_path: Path name of the model file
Return	Normal: Interpreter object instance
	Exception: None

# Build an interpreter from the Model file path; report an error if the return value is None
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_path(model_path=r"./640.dlc")
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Build Interpreter build_interpreter_from_path()

When building an inference interpreter object, different parameters can be provided. The simplest way is to provide only the path and name of the model file. If the model framework, such as NCNN or TNN, involves two model files, call this interface.

API	build_interpreter_from_path
Description	Directly create the corresponding interpreter object from the path names of the model files
Parameters	model_struct_path: Path name of the model structure file
	model_weight_path: Path name of the model weight parameter file
Return	Normal: Interpreter object instance
	Exception: None

Build Interpreter build_interpreter_from_model()

In addition to providing model file paths, you can also provide a Model object when building an inference interpreter object. This allows not only the model file path to be set, but also the model's input and output data types and the shapes of input and output data.

API	build_interpreter_from_model
Description	Create the corresponding interpreter object by passing a Model object. Config-related parameters all use default values
Parameters	model: Model type object containing model-related data
Return	Normal: Interpreter object instance
	Exception: None

# Build an interpreter from the Model object; report an error if the return value is None
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_model(model=model)
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Build Interpreter build_interpreter_from_model_and_config()

In addition to the previous methods, an inference interpreter object can also be built by providing both a Model object and a Config object, so that model-related information and more runtime configuration parameters can both be supplied.

API	build_interpreter_from_model_and_config
Description	Create the corresponding interpreter object by passing a Model object and Config.
Parameters	model: Model type object containing model-related data
	config: Config type object containing some configuration parameters
Return	Normal: Interpreter object instance
	Exception: None

# Build an interpreter from Model and Config objects; report an error if the return value is None
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_model_and_config(model=model, config=config)
if interpreter is None:
    print("Create Interpreter failed.")
    return False

Other Methods

Get SDK Version Information get_library_version()

API	get_library_version
Description	Used to obtain version-related information of the current Aidlite-SDK
Parameters	None
Return	The returned value is the current Aidlite-SDK version information string

Get Python SDK Version Information get_py_library_version()

API	get_py_library_version
Description	Used to obtain version-related information of the current Py-Aidlite-SDK
Parameters	None
Return	The returned value is the current Py-Aidlite-SDK version information string

Set Log Level set_log_level()

API	set_log_level
Description	Set the current minimum log level and output log data at or above that level. By default, logs at WARNING level and above are printed.
Parameters	log_level: Value of type LogLevel
Return	Returns 0 by default

Log Output to Standard Terminal log_to_stderr()

API	log_to_stderr
Description	Set log output to the standard error terminal
Parameters	None
Return	Returns 0 by default

Log Output to Text File log_to_file()

API	log_to_file
Description	Set log output to the specified text file
Parameters	path_and_prefix: Storage path and name prefix of the log file
	also_to_stderr: Whether to also output logs to the stderr terminal. The default value is False
Return	Normal: 0
	Exception: non-zero

Get Latest Log Information last_log_msg()

API	last_log_msg
Description	Get the latest log information of the specified log level, usually used to obtain the latest error information
Parameters	log_level: Value of type LogLevel
Return	Latest log information