7. Configuration file details
This section describes the Pulsar Config
section.
7.1. Overview of config.prototxt
Pulsar
is a powerful and complex set of tools for compiling, simulating, and debugging models that often require the necessary configuration parameters to work exactly as intendedCurrently, all parameters can be passed to
Pulsar
through the configuration file, :ref: A few parameters <some_params_called_by_cmdline> can be specified or modified on an ad hoc basis through the command line parameter interfaceIn addition to passing configuration parameters to the compiler, the configuration file also has an important role in guiding the compiler through the complex compilation process
Attention
The compilation parameters passed through the command line parameter interface will override the parameters provided by the configuration file
Content Format
The current configuration file format is a text format called
prototxt
, whose contents can be read and modified directly with a text editorThe configuration file in
prototxt
format can be commented internally, with comments starting with#
The version is
proto3
Noun Conventions
Due to the complex structure of the configuration file and the depth of the configuration parameters, in order to facilitate the description and minimize the misunderstanding caused by improper terminology, we have agreed on the meaning of some common terms. If you find any unclear expressions, wording errors, etc., or have better suggestions in reading this series of documentation, please feel free to criticize and correct
Parameter path
Parameter path is used to express the position of a configuration parameter in a multi-layer structure parameter
When a configuration parameter is located in other structures with multiple nested levels, the name of the structure parameter at each level is used plus a dot
. ``'' to express the location of the currently introduced parameter in the configuration file, e.g. the parameter represented by the string ``pulsar_conf.batch_size
is located in the configuration file as follows:1pulsar_conf { # Compiler-related configuration 2 batch_size: 1 3}
Note
Compilation Process
The compilation process is generally used as a proxy for compiling a model in one format into another format. For example, to compile a model in
onnx
format intojoint
format
compile step
Compilation steps are generally used when a compilation process can be explicitly divided into several steps. For example, two
onnx
models are first compiled separately intojoint
format, and then the twojoint
models are fused to form a singlejoint
model.When describing the configuration file, it may say “the entire compilation process is divided into three compilation steps, with configuration parameters for each compilation step”
But when describing a particular compile step within a compile process, we may refer to the compile step as the compile process within a subsection, when the two terms refer to the same object. Note the contextual distinction
compile parameters
Compilation parameters are used to refer to the parameters that need to be configured for a compile process or compile step.
7.2. config Internal structure overview
The config.prototxt
consists of the following six sections, including:
config Internal structure example
1# config.outline.prototxt
2
3# Basic input and output configuration
4input_path: # Relative path of the input model
5input_type: # Input model type, by default it is equal to INPUT_TYPE_AUTO, the compiler will infer the model file name automatically, but sometimes the inference result is not expected
6output_path: # The relative path of the output model
7output_type: # Output model type, if not specified, it will be automatically recognized by model file suffix, default is equivalent to OUTPUT_TYPE_AUTO
8
9# Hardware selection
10target_hardware: # Currently available AX620, AX630
11
12# Special handling of Tensor (old version), called tensor_conf, new version is recommended for more complex customization
13input_tensors {}
14output_tensors {}
15
16# Special handling of Tensor (new version)
17src_input_tensors {} # Attributes of the input tensor used to describe the input model, equivalent to input_tensors
18src_output_tensors {} # Attributes used to describe the output tensor of the input model
19dst_input_tensors {} # Attributes of the input tensor used to modify the output model, equivalent to output_tensors
20dst_output_tensors {} # Attributes used to modify the output tensor of the output model
21
22# cpu subgraph backend processing engine: ONNX OR AXE
23cpu_backend_settings {}
24
25# neuwizard parameters configuration
26neuwizard_conf { # Used to instruct Neuwizard to compile the onnx model into lava_joint format
27 operator_conf {} # Used to formulate various cap operators
28 dataset_conf_calibration {} # Used to describe the calibration dataset during compilation
29}
30
31# pulsar compiler configuration
32pulsar_conf {
33 # pulsar_compiler is used to instruct pulsar_compiler to compile a lava_joint or lava format model into a joint or neu format model
34 ...
35}
The config.prototxt
needs to be properly configured according to the above structure.
Attention
The input_tensors
, output_tensors
options are kept for compatibility with older toolchains, while src_input_tensors
and dst_input_tensors
are equivalent to input_tensors
and output_tensors
, and it is recommended to use the newer version ————————————– Detailed description of the different modules of the configuration file ————————————– This section details each sub_config
in config.prototxt
.
7.2.1. 输入输出配置
7.2.1.1. input_path
Property Description
Attributes |
Description |
---|---|
Parameter path |
|
Parameter role |
Specifies the path to the input model |
Parameter type |
String |
Optional list |
/ |
Caution |
|
Code example
1# input_path example
2input_path: "./model.onnx"
7.2.1.2. input_type
Property Description
Attributes |
Description |
---|---|
Parameter path |
|
Parameter role |
Specify the type of the input model
By default, the compiler will automatically infer the model file name by its suffix. Sometimes the inferred result may not be what is expected
|
Parameter type |
Enum |
Optional list |
|
Caution |
Note that enum parameter values do not need to be quoted |
Code example
1# input_type example
2
3input_type: INPUT_TYPE_ONNX
7.2.1.3. output_path
Property Description
Attributes |
Description |
---|---|
Parameter path |
|
Parameter role |
Specifies the path to the output model |
Parameter type |
String |
Optional list |
/ |
Notes |
Same as input_path |
Code example
1# output_path example
2
3output_path: "./compiled.joint"
7.2.1.4. output_type
Property Description
Attributes |
Description |
---|---|
Parameter path |
|
Parameter role |
Specifies the type of the output model |
parameter type |
Enum |
Optional list |
|
Caution |
Note that enum parameter values do not need to be quoted |
Code example
1# output_type example
2
3output_type: OUTPUT_TYPE_JOINT
7.2.2. target_hardware
Property Description
Attributes |
Description |
---|---|
parameter path |
|
Parameter role |
Specifies the hardware platform for which the compiled output model is to be used |
Parameter type |
Enum |
Optional list |
TARGET_HARDWARE_AX630 TARGET_HARDWARE_AX620 |
Caution |
None |
Code example
1# target_hardware example
2
3target_hardware: TARGET_HARDWARE_AX630
Tip
It is recommended to specify the hardware platform in the command line arguments to avoid model conversion errors due to the hardware platform.
7.2.3. tensor_conf
7.2.3.1. Overview
Note
The Pulsar
toolchain has the ability to adjust the properties of the input/output tensor
of the output model,
i.e., allow the output model (e.g., joint
model) to have input and output data properties (e.g., image size, color space, etc.) that do not match those of the original input model (e.g., onnx
model).
tensor_conf configuration includes src_input_tensors
, src_output_tensors
, dst_input_tensors
, dst_output_tensors
.
Property Description
Attributes |
Description |
---|---|
Parameter path |
|
Parameter role |
src_input_tensors for the input tensor attribute of the description (description) input modelsrc_output_tensors for describing (description) the output tensor property of the input modeldst_input_tensors for modifying the input tensor properties of the output modeldst_output_tensors for modifying the output model’s output tensor property |
parameter type |
Struct |
optional list |
/ |
Cautions |
None |
7.2.3.2. Optional list
7.2.3.2.1. tensor_name
Properties |
Description |
---|---|
Parameter name |
|
Parameter role |
Specifies the name of the |
Parameter type |
String |
Optional list |
/ |
Caution |
For each of the arrays |
7.2.3.2.2. color_space
Properties |
Description |
---|---|
Parameter name |
|
Parameter role |
Used to describe the color space of the |
Parameter type |
Enum |
Enum - optional list |
TENSOR_COLOR_SPACE_BGR TENSOR_COLOR_SPACE_RGB TENSOR_COLOR_SPACE_GRAY TENSOR_COLOR_SPACE_NV12 TENSOR_COLOR_SPACE_NV21 TENSOR_COLOR_SPACE_BGR0 TENSOR_COLOR_SPACE_AUTO DEFAULT:
TENSOR_COLOR_SPACE_AUTO , auto-identify based on model input channel number: 3-channel: BGR; 1-channel: GRAY |
Caution |
None |
7.2.3.2.3. data_type
Properties |
Description |
---|---|
Parameter name |
|
Parameter role |
Specifies the data type of the input and output |
Parameter type |
Enum |
Optional list |
DATA_TYPE_UNKNOWN UINT2 INT2 MINT2 UINT4 MINT4 UINT8 INT8 MINT8 UINT16 INT16 FLOAT32 DEFAULT:
UINT8 is the default value for the input tensor , `FLOAT32 is the default value for the output tensor |
Caution |
None |
7.2.3.2.4. quantization_value
An integer, often referred to as the Q
value. It takes effect when configured as a positive number, or as a recommended value if one of the following conditions is met
The source model outputs real, the target model outputs integer
Source model input real, target model input integer
Code example
# Configure Q values
dst_output_tensors {
data_type: INT16
quantization_value: 256 # dynamic Q value when not configured
}
Hint
The Q
value can be understood as a special affine
operation. The Q
value actually represents a scale
, which can be converted to a specified fixed-point value field by dividing the output of the real number field by sclae
.
into a specified fixed-point value field.
Note
- There are two kinds of
Q
values: Dynamic
Q
values are calculated dynamically from the maximum and minimum ranges in thecalibration
data set.Static
Q
values are usuallyscale
values that are manually specified by the user based on a priori information.
Hint
The joint
model contains information about the Q
value, and the specific Q
value is printed when run_joint
is run.
Attention
Using the Q
value on the AX630
saves a cpu affine
operation, and therefore allows for speedup. The AX620
supports float
output, so even with the Q
value, there is no speedup.
7.2.3.2.5. color_standard
Properties |
Description |
---|---|
Parameter name |
|
Parameter role |
Used to set the color space standard |
Parameter type |
Enum |
Optional list |
CSC_LEGACY CSS_ITU_BT601_STUDIO_SWING CSS_ITU_BT601_FULL_SWING DEFAULT:
CSC_LEGACY |
Caution |
None |
7.2.3.2.6. tensor_layout
Properties |
Description |
---|---|
Parameter name |
|
Parameter role |
Used to modify the data layout |
Parameter type |
Enum |
Optional list |
NHWC NCHW NATIVE Default, not recommended |
Notes |
None |
Code example
1# target_hardware example
2
3src_input_tensors {
4 color_space: TENSOR_COLOR_SPACE_AUTO
5}
6dst_output_tensors {
7 color_space: TENSOR_COLOR_SPACE_NV12
8}
7.2.4. CPU subgraph settings
Note
AXEngine
is AXera
’s own inference library, which can improve the FPS
of the model to some extent, essentially replacing the CPU
subgraph of ONNX
with the AXE
subgraph, and in terms of memory usage, the memory usage of the AXE
subgraph on some models will be significantly reduced, and in the worst case, it will be the same as the original ONNX
.
Properties |
Description |
---|---|
Parameter path |
|
Parameter role |
Controls the |
Parameter type |
Struct |
Optional list |
/ |
Caution |
If you need to make a |
Code example
1cpu_backend_settings {
2 onnx_setting {
3 mode: ENABLED
4 }
5 axe_setting {
6 mode: ENABLED
7 axe_param {
8 optimize_slim_model: true
9 }
10 }
11}
Field Description
field name |
Parameter path |
Parameter Type |
Parameter role |
model |
Notes |
---|---|---|---|---|---|
|
cpu_backend_settings.onnx_setting |
Struct |
Controls whether the |
DEFAULT / ENABLED / DISABLED, default is DEFAULT |
DEFAULT and ENABLED are equivalent for ONNX |
|
cpu_backend_settings.axe_setting |
Struct |
Controls whether the |
DEFAULT / ENABLED / DISABLED, default is DEFAULT |
AXEngine’s DEFAULT is equivalent to DISABLED |
|
cpu_backend_settings.axe_setting.axe_param.optimize_slim_model |
Bool |
Indicates whether optimization mode is enabled |
No |
Recommended when the network output feature map is small, otherwise not recommended |
Important
Users are recommended to use the CPU
backend of AXE
more often (the model initial
is faster and better optimized for speed), the current ONNX
backend support is for compatibility with older versions of the toolchain and will be deprecated in future releases.
7.2.5. neuwizard_conf
neuwizard_conf
contains a variety of configuration information, which can be configured to meet a variety of needs.
7.2.5.1. operator_conf
Note
The operator_conf
can be configured for input and output capping operations, where an additional capping operator appends an operation to the input or output tensor
of an existing operator; in the configuration file, the process of adding a capping operator is done by expanding or modifying the properties of the input or output tensor
of an existing operator.
Input-output capping operators enable pre-processing and post-processing of tensor
Algorithm list |
Type |
Description |
---|---|---|
|
Struct |
Preprocessing operator, used to preprocess the input data for the model |
|
Struct |
Post-processing operator, used to post-process the output data |
Code examples
1# Example code, cannot be copied and used directly
2neuwizard_conf {
3 operator_conf {
4 input_conf_items {
5 selector {
6 ...
7 selector { ... }
8 attributes {
9 # Array of preprocessing operators
10 ...
11 }
12 }
13 output_conf_items {
14 selector {
15 ...
16 }
17 attributes {
18 # Array of post-processing operators
19 ...
20 }
21 }
22 }
23}
7.2.5.1.1. Preprocessing and Preprocessing Operators
Parameter paths
neuwizard_conf.operator_conf.input_conf_items
Example code
1# Note that by parameter path, the following is placed in the appropriate location in the configuration file
2input_conf_items {
3 # selector to indicate which input tensor the additional preprocessor operator will act on
4 selector {
5 op_name: "inp" # The name of the input tensor
6 }
7 # attributes to wrap the cap operator on "inp"
8 attributes {
9 input_modifications {
10 # do an affine operation on the input data, which changes the input data type of the compiled model from floating point [0, 1) to uint8
11 affine_preprocess {
12 slope: 1
13 slope_divisor: 255
14 bias: 0
15 }
16 }
17 }
18}
Attention
affine
is essentially a * k + b
operation.
The affine
operation in affine_preprocess
is counter-intuitive, for example, changing the type of a floating-point field [0, 1) to UINT8 [0, 255] requires dividing by 255
instead of multiplying by 255
,
while converting [0, 255] to floating point [0, 1] requires multiplying by 255
(configuring slope_divisor as 0.00392156862745098
).
input_conf_items.selector
Property Description
Properties |
Description |
---|---|
parameter name |
|
Parameter path |
|
Parameter role |
Name of the input tensor on which the additional preprocessing operator will act |
field description |
op_name specifies the full name of the input tensor. For example, “inp”op_name_regex specifies a regular expression that will be used to adapt multiple tensors. The corresponding cap operator in the attributes structure will be applied to all adapted tensors |
Code Example
1# input_conf_items.selector 示例
2selector {
3 op_name: "inp"
4}
input_conf_items.attributes
属性说明
Properties |
Description |
---|---|
parameter name |
|
parameter path |
|
parameter type |
Struct |
Parameter role |
Used to describe changes to the attributes of the input |
field description |
type : Specifies or modifies the data type of the input tensor . Enumeration type, default value DATA_TYPE_UNKNOWN input_modifications : Array of preprocessing operators, capping operators added to the input tensor. There are several of them, you can specify more than one at the same time |
where type
is an enumeration type, click here to see the supported types. input_modifications
is specified as follows:
Properties |
Description |
---|---|
field name |
|
Type |
Struct |
Function |
Array of preprocessing operators that act on a particular input |
Caution |
All operators in the array of preprocessing operators are executed sequentially, with the second operator in the array taking the output of the previous operator as input, and so on |
Preprocessing operator
The preprocessing operators include input_normalization
and affine_preprocess
.
operator name |
|
---|---|
parameter path |
neuwizard_conf.operator_conf.input_conf_items.attributes.input_modifications.input_normalization |
field descriptions |
|
Effects |
Implementation \(y = (x - mean) / std\) . |
Caution: |
If the above variables are equal to
TENSOR_COLOR_SPACE_AUTO / TENSOR_COLOR_SPACE_BGR then the order of mean/std is BGR .If the above variables are equal to
TENSOR_COLOR_SPACE_RGB then the order of mean/std is RGB |
the name of the operator |
|
---|---|
parameter path |
neuwizard_conf.operator_conf.input_conf_items.attributes.input_modifications.affine_preprocess |
field descriptions |
slope : an array of floating point numbersslope_divisor : an array of floating point numbersbias : array of floating point numbers. The length of the array is the same as slope .bias_divisor : Array of floating point numbers. The length of the array is the same as slope . |
Effects |
Implementation \(y = x * (slope / slope\_divisor) + (bias / bias\_divisor)\) . |
Caution: |
None |
Code example
1# Change the input data type from the number field {k / 255} (k=0, 1, ... , 255) to the integer field [0, 255], expecting the compiled model input data type to be uint8
2affine_preprocess {
3 slope: 1
4 slope_divisor: 255
5 bias: 0
6}
7.2.5.1.2. Postprocessing and Postprocessing Operators
Parameter path
neuwizard_conf.operator_conf.output_conf_items
Code example
1# Note that by parameter path, the following is placed in the appropriate location in the configuration file
2output_conf_items {
3 # selector to indicate the output tensor
4 selector {
5 op_name: "oup" # The name of the output tensor
6 }
7 # attributes for wrapping the cap operator on "oup"
8 attributes {
9 output_modifications {
10 # do an affine operation on the output data to change the output data type of the compiled model from floating point [0, 1) to uint8
11 affine_preprocess {
12 slope: 1
13 slope_divisor: 255
14 bias: 0
15 }
16 }
17 }
18}
output_conf_items.selector
same as input_conf_items.selector , output_conf_items.attributes
same as input_conf_items.attribute .
Postprocessing operator
Postprocessing operator affine_preprocess
.
operator name |
Operator description |
---|---|
affine_preprocess |
Do the |
The rest is the same as input_modifications.affine_preprocess
7.2.5.2. dataset_conf_calibration
operator name |
|
---|---|
parameter path |
neuwizard_conf.dataset_conf_calibration |
Role |
To describe the dataset needed for the calibration process |
Caution: |
The default |
Code example
1dataset_conf_calibration {
2 path: "... /imagenet-1k-images.tar" # needs to be replaced with your own quantified data
3 type: DATASET_TYPE_TAR # The type is tar
4 size: 256 # An integer to represent the size of the dataset, which will be randomly sampled from the full set
5 batch_size: 32 # An integer to represent the batch_size of the data used for internal parameter training, calibration or error detection during the model transfer process, default value is 32
6}
7.2.6. pulsar_conf
Property Description
Attributes |
Description |
---|---|
parameter path |
|
Parameter role |
Configuration parameters for the compiler sub-tool Used to instruct |
parameter types |
Struct |
optional list |
/ |
Caution |
Be careful to follow the path of the parameters to the correct location in the configuration file |
Code examples
1pulsar_conf {
2 ax620_virtual_npu: AX620_VIRTUAL_NPU_MODE_111 # Compiled model uses virtual core #1 of ax620 virtual NPU 1+1 mode
3 batch_size_option: BSO_DYNAMIC # The compiled model supports dynamic batch
4 batch_size: 1
5 batch_size: 2
6 batch_size: 4 # Maximum batch_size is 4; requires high performance for inference with batch_size of 1 2 or 4
7}
Structure field descriptions
field name |
Parameter Path |
Parameter Type |
Parameter role |
Optional list |
Notes |
---|---|---|---|---|---|
|
pulsar_conf.virtual_npu |
Enum |
Specifies the |
VIRTUAL_NPU_MODE_AUTO VIRTUAL_NPU_MODE_0 VIRTUAL_NPU_MODE_311 VIRTUAL_NPU_MODE_312 VIRTUAL_NPU_MODE_221 VIRTUAL_NPU_MODE_222 DEFAULT:
VIRTUAL_NPU_MODE_AUTO |
MODE_0 means no virtual NPU is used
This configuration item needs to be used if
PulsarConfiguration.target_hardware is specified as TARGET_HARDWARE_AX630 .This configuration item is used with
ax620_virtual_npu . |
|
pulsar_conf.ax620_virtual_npu |
Enum |
Specifies the |
AX620_VIRTUAL_NPU_MODE_AUTO AX620_VIRTUAL_NPU_MODE_0 AX620_VIRTUAL_NPU_MODE_111 AX620_VIRTUAL_NPU_MODE_112 |
MODE_0 means no virtual NPU is used
This configuration item needs to be used if
PulsarConfiguration.target_hardware is specified as TARGET_HARDWARE_AX620 .This configuration item is used with virtual_npu
|
|
pulsar_conf.batch_size_option |
Enum |
Sets the |
BSO_AUTO BSO_STATIC # Static batch , fixed batch_size for inference, optimal performanceBSO_DYNAMIC # dynamic batch , supports arbitrary batch_size up to the maximum value when inferring, more flexible to useDEFAULT:
BSO_AUTO , default is static batch |
None |
|
pulsar_conf.batch_size |
IntArray |
Sets the |
/ |
When
batch_size_option is specified as BSO_STATIC , batch_size indicates the unique batch size that the joint format model can use when reasoningWhen
batch_size_option is specified as BSO_DYNAMIC , batch_size indicates the maximum batch size that can be used for joint format model inference.When generating a
joint format model that supports dynamic batch , multiple values can be configured to improve performance when reasoning with batch size up to these valuesIncrease the size of
joint format model files when multiple batch sizes are specifiedbatch_size_option will default to BSO_DYNAMIC when multiple batch_sizes are configured |
7.3. Parameters that can be passed via the command line
Hint
Command line arguments override some of the corresponding configuration in the configuration file, and are only used to help with more complex functions that can be implemented through the configuration file.
Parameters |
Description |
---|---|
input |
Input model path |
output |
Output model path |
calibration_batch_size |
The batch_size of the calibration dataset |
batch_size_option |
{BSO_AUTO,BSO_STATIC,BSO_DYNAMIC} |
output_dir |
Specify the output directory |
virtual_npu |
Specify the virtual NPU |
input_tensor_color |
{auto,rgb,bgr,gray,nv12,nv21} |
output_tensor_layout |
{native,nchw,nhwc} |
color_std |
{studio,full} only support nv12/nv21 now |
target_hardware |
{AX630,AX620,AX170} target hardware to compile |
enable_progress_bar |
Whether to print progress bar, not enabled by default |
7.4. config.prototxt Minimal configuration
Example of simplest_config.prototxt, which can be copied directly into a file and run.
1# simplest_config.prototxt example, can be copied directly into the file and run
2input_type: INPUT_TYPE_ONNX # Specifies that the input model is of type onnx, if this field is omitted, the compiler will automatically infer the model file by its suffix, however, sometimes the inference may not be the desired result
3output_type: OUTPUT_TYPE_JOINT # Specifies that the output model is of type Joint
4src_input_tensors { # Attributes of the input tensor used to describe the input model
5 color_space: TENSOR_COLOR_SPACE_AUTO # The compiler determines the color space itself
6}
7dst_input_tensors { # Attributes of the input tensor used to modify the output model
8 color_space: TENSOR_COLOR_SPACE_AUTO # The compiler determines the color space itself
9}
10neuwizard_conf { # neuwizard parameter configuration
11 operator_conf { # input_output_capping_configuration: additional input and output capping operators add an operation to the input or output tensor of an existing operator; in the configuration file, the process of adding a capping operator is done by expanding or modifying the properties of the input or output tensor of an existing operator
12 input_conf_items { # used to preprocess the input data for the model
13 attributes { # describe changes to the attributes of the input tensor, the target input tensor is specified by input_conf_items.selector, not specified, default is ?
14 input_modifications { # array of preprocessing operators, cap operators to be added to the input tensor, there are several, you can specify more than one, all operators in the preprocessing array are executed sequentially, the second operator in the array is the output of the previous operator as input, and so on
15 affine_preprocess { # do an affine (i.e. x * k + b) operation on the input data, which changes the input data type of the compiled model from floating point [0, 1) to uint8
16 slope: 1 # Array of floating point numbers. The length of the array is equal to 1 or the number of channels of the data. When the length is 1, the compiler will automatically copy the channel times
17 slope_divisor: 255 # Array of floating point numbers. The length of the array is the same as slope
18 bias: 0 # Array of floating point numbers. The length of the array is the same as slope
19 # The effect is the same as: y = x * (slope / slope_divisor) + (bias / bias_divisor)
20 }
21 }
22 }
23 }
24 }
25 dataset_conf_calibration {
26 path: ". /imagenet-1k-images.tar" # A tarball with 1000 images, used to calibrate the model during compilation
27 type: DATASET_TYPE_TAR # The type is tar
28 size: 256 # indicates the size of the dataset, which will be randomly sampled from the full set, batch_size defaults to 32
29 batch_size defaults to 32.}
30}
31pulsar_conf { # pulsar compiler parameters configuration
32 batch_size: 1 # Set the batch size supported by the joint format model, default is 1
33}