量化tcn模型的时候，报错input does not bind exponents parameter (AIS-2061)

[Uniquelock]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

希望可以正确通过量化过程，导出对应的espdl文件等

Actual behavior (suspected bug)

从最开始的两个inputs（input和cache）就无法绑定量化参数。

Error logs or terminal output

(idf5.3_py3.8_env) xcl@xcl:~/esp32/esp-dl/examples/tutorial/how_to_quantize_model$ python /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py --config /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/kws_model_cfg.toml -o /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model --target esp32s3

    ___________ ____        ____  ____  ____ 
   / ____/ ___// __ \      / __ \/ __ \/ __ \
  / __/  \__ \/ /_/ /_____/ /_/ / /_/ / / / /
 / /___ ___/ / ____/_____/ ____/ ____/ /_/ / 
/_____//____/_/         /_/   /_/    \___\_\ 


/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/kws_model.py:465: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  model_py.load_state_dict(torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/avg_20.pt"))
tensor([[[4.7930e-06, 4.4210e-06]]], grad_fn=<SigmoidBackward0>)
Input 0: name=input, shape=[1, 40, 1]
Input 1: name=cache, shape=[1, 105, 64]
/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py:508: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  input_dataset = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted_all.pt")
[06:15:02] PPQ Quantization Fusion Pass Running ...       Finished.
[06:15:02] PPQ Quantize Simplify Pass Running ...         Finished.
[06:15:02] PPQ Parameter Quantization Pass Running ...    Finished.
Calibration Progress(Phase 1):  31%|█████████████████████████▋                                                        | 5Calibration Progress(Phase 1):  38%|██████████████████████████████▊                                                   | 6Calibration Progress(Phase 1):  44%|███████████████████████████████████▉                                              | 7Calibration Progress(Phase 1):  50%|█████████████████████████████████████████                                         | 8Calibration Progress(Phase 1):  56%|██████████████████████████████████████████████▏                                   | 9Calibration Progress(Phase 1):  62%|██████████████████████████████████████████████████▋                              | 10Calibration Progress(Phase 1):  69%|███████████████████████████████████████████████████████▋                         | 11Calibration Progress(Phase 1):  75%|████████████████████████████████████████████████████████████▊                    | 12Calibration Progress(Phase 1):  81%|█████████████████████████████████████████████████████████████████▊               | 13Calibration Progress(Phase 1):  88%|██████████████████████████████████████████████████████████████████████▉          | 14Calibration Progress(Phase 1):  94%|███████████████████████████████████████████████████████████████████████████▉     | 15Calibration Progress(Phase 1): 100%|█████████████████████████████████████████████████████████████████████████████████| 16Calibration Progress(Phase 1): 100%|█████████████████████████████████████████████████████████████████████████████████| 16/16 [00:09<00:00,  1.70it/s]
Calibration Progress(Phase 2): 100%|█████████████████████████████████████████████████████| 16/16 [00:42<00:00,  2.68s/it]
Finished.
[06:16:26] PPQ Quantization Alignment Pass Running ...    Finished.
[06:16:26] PPQ Passive Parameter Quantization Running ... Finished.
--------- Network Snapshot ---------
Num of Op:                    [43]
Num of Quantized Op:          [37]
Num of Variable:              [99]
Num of Quantized Var:         [78]
------- Quantization Snapshot ------
Num of Quant Config:          [121]
ACTIVATED:                    [27]
OVERLAPPED:                   [40]
PASSIVE:                      [38]
FP32:                         [16]
Network Quantization Finished.
Analysing Graphwise Quantization Error(Phrase 1):: 100%|███████████████████████████████████| 8/8 [00:03<00:00,  2.02it/s]
Analysing Graphwise Quantization Error(Phrase 2):: 100%|███████████████████████████████████| 8/8 [00:06<00:00,  1.23it/s]
Layer                            | NOISE:SIGNAL POWER RATIO 
/backbone/network.2/cnn.3/Conv:  | ████████████████████ | 3.579%
/backbone/network.1/cnn.3/Conv:  | ███████████          | 1.918%
/preprocessing/out/out.0/MatMul: | ███                  | 0.534%
/backbone/network.3/cnn.3/Conv:  | █                    | 0.226%
/backbone/network.3/cnn.0/Conv:  | █                    | 0.140%
/backbone/network.0/cnn.0/Conv:  |                      | 0.105%
/classifier/linear/MatMul:       |                      | 0.086%
/backbone/network.2/cnn.0/Conv:  |                      | 0.078%
/backbone/network.0/cnn.3/Conv:  |                      | 0.077%
/backbone/network.1/cnn.0/Conv:  |                      | 0.035%
Analysing Layerwise quantization error:: 100%|███████████████████████████████████████████| 10/10 [01:13<00:00,  7.33s/it]
Layer                            | NOISE:SIGNAL POWER RATIO 
/preprocessing/out/out.0/MatMul: | ████████████████████ | 0.005%
/backbone/network.0/cnn.3/Conv:  | ███████              | 0.002%
/backbone/network.0/cnn.0/Conv:  | ███                  | 0.001%
/backbone/network.1/cnn.3/Conv:  | ███                  | 0.001%
/backbone/network.2/cnn.3/Conv:  | ██                   | 0.000%
/backbone/network.1/cnn.0/Conv:  | █                    | 0.000%
/classifier/linear/MatMul:       |                      | 0.000%
/backbone/network.2/cnn.0/Conv:  |                      | 0.000%
/backbone/network.3/cnn.3/Conv:  |                      | 0.000%
/backbone/network.3/cnn.0/Conv:  |                      | 0.000%
test_inputs_shape: torch.Size([1, 1, 40]) torch.Size([1, 64, 105])
test_outputs_shape: torch.Size([1, 1, 2]) torch.Size([1, 64, 105])
test_inputs_value: {'input': tensor([[[ 8.8873,  8.1619,  8.0626,  9.1510,  9.7362,  9.8662, 11.0428,
          10.6253,  9.4686, 12.0472, 12.7222, 11.7752, 11.8694, 12.9231,
          13.7552, 12.9735, 13.2906, 13.9537, 13.7025, 12.6923, 13.9152,
          13.3460, 14.1537, 14.5433, 14.8616, 14.3582, 15.3774, 15.3107,
          14.6493, 13.8590, 13.8440, 14.5063, 15.1803, 15.0804, 14.6082,
          13.6197, 13.4577, 13.1789, 13.4074, 13.5816]]]), 'cache': tensor([[[1.2228, 1.3030, 1.2440,  ..., 1.2343, 1.2358, 1.3491],
         [0.4020, 0.3417, 0.3197,  ..., 0.8657, 0.6790, 1.3203],
         [0.0000, 0.0000, 0.0000,  ..., 0.3191, 0.5939, 0.6753],
         ...,
         [0.5019, 0.5771, 0.5291,  ..., 0.6183, 0.4323, 0.3718],
         [0.0000, 0.0000, 0.1158,  ..., 0.1437, 0.3306, 0.7760],
         [0.0000, 0.0000, 0.0000,  ..., 0.0404, 0.0640, 0.1074]]])}
test_outputs_value: {'output': tensor([[[1.4305e-06, 3.0279e-05]]]), 'r_cache': tensor([[[1.0000, 1.0000, 1.0000,  ..., 1.2500, 1.3438, 1.0000],
         [0.3438, 0.3125, 0.2812,  ..., 0.6875, 1.3125, 1.4688],
         [0.0000, 0.0000, 0.0000,  ..., 0.5938, 0.6875, 1.0625],
         ...,
         [0.5938, 0.5312, 0.5312,  ..., 0.4375, 0.3750, 0.3750],
         [0.0000, 0.1250, 0.0000,  ..., 0.3438, 0.7812, 0.4062],
         [0.0000, 0.0000, 0.0000,  ..., 0.0625, 0.0938, 0.1250]]])}
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.0/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_6 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_19 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.1/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_7 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_21 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.2/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_23 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.3/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_25 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_17 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.0/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_6 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_19 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.1/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_7 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_21 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.2/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_23 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.3/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_25 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_17 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.0/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_6 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_19 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.1/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_7 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_21 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.2/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_23 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.3/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_25 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_17 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.0/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_6 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_19 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.1/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_7 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_21 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.2/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_23 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  skip not QuantableOperation
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip /backbone/network.3/Constant_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_25 because it's not exportable
[INFO][ESPDL][2025-06-25 06:17:51]:  Skip PPQ_Variable_17 because it's not exportable
[WARNING][ESPDL][2025-06-25 06:17:51]:  /backbone/Slice_3_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  /backbone/Slice_1_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  input does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  /global_cmvn/Mul_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  cache does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  /backbone/Slice_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  /backbone/Slice_2_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 06:17:51]:  /global_cmvn/Sub_output_0 does not bind exponents parameter
Traceback (most recent call last):
  File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py", line 553, in <module>
    quant_ppq_graph = espdl_quantize_onnx(
  File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper
    return func(*args, **kwargs)
  File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 285, in espdl_quantize_onnx
    PFL.Exporter(platform=target_platform).export(
  File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export
    graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step)
  File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 241, in prepare_graph
    if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0:
TypeError: 'NoneType' object is not subscriptable

Steps to reproduce the behavior

1. git clone了最新的esp-dl，按照tutorial中的流式模型的example修改了quantize_kws_streaming_model.py（quantize_kws_streaming_model.txt）；
2. 在文件夹下esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model运行代码 python /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py --config /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/kws_model_cfg.toml -o /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model --target esp32s3
3. 好像不能上传onnx文件，我的模型结构如图，因为一直没有办法绑定上参数，我是新手，又不知道哪里有问题，问了gpt，在输入后加了个恒等的conv算子，结果是可以绑定上，但是量化的时候误差达到了100%，测试结果对不上。
4. 我的校准数据集总共有1097条[input,cache]组成。其中input的shape为[1,1,40];cache的shape为[1,64,105]。
5. 还有一个问题就是，加了恒等算子之后量化出来的模型的输入维度调换变成[1,40,1],[1,105,64]；如果flash到板子上，cpp代码中的输入是不是也要reshape一下？

quantize_kws_streaming_model.txt

Project release version

latest

System architecture

Intel/AMD 64-bit (modern PC, older Mac)

Operating system

Linux

Operating system version

Ubuntu 20.04.6 LTS

Shell

Bash

Additional context

这个是我在两个输入后面都加上了恒等conv之后的onnx，以及terminal输出outlog信息

outlog.txt

[BlueSkyB]

可以先尝试下非流式模型，是否能正常量化导出。
将onnx文件打包成zip文件，应该就可以上传了

[Uniquelock]

您好！感谢回复！这是我的onnx模型。是不是我的导出脚本改的有问题，我直接把两个输入同时喂入了，代码片段：
inputs =[a3, cache2]
dataloader = input_dataset

ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]] # 1 个输入特征
TARGET = "esp32s3" # 目标量化平台
NUM_OF_BITS = 8 # 量化位数
DEVICE = "cpu" # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
onnx_import_file=ONNX_MODEL_PATH,
espdl_export_file=ESPDL_MODEL_PATH,
calib_dataloader=dataloader,
calib_steps=16, # 校准的步数
input_shape=INPUT_SHAPE, # 输入形状，批次为 1
inputs=inputs,
target=TARGET, # 目标量化类型
num_of_bits=NUM_OF_BITS, # 量化位数
# collate_fn=my_collate_fn,
dispatching_override=None,
device=DEVICE,
error_report=True,
skip_export=False,
export_test_values=True,
verbose=1, # 输出详细日志信息
)
avg_20.zip

[Uniquelock]

哈咯，我又来啦。刚刚试了一下非流式的。还是没有办法绑定。
(idf5.3_py3.8_env) xcl@xcl:~/esp32/esp-dl/examples/tutorial/how_to_quantize_model$ python /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py --config /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/kws_model_cfg.toml -o /home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model --target esp32s3

___________ ____        ____  ____  ____ 

/ / // __ \ / __ / __ / __
/ _/ _ / // /___/ // / // / / / /
/ /_ / / ____// ____/ ___/ // /
/_//_// // // __\

/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/kws_model.py:465: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
model_py.load_state_dict(torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/avg_20.pt"))
tensor([[[4.7930e-06, 4.4210e-06]]], grad_fn=)
/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py:508: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
input_dataset = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted_first_slice.pt")
[08:47:56] PPQ Quantization Fusion Pass Running ... Finished.
[08:47:56] PPQ Quantize Simplify Pass Running ... Finished.
[08:47:56] PPQ Parameter Quantization Pass Running ... Finished.
Calibration Progress(Phase 1): 100%|███████████████████████████████████████████████████████████████████████████████████| 16/16 [00:07<00:00, 2.09it/s]
Calibration Progress(Phase 2): 100%|███████████████████████████████████████████████████████████████████████████████████| 16/16 [00:25<00:00, 1.57s/it]
Finished.
[08:48:49] PPQ Quantization Alignment Pass Running ... Finished.
[08:48:49] PPQ Passive Parameter Quantization Running ... Finished.
--------- Network Snapshot ---------
Num of Op: [39]
Num of Quantized Op: [37]
Num of Variable: [86]
Num of Quantized Var: [82]
------- Quantization Snapshot ------
Num of Quant Config: [125]
ACTIVATED: [23]
OVERLAPPED: [48]
PASSIVE: [34]
FP32: [20]
Network Quantization Finished.
Analysing Graphwise Quantization Error(Phrase 1):: 100%|█████████████████████████████████████████████████████████████████| 8/8 [00:03<00:00, 2.63it/s]
Analysing Graphwise Quantization Error(Phrase 2):: 100%|█████████████████████████████████████████████████████████████████| 8/8 [00:05<00:00, 1.60it/s]
Layer | NOISE:SIGNAL POWER RATIO
/backbone/network.2/cnn.3/Conv: | ████████████████████ | 3.998%
/backbone/network.1/cnn.3/Conv: | ██████████ | 1.964%
/preprocessing/out/out.0/MatMul: | ███ | 0.534%
/backbone/network.3/cnn.3/Conv: | ██ | 0.491%
/backbone/network.2/cnn.0/Conv: | █ | 0.186%
/backbone/network.3/cnn.0/Conv: | █ | 0.175%
/backbone/network.0/cnn.0/Conv: | | 0.077%
/backbone/network.0/cnn.3/Conv: | | 0.043%
/classifier/linear/MatMul: | | 0.023%
/backbone/network.1/cnn.0/Conv: | | 0.017%
Analysing Layerwise quantization error:: 100%|█████████████████████████████████████████████████████████████████████████| 10/10 [01:02<00:00, 6.25s/it]
Layer | NOISE:SIGNAL POWER RATIO
/preprocessing/out/out.0/MatMul: | ████████████████████ | 0.026%
/backbone/network.0/cnn.3/Conv: | ████ | 0.005%
/backbone/network.1/cnn.3/Conv: | ██ | 0.002%
/backbone/network.0/cnn.0/Conv: | █ | 0.002%
/backbone/network.2/cnn.3/Conv: | █ | 0.001%
/backbone/network.1/cnn.0/Conv: | | 0.000%
/classifier/linear/MatMul: | | 0.000%
/backbone/network.2/cnn.0/Conv: | | 0.000%
/backbone/network.3/cnn.3/Conv: | | 0.000%
/backbone/network.3/cnn.0/Conv: | | 0.000%
test_inputs_value: {'input': tensor([[[ 8.8873, 8.1619, 8.0626, 9.1510, 9.7362, 9.8662, 11.0428,
10.6253, 9.4686, 12.0472, 12.7222, 11.7752, 11.8694, 12.9231,
13.7552, 12.9735, 13.2906, 13.9537, 13.7025, 12.6923, 13.9152,
13.3460, 14.1537, 14.5433, 14.8616, 14.3582, 15.3774, 15.3107,
14.6493, 13.8590, 13.8440, 14.5063, 15.1803, 15.0804, 14.6082,
13.6197, 13.4577, 13.1789, 13.4074, 13.5816]]])}
test_outputs_value: {'output': tensor([[[1.4782e-05, 1.2994e-05]]]), '239': tensor([[[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 1.0000],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 1.5625],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.6562],
...,
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 1.1250],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 1.6250],
[0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.4375]]])}
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_12 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_1 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_2 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_14 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_10 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_3 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_12 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_1 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_2 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_14 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_10 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_3 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_12 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_1 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_2 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_14 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_10 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_3 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_11 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.0/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_12 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_8 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.1/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_1 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_13 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_9 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.2/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_2 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_14 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_10 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Reshape_1_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: skip not QuantableOperation
[INFO][ESPDL][2025-06-25 08:50:00]: Skip /backbone/network.3/Constant_10_output_0 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_3 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_15 because it's not exportable
[INFO][ESPDL][2025-06-25 08:50:00]: Skip PPQ_Variable_11 because it's not exportable
[WARNING][ESPDL][2025-06-25 08:50:00]: input does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 08:50:00]: /global_cmvn/Sub_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-25 08:50:00]: /global_cmvn/Mul_output_0 does not bind exponents parameter
Traceback (most recent call last):
File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model.py", line 554, in
quant_ppq_graph = espdl_quantize_onnx(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper
return func(*args, **kwargs)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 285, in espdl_quantize_onnx
PFL.Exporter(platform=target_platform).export(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export
graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 241, in prepare_graph
if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0:
TypeError: 'NoneType' object is not subscriptable

avg_20_nonstream.zip

[BlueSkyB]

传给espdl_quantize_onnx接口的参数calib_dataloader，需要是pytorch的DataLoader类型，不了解的话，可以网络搜索下概念。
我量化你的流式模型，没有什么问题。可以再多对比下多个量化脚本示例。

` input_dataset = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted_all.pt")
inputs =[a3, cache2]
dataloader = input_dataset # 用 list 代替 DataLoader

ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=16,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    # collate_fn=my_collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=True,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

[Uniquelock]

我按照你的建议修改了量化脚本，quantize_kws_streaming_model1.txt是按照例子中的quantize_streaming_model.py修改的，除了增加了模型load代码和squeeze了我的input数据，其他都没有修改。但是还是报错。代码如下，链接也是。

-- coding: utf-8 --

import argparse
import os
import re
import sys
from typing import Iterable, List, Tuple, Union, Any, Optional

from ppq import TargetPlatform
import onnx
import torch
import toml
import torch.nn.functional as F
from ppq import QuantizationSettingFactory
from ppq.api import (
espdl_quantize_onnx,
espdl_quantize_torch,
get_target_platform,
generate_test_value,
)
from torch.utils.data import TensorDataset, DataLoader
from torch import Tensor, nn
from test_model import TestModel_0, TestModel_1, TestModel_2
import onnxruntime as ort
import numpy as np

from kws_model import init_model

torch.manual_seed(42)

from torch.utils.data import Dataset

class QuantModel:
def init(self, model, args, meta_config, model_config, dataset):
if not os.path.exists(args.output_path):
os.makedirs(args.output_path)
self.export_path = args.output_path

    self.model = model
    self.meta_config = meta_config
    self.model_config = model_config
    self.dataset = dataset
    self.onnx_file = None

    if isinstance(self.model, onnx.ModelProto):
        self.onnx_file = os.path.join(
            self.export_path, self.model_config["export_name_prefix"] + ".onnx"
        )
        onnx.save(self.model, self.onnx_file)

    # config device
    self.device_str = args.device
    self.calib_steps = (
        self.meta_config["calib_steps"] if self.meta_config is not None else 32
    )
    self.batch_size = (
        self.meta_config["batch_size"] if self.meta_config is not None else 1
    )
    self.streaming = args.streaming
    self.streaming_window_size = 0
    self.streaming_cache_size = 0
    if self.streaming:
        if self.model_config.get("streaming_input_shape", []):
            self.streaming_window_size = self.model_config["streaming_input_shape"][
                2
            ]
        if self.model_config.get("streaming_cache_shape", []):
            self.streaming_cache_size = self.model_config["streaming_cache_shape"][
                2
            ]

    self.offline_input_shape = self.model_config["offline_input_shape"]
    if not isinstance(self.offline_input_shape[0], list):
        self.multi_input = False
    else:
        self.multi_input = True

    # get calibration dataset.
    calibration_dataset = self.load_calibration_dataset()
    self.calib_dataloader = DataLoader(
        dataset=calibration_dataset,
        batch_size=self.batch_size,
        shuffle=False,
    )

    # get quantization config.
    self.num_of_bits = args.bits
    self.model_version = args.version
    self.target = args.target
    self.input_dtype = torch.float

def load_calibration_dataset(self) -> Iterable:
    if self.streaming:
        data_total = []
        if self.model_config.get("streaming_cache_shape", []):
            caches = []
            caches.append(
                torch.zeros(size=self.model_config["streaming_cache_shape"][1:])
                # torch.zeros(size=self.model_config["streaming_cache_shape"])
            )
            if not self.multi_input:
                for data in self.dataset:
                    # Ensure that the size of the W dimension is divisible by self.streaming_window_size.
                    data = data.squeeze(0)
                    split_tensors = torch.split(
                        data[0] if isinstance(data, tuple) else data,
                        self.streaming_window_size,
                        dim=1,
                    )
                    # print(split_tensors)
                    for index, split_tensor in enumerate(split_tensors):
                        _, cache = self.model(
                            split_tensor.unsqueeze(0), caches[len(caches)-1].unsqueeze(0)
                        )
                        caches.append(cache.squeeze(0))
                    data_total += [
                        list(pair) for pair in zip(list(split_tensors), caches)
                    ]   
                
            else:
                # It depends on which inputs of the model require streaming, so multiple inputs have not been added.
                pass
            return data_total
        else:
            if not self.multi_input:
                for data in self.dataset:
                    # Ensure that the size of the W dimension is divisible by self.streaming_window_size.
                    # Split the input and collect cache data.
                    split_tensors = torch.split(
                        data[0] if isinstance(data, tuple) else data,
                        self.streaming_window_size,
                        dim=1,
                    )
                    data_total += list(split_tensors)
            else:
                pass
            return data_total
    else:
        return self.dataset

def __call__(self):
    # get the export files path
    name_prefix = (
        self.model_config["export_name_prefix"] + "_s" + "_example" + str(self.num_of_bits)
    )
    if self.streaming:
        name_prefix += "_streaming"
    espdl_export_file = os.path.join(self.export_path, name_prefix + ".espdl")

    collate_fn = lambda x: (
        x.type(self.input_dtype).to(self.device_str)
        if not isinstance(x, list)
        else [xx.type(self.input_dtype).to(self.device_str) for xx in x]
    )

    print("start PTQ")
    # create a setting for quantizing your network with ESPDL.
    quant_setting = QuantizationSettingFactory.espdl_setting()
    if self.model_config.get("dispatch_table", {}):
        for op_name, num_bit in self.model_config.get("dispatch_table", {}).items():
            quant_setting.dispatching_table.append(
                op_name, get_target_platform(self.target, num_bit)
            )
    # quant_setting.dispatcher = "allin"

    # Batch size must be 1.
    export_input_shape = self.offline_input_shape
    if self.streaming:
        if self.model_config.get("streaming_cache_shape", []):
            export_input_shape = [
                [1] + self.model_config["streaming_input_shape"][1:],
                [1] + self.model_config["streaming_cache_shape"][1:],
            ]
        else:
            export_input_shape = [1] + self.model_config["streaming_input_shape"][
                1:
            ]
    print("dataloader length:", len(self.calib_dataloader))

    ppq_graph = None
    if self.onnx_file is None:
        ppq_graph = espdl_quantize_torch(
            model=self.model,
            espdl_export_file=espdl_export_file,
            calib_dataloader=self.calib_dataloader,
            calib_steps=self.calib_steps,
            input_shape=export_input_shape,
            target=self.target,
            num_of_bits=self.num_of_bits,
            collate_fn=collate_fn,
            setting=quant_setting,
            device=self.device_str,
            error_report=True,
            skip_export=False,
            export_test_values=True,
            export_config=True,
            verbose=1,
            int16_lut_step=1,
        )

    else:
        ppq_graph = espdl_quantize_onnx(
            onnx_import_file=self.onnx_file,
            espdl_export_file=espdl_export_file,
            calib_dataloader=self.calib_dataloader,
            calib_steps=self.calib_steps,
            input_shape=export_input_shape,
            target=self.target,
            num_of_bits=self.num_of_bits,
            collate_fn=collate_fn,
            setting=quant_setting,
            device=self.device_str,
            error_report=False,
            skip_export=False,
            export_test_values=False,
            export_config=True,
            verbose=1,
            int16_lut_step=1,
        )
    return ppq_graph

if name == "main":
parser = argparse.ArgumentParser(description="")
parser.add_argument(
"-c", "--config", required=True, type=str, help="Config (*.toml)."
)
parser.add_argument(
"-o", "--output-path", type=str, default="./", help="Output Path."
)
parser.add_argument(
"-t",
"--target",
type=str,
default="esp32p4",
help="esp32p4 or esp32s3, (defaults: esp32p4).",
)
parser.add_argument(
"-b",
"--bits",
type=int,
default=8,
help="the number of bits, support 8 or 16, (defaults: 8).",
)
parser.add_argument(
"-d",
"--device",
type=str,
default="cpu",
help="execution device, (defaults: cpu).",
)
parser.add_argument(
"-v",
"--version",
type=str,
default="1.0",
help="the version of the test case, (defaults: 1.0)",
)
parser.add_argument(
"--streaming",
action="store_true",
help="Whether to export the model in a streaming manner.",
)
parser.add_argument(
"--export_test_values",
action="store_true",
help="Whether to export the test inputs and outputs.",
)

args = parser.parse_args()
config = toml.load(args.config)
meta_config = config["meta"]
model_0_config = config["model_0_config"]
input_dataset = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted_first_slice.pt")

configs = {
    'cmvn': {
        'cmvn_file': '/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/global_cmvn',
        'norm_var': 'true'},
    'input_dim': 40,
    'output_dim': 2,
    'hidden_dim': 64,
    'preprocessing': {'type': 'linear'},
    'backbone': {
        'type': 'tcn',
        'num_layers': 4,
        'ds': True,
        'kernel_size': 8,
        'dropout': 0.1
    },
}

model = init_model(configs)
model.load_state_dict(torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/avg_20.pt"))
model.eval()
ppq_graph = QuantModel(
    model, args, meta_config, model_0_config, input_dataset
)()

[WARNING][ESPDL][2025-06-26 09:02:54]: /global_cmvn/Sub_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: onnx::Sub_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: onnx::Slice_1 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: /backbone/Slice_1_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: /backbone/Slice_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: /backbone/Slice_2_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: /global_cmvn/Mul_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:02:54]: /backbone/Slice_3_output_0 does not bind exponents parameter
Traceback (most recent call last):
File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model1.py", line 305, in
ppq_graph = QuantModel(
File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model1.py", line 192, in call
ppq_graph = espdl_quantize_torch(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 377, in espdl_quantize_torch
return espdl_quantize_onnx(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper
return func(*args, **kwargs)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx
PFL.Exporter(platform=target_platform).export(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export
graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph
if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0:
TypeError: 'NoneType' object is not subscriptable

quantize_kws_streaming_model1.txt

[Uniquelock]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse
import os
import re
import sys
from typing import Iterable, List, Tuple, Union, Any, Optional

from ppq import TargetPlatform
import onnx
import torch
import toml
import torch.nn.functional as F
from ppq import QuantizationSettingFactory
from ppq.api import (
espdl_quantize_onnx,
espdl_quantize_torch,
get_target_platform,
generate_test_value,
)
from torch.utils.data import TensorDataset, DataLoader
from torch import Tensor, nn
from test_model import TestModel_0, TestModel_1, TestModel_2
import onnxruntime as ort
import numpy as np

from kws_model import init_model

torch.manual_seed(42)

from torch.utils.data import Dataset

def collate_fn(batch):
return [b.type(torch.float).to(DEVICE) for b in batch]

if name == "main":
parser = argparse.ArgumentParser(description="")
parser.add_argument(
"-o", "--output-path", type=str, default="./", help="Output Path."
)
parser.add_argument(
"-t",
"--target",
type=str,
default="esp32p4",
help="esp32p4 or esp32s3, (defaults: esp32p4).",
)
parser.add_argument(
"-b",
"--bits",
type=int,
default=8,
help="the number of bits, support 8 or 16, (defaults: 8).",
)
parser.add_argument(
"-d",
"--device",
type=str,
default="cpu",
help="execution device, (defaults: cpu).",
)
parser.add_argument(
"-v",
"--version",
type=str,
default="1.0",
help="the version of the test case, (defaults: 1.0)",
)
parser.add_argument(
"--streaming",
action="store_true",
help="Whether to export the model in a streaming manner.",
)
parser.add_argument(
"--export_test_values",
action="store_true",
help="Whether to export the test inputs and outputs.",
)

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？
[WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter
[WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter
Traceback (most recent call last):
File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in
quant_ppq_graph = espdl_quantize_onnx(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper
return func(*args, **kwargs)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx
PFL.Exporter(platform=target_platform).export(
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export
graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step)
File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph
if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0:
TypeError: 'NoneType' object is not subscriptable

quantize_kws_streaming_model2.txt

[yushanyong]

这个模型conservative量化方法，默认/global_cmvn/Sub 和/global_cmvn/Mul是fp32是不量化的，不量化就不会有量化参数，所以会报这个错；而且对于esp-dl，这两个算子目前也只支持int8和int16

[BlueSkyB]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse import os import re import sys from typing import Iterable, List, Tuple, Union, Any, Optional

from ppq import TargetPlatform import onnx import torch import toml import torch.nn.functional as F from ppq import QuantizationSettingFactory from ppq.api import ( espdl_quantize_onnx, espdl_quantize_torch, get_target_platform, generate_test_value, ) from torch.utils.data import TensorDataset, DataLoader from torch import Tensor, nn from test_model import TestModel_0, TestModel_1, TestModel_2 import onnxruntime as ort import numpy as np

from kws_model import init_model

torch.manual_seed(42)

from torch.utils.data import Dataset

def collate_fn(batch): return [b.type(torch.float).to(DEVICE) for b in batch]

if name == "main": parser = argparse.ArgumentParser(description="") parser.add_argument( "-o", "--output-path", type=str, default="./", help="Output Path." ) parser.add_argument( "-t", "--target", type=str, default="esp32p4", help="esp32p4 or esp32s3, (defaults: esp32p4).", ) parser.add_argument( "-b", "--bits", type=int, default=8, help="the number of bits, support 8 or 16, (defaults: 8).", ) parser.add_argument( "-d", "--device", type=str, default="cpu", help="execution device, (defaults: cpu).", ) parser.add_argument( "-v", "--version", type=str, default="1.0", help="the version of the test case, (defaults: 1.0)", ) parser.add_argument( "--streaming", action="store_true", help="Whether to export the model in a streaming manner.", ) parser.add_argument( "--export_test_values", action="store_true", help="Whether to export the test inputs and outputs.", )

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？ [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter Traceback (most recent call last): File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in quant_ppq_graph = espdl_quantize_onnx( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper return func(*args, **kwargs) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx PFL.Exporter(platform=target_platform).export( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0: TypeError: 'NoneType' object is not subscriptable

quantize_kws_streaming_model2.txt

刚才我用你的这个脚本跑了一下，确实有问题，之前我用测试脚本跑的，没细对比。
在准备回复的时候，发现有人已经帮忙解答了。

原因如 @yushanyong 所说(对ESP-PPQ挺了解，赞一个)。
对于conservative方法，模型的第一层算子若为sub，slice之类的，没有被加入量化算子，导致没有被量化。对于dispatch方法，我们后面会重新评估一下其它方式是否更适合espdl。

针对你的问题，模型应该可以将所有算子都设为量化，可以解决你的问题。方式如下：
` quant_setting = QuantizationSettingFactory.espdl_setting()
quant_setting.dispatcher = "allin"

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=calib_dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    # dispatching_override=None,
    setting=quant_setting,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

[yushanyong]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse import os import re import sys from typing import Iterable, List, Tuple, Union, Any, Optional
from ppq import TargetPlatform import onnx import torch import toml import torch.nn.functional as F from ppq import QuantizationSettingFactory from ppq.api import ( espdl_quantize_onnx, espdl_quantize_torch, get_target_platform, generate_test_value, ) from torch.utils.data import TensorDataset, DataLoader from torch import Tensor, nn from test_model import TestModel_0, TestModel_1, TestModel_2 import onnxruntime as ort import numpy as np
from kws_model import init_model
torch.manual_seed(42)
from torch.utils.data import Dataset
def collate_fn(batch): return [b.type(torch.float).to(DEVICE) for b in batch]
if name == "main": parser = argparse.ArgumentParser(description="") parser.add_argument( "-o", "--output-path", type=str, default="./", help="Output Path." ) parser.add_argument( "-t", "--target", type=str, default="esp32p4", help="esp32p4 or esp32s3, (defaults: esp32p4).", ) parser.add_argument( "-b", "--bits", type=int, default=8, help="the number of bits, support 8 or 16, (defaults: 8).", ) parser.add_argument( "-d", "--device", type=str, default="cpu", help="execution device, (defaults: cpu).", ) parser.add_argument( "-v", "--version", type=str, default="1.0", help="the version of the test case, (defaults: 1.0)", ) parser.add_argument( "--streaming", action="store_true", help="Whether to export the model in a streaming manner.", ) parser.add_argument( "--export_test_values", action="store_true", help="Whether to export the test inputs and outputs.", )

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？ [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter Traceback (most recent call last): File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in quant_ppq_graph = espdl_quantize_onnx( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper return func(*args, **kwargs) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx PFL.Exporter(platform=target_platform).export( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0: TypeError: 'NoneType' object is not subscriptable
quantize_kws_streaming_model2.txt

刚才我用你的这个脚本跑了一下，确实有问题，之前我用测试脚本跑的，没细对比。 在准备回复的时候，发现有人已经帮忙解答了。

原因如 @yushanyong 所说(对ESP-PPQ挺了解，赞一个)。 对于conservative方法，模型的第一层算子若为sub，slice之类的，没有被加入量化算子，导致没有被量化。对于dispatch方法，我们后面会重新评估一下其它方式是否更适合espdl。

针对你的问题，模型应该可以将所有算子都设为量化，可以解决你的问题。方式如下： ` quant_setting = QuantizationSettingFactory.espdl_setting() quant_setting.dispatcher = "allin"

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=calib_dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    # dispatching_override=None,
    setting=quant_setting,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

allin int8量化，之前尝试误差很大，你的不会吗，如果是的话可以分享一下量化脚本吗

[Uniquelock]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse import os import re import sys from typing import Iterable, List, Tuple, Union, Any, Optional
from ppq import TargetPlatform import onnx import torch import toml import torch.nn.functional as F from ppq import QuantizationSettingFactory from ppq.api import ( espdl_quantize_onnx, espdl_quantize_torch, get_target_platform, generate_test_value, ) from torch.utils.data import TensorDataset, DataLoader from torch import Tensor, nn from test_model import TestModel_0, TestModel_1, TestModel_2 import onnxruntime as ort import numpy as np
from kws_model import init_model
torch.manual_seed(42)
from torch.utils.data import Dataset
def collate_fn(batch): return [b.type(torch.float).to(DEVICE) for b in batch]
if name == "main": parser = argparse.ArgumentParser(description="") parser.add_argument( "-o", "--output-path", type=str, default="./", help="Output Path." ) parser.add_argument( "-t", "--target", type=str, default="esp32p4", help="esp32p4 or esp32s3, (defaults: esp32p4).", ) parser.add_argument( "-b", "--bits", type=int, default=8, help="the number of bits, support 8 or 16, (defaults: 8).", ) parser.add_argument( "-d", "--device", type=str, default="cpu", help="execution device, (defaults: cpu).", ) parser.add_argument( "-v", "--version", type=str, default="1.0", help="the version of the test case, (defaults: 1.0)", ) parser.add_argument( "--streaming", action="store_true", help="Whether to export the model in a streaming manner.", ) parser.add_argument( "--export_test_values", action="store_true", help="Whether to export the test inputs and outputs.", )

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？ [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter Traceback (most recent call last): File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in quant_ppq_graph = espdl_quantize_onnx( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper return func(*args, **kwargs) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx PFL.Exporter(platform=target_platform).export( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0: TypeError: 'NoneType' object is not subscriptable
quantize_kws_streaming_model2.txt

刚才我用你的这个脚本跑了一下，确实有问题，之前我用测试脚本跑的，没细对比。 在准备回复的时候，发现有人已经帮忙解答了。
原因如 @yushanyong 所说(对ESP-PPQ挺了解，赞一个)。 对于conservative方法，模型的第一层算子若为sub，slice之类的，没有被加入量化算子，导致没有被量化。对于dispatch方法，我们后面会重新评估一下其它方式是否更适合espdl。
针对你的问题，模型应该可以将所有算子都设为量化，可以解决你的问题。方式如下： ` quant_setting = QuantizationSettingFactory.espdl_setting() quant_setting.dispatcher = "allin"

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=calib_dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    # dispatching_override=None,
    setting=quant_setting,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

allin int8量化，之前尝试误差很大，你的不会吗，如果是的话可以分享一下量化脚本吗

对的，我也是allin的话误差很大。
quant_setting = QuantizationSettingFactory.espdl_setting()

quant_setting.dispatching_table.append(
    operation='/global_cmvn/Sub',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/global_cmvn/Mul',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/global_cmvn/Mul',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/backbone/Slice',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/backbone/Slice_1',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/backbone/Slice_2',
    platform=get_target_platform(TARGET, 16)
)
quant_setting.dispatching_table.append(
    operation='/backbone/Slice_3',
    platform=get_target_platform(TARGET, 16)
)

设置这几个用int16误差也是比较大。

[Uniquelock]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse import os import re import sys from typing import Iterable, List, Tuple, Union, Any, Optional
from ppq import TargetPlatform import onnx import torch import toml import torch.nn.functional as F from ppq import QuantizationSettingFactory from ppq.api import ( espdl_quantize_onnx, espdl_quantize_torch, get_target_platform, generate_test_value, ) from torch.utils.data import TensorDataset, DataLoader from torch import Tensor, nn from test_model import TestModel_0, TestModel_1, TestModel_2 import onnxruntime as ort import numpy as np
from kws_model import init_model
torch.manual_seed(42)
from torch.utils.data import Dataset
def collate_fn(batch): return [b.type(torch.float).to(DEVICE) for b in batch]
if name == "main": parser = argparse.ArgumentParser(description="") parser.add_argument( "-o", "--output-path", type=str, default="./", help="Output Path." ) parser.add_argument( "-t", "--target", type=str, default="esp32p4", help="esp32p4 or esp32s3, (defaults: esp32p4).", ) parser.add_argument( "-b", "--bits", type=int, default=8, help="the number of bits, support 8 or 16, (defaults: 8).", ) parser.add_argument( "-d", "--device", type=str, default="cpu", help="execution device, (defaults: cpu).", ) parser.add_argument( "-v", "--version", type=str, default="1.0", help="the version of the test case, (defaults: 1.0)", ) parser.add_argument( "--streaming", action="store_true", help="Whether to export the model in a streaming manner.", ) parser.add_argument( "--export_test_values", action="store_true", help="Whether to export the test inputs and outputs.", )

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？ [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter Traceback (most recent call last): File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in quant_ppq_graph = espdl_quantize_onnx( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper return func(*args, **kwargs) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx PFL.Exporter(platform=target_platform).export( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0: TypeError: 'NoneType' object is not subscriptable
quantize_kws_streaming_model2.txt

刚才我用你的这个脚本跑了一下，确实有问题，之前我用测试脚本跑的，没细对比。 在准备回复的时候，发现有人已经帮忙解答了。

原因如 @yushanyong 所说(对ESP-PPQ挺了解，赞一个)。 对于conservative方法，模型的第一层算子若为sub，slice之类的，没有被加入量化算子，导致没有被量化。对于dispatch方法，我们后面会重新评估一下其它方式是否更适合espdl。

针对你的问题，模型应该可以将所有算子都设为量化，可以解决你的问题。方式如下： ` quant_setting = QuantizationSettingFactory.espdl_setting() quant_setting.dispatcher = "allin"

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=calib_dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    # dispatching_override=None,
    setting=quant_setting,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

棒棒棒！现在知道为啥啦。但是allin误差略微有点大...

[sun-xiangyu]

我还尝试了增加dataloader，INPUT_SHAPE = [[1,1,40],[1,64,105]]两个输入一起的脚本(quantize_kws_streaming_model2.txt)，但是也是绑定不上，实在不知道哪里有问题...可以分享一下你的量化代码吗？还是我的环境部署的有问题呢？附上代码：

-- coding: utf-8 --

import argparse import os import re import sys from typing import Iterable, List, Tuple, Union, Any, Optional
from ppq import TargetPlatform import onnx import torch import toml import torch.nn.functional as F from ppq import QuantizationSettingFactory from ppq.api import ( espdl_quantize_onnx, espdl_quantize_torch, get_target_platform, generate_test_value, ) from torch.utils.data import TensorDataset, DataLoader from torch import Tensor, nn from test_model import TestModel_0, TestModel_1, TestModel_2 import onnxruntime as ort import numpy as np
from kws_model import init_model
torch.manual_seed(42)
from torch.utils.data import Dataset
def collate_fn(batch): return [b.type(torch.float).to(DEVICE) for b in batch]
if name == "main": parser = argparse.ArgumentParser(description="") parser.add_argument( "-o", "--output-path", type=str, default="./", help="Output Path." ) parser.add_argument( "-t", "--target", type=str, default="esp32p4", help="esp32p4 or esp32s3, (defaults: esp32p4).", ) parser.add_argument( "-b", "--bits", type=int, default=8, help="the number of bits, support 8 or 16, (defaults: 8).", ) parser.add_argument( "-d", "--device", type=str, default="cpu", help="execution device, (defaults: cpu).", ) parser.add_argument( "-v", "--version", type=str, default="1.0", help="the version of the test case, (defaults: 1.0)", ) parser.add_argument( "--streaming", action="store_true", help="Whether to export the model in a streaming manner.", ) parser.add_argument( "--export_test_values", action="store_true", help="Whether to export the test inputs and outputs.", )

args = parser.parse_args()
inputs = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input_cache_list_shifted.pt")
inputs = inputs[109]
input_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/input.pt")
cache_tensor = torch.load("/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/cache.pt")
# 加载的input_tensor 是[1097,1,40]的torch tensor，cache_tensor 是[1097,64,105]的torch tensor
dataset = TensorDataset(input_tensor, cache_tensor)
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)
ONNX_MODEL_PATH = "/home/xcl/avg_20.onnx"
ESPDL_MODEL_PATH = "avg20_test.espdl"
INPUT_SHAPE = [[1,1,40],[1,64,105]]  # 1 个输入特征
TARGET = "esp32s3"  # 目标量化平台
NUM_OF_BITS = 8  # 量化位数
DEVICE = "cpu"  # 'cuda' or 'cpu', if you use cuda, please make sure that cuda is available

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    inputs=inputs,
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    dispatching_override=None,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)

PS：另外发现一个问题，calib_steps=8，batch_size=1，8，16，32，64，量化误差随着batchsize增加变小。这个参数是需要测试调整的吗？ [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_1_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_2_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /backbone/Slice_3_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: input does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: cache does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Mul_output_0 does not bind exponents parameter [WARNING][ESPDL][2025-06-26 09:15:28]: /global_cmvn/Sub_output_0 does not bind exponents parameter Traceback (most recent call last): File "/home/xcl/esp32/esp-dl/examples/tutorial/how_to_quantize_model/quantize_streaming_model/quantize_kws_streaming_model2.py", line 109, in quant_ppq_graph = espdl_quantize_onnx( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/core/defs.py", line 54, in _wrapper return func(*args, **kwargs) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/api/espdl_interface.py", line 284, in espdl_quantize_onnx PFL.Exporter(platform=target_platform).export( File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 126, in export graph = self.prepare_graph(graph, exporter_patterns, int16_lut_step) File "/home/xcl/.espressif/python_env/idf5.3_py3.8_env/lib/python3.8/site-packages/ppq/parser/espdl_exporter.py", line 236, in prepare_graph if (output_exponent[0] - input0_exponent[0] - input1_exponent[0]) < 0: TypeError: 'NoneType' object is not subscriptable
quantize_kws_streaming_model2.txt

刚才我用你的这个脚本跑了一下，确实有问题，之前我用测试脚本跑的，没细对比。 在准备回复的时候，发现有人已经帮忙解答了。
原因如 @yushanyong 所说(对ESP-PPQ挺了解，赞一个)。 对于conservative方法，模型的第一层算子若为sub，slice之类的，没有被加入量化算子，导致没有被量化。对于dispatch方法，我们后面会重新评估一下其它方式是否更适合espdl。
针对你的问题，模型应该可以将所有算子都设为量化，可以解决你的问题。方式如下： ` quant_setting = QuantizationSettingFactory.espdl_setting() quant_setting.dispatcher = "allin"

quant_ppq_graph = espdl_quantize_onnx(
    onnx_import_file=ONNX_MODEL_PATH,
    espdl_export_file=ESPDL_MODEL_PATH,
    calib_dataloader=calib_dataloader,
    calib_steps=8,  # 校准的步数
    input_shape=INPUT_SHAPE,  # 输入形状，批次为 1
    target=TARGET,  # 目标量化类型
    num_of_bits=NUM_OF_BITS,  # 量化位数
    collate_fn=collate_fn,
    # dispatching_override=None,
    setting=quant_setting,
    device=DEVICE,
    error_report=False,
    skip_export=False,
    export_test_values=True,
    verbose=1,  # 输出详细日志信息
)`

棒棒棒！现在知道为啥啦。但是allin误差略微有点大...

一个比较快的方法把这几层从模型中拿出去，在外面用浮点计算。或者你可以添加一下Add,Mul 算子的浮点实现，也不是很复杂

[BlueSkyB]

@yushanyong 理论上来说，allin相较于conservative，应该不会太影响量化精度，处理差异在'Shape', 'NonMaxSuppression', 'Constant', 'ConstantOfShape', 'TopK'这类算子，而我们并没有将这类算子加入量化算子，esp-dl现在也还没有支持。对于目前支持的算子，esp-dl目前也是将所有算子当做量化算子处理。我在mobilenet v2上测试，没有差异。如果是拿 allin 与之前非 allin 时有算子为 float 来对比，那精度肯定是会下降的。

@Xiangenius-cl 你之前量化，像最开始的"/global_cmvn/Sub", "/global_cmvn/Mul"算子是float，非量化状态，所以你感觉量化误差增加了。如果实在影响精度，如 @sun-xiangyu 所说，可以考虑将最开始的sub, mul算子从onnx文件中去除，在外面，自己用float实现一下。