Lightweight facial landmark pipeline for ESP32-S3 using ESP-WHO + ESP-DL. Goal: a compact model that predicts ~22 2D facial keypoints (AIS-2141)

[ryan-sheriff-b]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate.
• Provided a clear description of your suggestion.
• Included any relevant context or examples.

Issue or Suggestion Description

Hello everyone — I’m building a lightweight facial landmark pipeline for ESP32-S3 using ESP-WHO + ESP-DL. Goal: a compact model that predicts ~22 2D facial keypoints (eyes, pupils, mouth corners, nose, chin) for blink, yawn and coarse gaze detection.

Current status:

• ESP-WHO example (human_face_detect) runs on my ESP32-S3 with GC2145 camera.
• I can flash and run .espdl models on the board.

Questions / requests:

1. Recommended training → quantization flow for a 22-point regressor: PyTorch→ONNX→esp-ppq (PTQ) or prefer QAT? Any example repo?
2. Best input resolution for ESP32-S3 (tradeoff fps vs accuracy)? 128×128 / 160×160 / 224×224 suggestions please.
3. Operator support checklist: which ops commonly trip up conversion to .espdl? (conv2d, depthwise, relu6, upsample, transpose conv, gather/index ops?)
4. Example quantization script or minimal esp-ppq template for converting ONNX→.espdl with a representative calibration dataset.
5. Any tips to design a regression head (heatmap vs direct coordinate regression) that works best with ESP-DL and quantization?

help me with more tips
Thanks — looking for practical, minimal working example and gotchas to avoid.

[100312dog]

@ryan-sheriff-b

1. You can refer to quantization doc to see how to do PTQ/QAT with esp-ppq.
2. esp-dl model zoo provides some models, you can find input resolution and performance of them in the README. You can also pay attention to the .espdl model file size, this indicates the model parameter size in int8.
3. Op supported list. Transpose conv is not supported in esp-dl now. And it's recommended to avoid or reduce the use of gather/index. Non consistant memory access may reduce the model performance, although there's no computation here.
4. see 1.
5. Actually we haven't tried to use the heapmap as a regression head. So you can try it, and see which one is better. The quantization accuracy loss is determined by the distribution of model parameters and immediate feature. Even the same model architecture may have significantly different quantization accuracy loss for we only support symmetric power-of-two quantazation strategy now.