-
Notifications
You must be signed in to change notification settings - Fork 943
Description
Description
Three VLM input plugins call cap.read() but never check the ret return value before passing the frame to downstream processing. When the camera drops a frame (returns ret=False, frame=None), the plugins crash instead of gracefully skipping.
Affected files
-
src/inputs/plugins/vlm_local_yolo.py:221—retis ignored,frame(potentiallyNone) is passed directly toself.model.predict(), which raises a runtime error. -
src/inputs/plugins/vlm_coco_local.py:131—retis ignored,frame(potentiallyNone) is returned and passed to PyTorch model inference. -
src/inputs/plugins/webcam_to_face_emotion.py:89— Same pattern,retignored.
When this happens
- Virtual camera devices in simulation (Gazebo/Isaac Sim) may not have frames ready immediately at startup
- Physical cameras that temporarily lose connection (USB disconnect, power fluctuation)
- High system load causing the camera driver to drop frames
Expected behavior
The plugins should check ret and return None (skip the frame) when cap.read() fails, matching the pattern already used in unitree_realsense_dev_vlm_provider.py:98-109.
Steps to reproduce
- Configure
VLM_Local_YOLOwith a camera index that points to a virtual device not yet streaming - Start the OM1 runtime
- The plugin crashes on the first
_poll()call whenframeisNone