feat: add CameraMotionCompensator to improve tracking performance with camera motion

[westlinkin]

Description

This PR introduces CameraMotionCompensator, a utility class that improves tracking performance with camera motion. This addresses the problem where camera movement (panning, shaking, etc.) degrades tracker accuracy, leading to ID switches and lost tracks.

No extra dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

The feature has been tested using a real-world video processing script. The following example demonstrates how to use the CameraMotionCompensator in "Simple Mode" within the sv.process_video callback structure. This confirms that the motion calculation is performed once per frame and correctly applied to the tracker.

import numpy as np
import cv2
import supervision as sv
from ultralytics import RTDETR

def process_video_with_cmc():
    # Setup models, video paths, etc.
    helmet_model = RTDETR(<HELMET_MODEL_PATH>)
    clip_path = <VIDEO_PATH>
    output_path = "result.mp4"

    # Initialize tools
    box_annotator = sv.BoxAnnotator()
    label_annotator = sv.LabelAnnotator()
    
    # Initialize tracker and wrap it with CameraMotionCompensator
    byte_tracker = sv.ByteTrack(frame_rate=<FRAME_RATE>)
    cmc = sv.CameraMotionCompensator(tracker=byte_tracker)
    cmc.reset()

    def callback(frame: np.ndarray, index: int) -> np.ndarray:
        # 1. Perform detection
        results = helmet_model(frame)[0]
        detections = sv.Detections.from_ultralytics(results)

        # 2. Update with detections using the simple wrapper method
        final_detections = cmc.update_with_detections(detections, frame=frame)

        # 3. Annotation
        labels = [
            f"#{tracker_id} {class_name} {confidence:.2f}"
            for class_name, tracker_id, confidence
            in zip(final_detections.data["class_name"], final_detections.tracker_id, final_detections.confidence)
        ]
        annotated_frame = box_annotator.annotate(frame.copy(), detections=final_detections)
        return label_annotator.annotate(annotated_frame, detections=final_detections, labels=labels)

    sv.process_video(
        source_path=clip_path,
        target_path=output_path,
        callback=callback
    )

if __name__ == "__main__":
    process_video_with_cmc()

Any specific deployment considerations

None

Docs

• Docs updated? What were the changes:

[SkalskiP]

Hi @westlinkin 👋🏻 Thanks a lot for your interest in Supervision. Could you share a Google Colab notebook showing how CMC is used in practice? It would help me review this PR faster.

[westlinkin]

I cannot share my custom trained model, but here is the results of without cmc video and cmc video:

No. 4 is missed in no-cmc version at the end of the video

Here is the colab notebook with a default YOLO model.

[Borda]

This is a really cool feature — camera motion compensation would be a huge win for tracking accuracy! Since we're deprecating the tracker module in supervision (see #2215) in favor of the new standalone trackers package, this would be an excellent addition over there. Closing this here, but we'd love to see you port this PR to the new repo. Thanks for the great work! 🙏