Feature Request: Virtual Background & Frame Processing with AI Models

[boussaidev]

I would like to request the implementation of features for handling virtual backgrounds and frame processing using AI models in the react-native-webrtc library. This would include capabilities such as background blurring, virtual backgrounds, and general frame processing using TensorFlow Lite or other machine learning frameworks.

Use Case: These features are essential for enhancing video call experiences in webrtc applications, allowing users to blur their background or replace it with custom images.

Proposed Solution:

Integrate TensorFlow Lite for background segmentation.
Provide APIs to apply custom image backgrounds or blur effects.
Ensure compatibility with existing react-native-webrtc functionalities.
Benefits:

Enhanced user experience with advanced video effects.
Competitive edge for applications using react-native-webrtc.
Thank you for considering this request!

this is an example

[dev-lukian]

+1

[Arjit0762]

@boussaidev @dev-lukian Frame Processing exists in react native webrtc.. You will need to use native code for frame processor.. You can look at this example
https://github.com/wjaykim/react-native-webrtc-processors-example