This repo contains video processing service (microservice) for Mock-buddy, uses Flask to build WebSocket and RESTful APIs, you can see project description here. This service is deployed on Heroku.
There are many things we need to consider when doing an effective presentation or speech. We need to face the audience, maintain a good body language, present without fear in our tone many more. This is a vast problem, we are only considering online-based presentations/speeches. So, things considered in this system are, keep the eye on the camera, presenting without fear & interruption. The facial key points detection model will track the user’s activity and give feedback.
Workflow is,
- Interaction detection
-
System will constantly monitor how user's face is visibile on camera. Lightning and other factors may affect this score. This is one of important things to consider as if auidence can't see speakers face then all the efforts made by speaker to give interactive speech or presentation will be wasted. Also, montitor how interactive the speaker throught the speech/presentation. This is important whether remote or physical speech. Becuase giving presentation/speech with a blank face, without any particular facial expression is like speaking in monotone - no matter how great content is, audience will not be engaged.
These are computed using face landmark detector. Top-down approach is used to detect landmarks. The workflow is,
- Face detection - Detected and cropped face from frames using OpenCV DNN module and ResNet10 SSD.
- Face landmark detection - Convolutional Neural Network (CNN) architecture used to built face landmark detector. Model trained on 300W datasets (iBUG, HELEN, AFW, LFPW) (68 landmarks). Training details are in this repo. Cropped faces from face detector fed to trained face landmark detector for prediction.
- Python 3.7 or newer
pip install -r requirements.txt
python3 app.py👤 Karthick T. Sharma
- Github: @karthick47v2
- LinkedIn: @Karthick T. Sharma
Contributions, issues and feature requests are welcome!
Feel free to check issues page.
Give a ⭐️ if this project helped you!