S26: Improve and Finetune the Computer Vision Model

[notzabir]

Description

Real-Time Computer Vision — Go Beyond Face Detection

Upgrades
1. Face Recognition (Not Just Detection)
Use:

• ArcFace

• FaceNet

• DeepFace

This allows:

• Personalized greeting

• Multi-user support

2. Gaze Detection
Detect:

• Is user actually looking at robot?

• Are they distracted?

Look into:

• MediaPipe Face Mesh

• Gaze tracking algorithms

3. Emotion Detection
Use light models:

• FER2013-trained CNN

• DeepFace emotion module

Don’t overinterpret — just soft signal.

Jetson Optimization Resources
Look into:

• TensorRT conversion

• ONNX model conversion

• NVIDIA DeepStream SDK

DeepStream is powerful for:

• Real-time video pipelines

• Hardware acceleration

More Resources

Real-Time Computer Vision (YOLO + MediaPipe + Multithreading)
YOLOv8 Real-Time Inference

• YOLOv8 Real-Time Object Detection with Python
  https://www.youtube.com/watch?v=Z-65nqxUdl4

• Deploy YOLO on Jetson Nano
  https://www.youtube.com/watch?v=O7R5P7jY6S8

MediaPipe Face Detection / Face Mesh

• MediaPipe Face Mesh Full Implementation
  https://www.youtube.com/watch?v=Hn4i2H3R_8E

• Real-Time Face Detection + Tracking (OpenCV Python)
  https://www.youtube.com/watch?v=tl2eEBFEHqM

These directly help your:

• Face scanning activation logic
• Multi-user detection
• Latency profiling

[mikehquan19]

Significant progress on this already, just repeatedly finetuning the CV models with different configuration tweaks to find the best result for now.