Real-Time Facial Emotion Recognition (FER)

An end-to-end Computer Vision and Affective Computing project utilizing PyTorch and MobileNetV2 for robust facial emotion detection across static images and real-time webcam streams.

Project Overview

This repository implements a lightweight deep learning pipeline designed to bridge the gap between high-accuracy computer vision and constrained hardware environments. By leveraging Transfer Learning and Fine-Tuning, the model effectively maps facial geometry to 7 universal affective states: Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral.

Training Performance & Results

The training architecture employs a rigorous two-stage optimization strategy to combat the domain gap between ImageNet pre-training and human micro-expressions:

1. Feature Extraction (Frozen Backbone):
   • Initialized with pre-trained MobileNetV2 weights while freezing core convolutional layers.
   • Trained for 10 epochs, stabilizing at an average cross-entropy loss of 0.7407.
2. Global Fine-Tuning (Unfrozen Backbone):
   • Unfroze the entire network and applied a micro-learning rate (lr=1e-5) to adapt internal feature maps specifically to facial structures.
   • Successfully drove the average training loss down to 0.3989 over 10 fine-tuning epochs.
3. Generalization & Evaluation:
   • Evaluated against the unseen FER-2013 test split, achieving a final test accuracy of 59.42% (closely approximating human-level perceptual baselines for low-resolution 48x48 imagery).

Dataset Configuration

This project is built around the FER-2013 dataset. Download the raw data via Kaggle: msambare/fer2013

Structure your local workspace directory as follows:

data/
├── train/
│   ├── angry/
│   ├── happy/
│   └── ...
└── test/
    ├── angry/
    ├── happy/
    └── ...

Running Inference

1. Model Artifacts

   Ensure your trained model weights (mobilenet_fer_gpu.pth) are placed in the root directory (or downloaded from the repository's Releases section).

2. Static Image Inference

   To analyze a single image file (detects faces, computes emotion probabilities, and outputs an annotated preview):

   python image_fer.py
   

3. Real-Time Webcam Stream

   To launch live inference using your local webcam feed equipped with a queue-based moving average filter for temporal smoothing:

   python live_webcam_fer.py
   

   

   Press q to terminate the stream.