Computer Vision — Practical Coursework (Visión Artificial)

---

📌 Academic Context

• Subject: Computer Vision (Visión Artificial)
• Degree: Bachelor's Degree in Computer Engineering (Grado en Ingeniería Informática)
• Institution: Escuela Politécnica, Universidad de Extremadura (UEx), Spain
• Lecturer: Prof. Dr. Pilar Bachiller Burgos

---

📖 Overview

This repository contains the complete set of practical assignments developed for the Computer Vision course at the University of Extremadura. The coursework spans fundamental digital image processing, spatial filtering, geometric transformations, multi-view feature matching, and modern deep learning-based semantic segmentation techniques.

All applications feature interactive Graphical User Interfaces (GUIs) engineered using PyQt5 / Qt Designer integrated with OpenCV and NumPy for real-time video stream processing and image analysis.

---

📁 Repository Structure

.
├── prac0/             # Practice 0: Introduction to OpenCV & PyQt5 Integration
├── prac1/             # Practice 1: Image Processing & BGR Color/Histogram Adjustments
├── prac2/             # Practice 2: Spatial Filtering & Linear Transformations
├── prac3/             # Practice 3: Multi-View Person Matching & Occlusion Detection
├── prac4/             # Practice 4: Deep Learning Semantic Image Segmentation (FCN & ENet)
├── .gitignore         # Git ignore rules for datasets, binary videos, and build artifacts
├── LICENSE            # MIT Open Source License
└── README.md          # Repository documentation (British English)

---

🚀 Detailed Practice Descriptions

🔹 Practice 0: Setup & GUI Integration

• Objective: Establish the development environment connecting OpenCV frame capture with PyQt5 windowing systems.
• Key Concepts: Camera stream reading, QImage/QPixmap conversion, real-time widget rendering.

🔹 Practice 1: Image Processing & Color Manipulations

• Objective: Develop an interactive application for digital image enhancements and color space adjustments.
• Features:
  • Independent BGR channel modification.
  • Brightness and contrast dynamic scaling.
  • Real-time image histogram computation and display.

🔹 Practice 2: Spatial Filtering & Custom Processing Pipelines

• Objective: Apply linear spatial filtering, pixel-level transformations, and morphological operations.
• Features:
  • Custom filter kernels (Gaussian, Laplacian, Sobel, Sharpening).
  • Morphological operations (Erosion, Dilation, Opening, Closing).
  • Dynamic operation execution order pipeline managed via graphical dialogs (operOrderForm.ui).

🔹 Practice 3: Multi-View Correspondence Matching & Occlusion Handling

• Objective: Establish unique person-to-person matches across different camera angles in a multi-camera system using datasets from the CMU Panoptic Studio.
• Key Methodology:
  1. Joint Window Descriptors: For each person detected (provided via JSON body joint annotations), square image windows ($31 \times 31$ to $41 \times 41$ pixels) are extracted around key skeletal joints (neck, shoulders, elbows, knees, eyes, etc.).
  2. HSV Histogram Similarity: Color histograms over the Hue (H) and Saturation (S) planes are calculated to maintain illumination invariance.
  3. Pairwise Person Similarity:

$$\text{similitudP}(p_1, p_2) = \frac{\sum_{j \in \text{common}} \text{similitudH}(H_{p_1}[j], H_{p_2}[j])}{n_{\text{common}}}$$

4. Global Combination Optimization: Instead of greedy per-person matching, the algorithm computes all permutation combinations across the correspondence matrix $M_c$ to select the global configuration maximizing total similarity:

$$\max_{P} \sum_{i} M_c[i][P[i]]$$

5. Occlusion Detection: Evaluates bounding box intersection-over-area ratios ($\frac{a_C}{a_1} > \tau$) to filter out occluded individuals prior to global matching, preventing erroneous matches.

🔹 Practice 4: Deep Learning Semantic Image Segmentation

• Objective: Perform pixel-wise semantic image segmentation using deep neural network architectures loaded via OpenCV's dnn module.
• Architectures Supported:
  • FCN (Fully Convolutional Network): Caffe model (fcn.prototxt, fcn.caffemodel), trained on 21 categories (PASCAL VOC standard). Preprocessed with unnormalized BGR input ($500 \times 330$) and mean subtraction.
  • ENet (Efficient Neural Network): Torch model (enet-model.net), trained on 20 categories (Cityscapes / indoor-outdoor standard). Preprocessed with RGB normalized input ($1024 \times 512$, scaled by $\frac{1}{255}$).
• Features:
  • Category Filtering: Interactive selection dialog (classes_dialog.ui) enabling users to isolate specific semantic classes (e.g. person, car, road).
  • Weighted Blending: Adjustable overlay opacity blending category color maps with original frames:

$$I_{\text{result}} = p \cdot I_{\text{category}} + (1 - p) \cdot I_{\text{original}}$$

• Background Replacement (Video Matting Extension): Replaces background pixels (Category 0 / unselected categories) with custom background images loaded at runtime.

---

🛠️ Prerequisites & Installation

Requirements

• Python 3.8 or higher
• Qt5 / PyQt5

Dependencies

Install all required libraries using pip:

pip install numpy opencv-python PyQt5 matplotlib shapely

---

⚙️ Running the Applications

Each practice directory contains a self-contained execution script. Navigate to the desired practice directory and run:

# Example: Launching Practice 3 (Multi-View Matching)
cd prac3
python3 proyVA.py

# Example: Launching Practice 4 (Semantic Segmentation)
cd prac4
python3 proyVA.py

Alternatively, executing shell scripts (where available):

./run.sh

---

📜 License

This repository is licensed under the MIT License.

---

👤 Author & Acknowledgements

• Author: Computer Science Student, Escuela Politécnica, Universidad de Extremadura (UEx)
• Academic Guidance: Prof. Pilar Bachiller Burgos
• Dataset Acknowledgements: CMU Panoptic Studio Dataset (Perception Lab, Carnegie Mellon University)