ICFOSS Internship — Micro Level Environment Prediction and Advisory

Intern: JAGAN J S
Organisation: ICFOSS (International Centre for Free and Open Source Solution), Thiruvananthapuram
Duration: December 2025 – March 2026
Project Title: Micro Level Environment Prediction and Advisory

---

Overview

This repository documents my internship work at ICFOSS under the OSDISG division. The internship began with hands-on embedded systems training and progressed into a full machine learning project: building a micro-climate forecasting system for 14 districts of Kerala using real IoT sensor data from the LoRaWAN Kerala network.

The project successfully transitioned a traditional "Sense and Send" IoT architecture into a "Sense, Predict, and Act" intelligent system capable of generating 24-hour and 5-day hourly temperature forecasts.

---

Repository Structure

ICFOSS-Internship/
├── README.md                        ← This file
├── USAGE.md                         ← Detailed usage guide for the forecast pipeline
├── requirements.txt                 ← Python dependencies
│
├── scripts/
│   ├── forecast_pipeline.py         ← Main pipeline: merge raw CSVs → 5-day forecast
│   └── script.py                    ← Standalone forecaster (pre-merged CSV input)
│
├── notebooks/                       ← Jupyter notebooks (EDA, model development)
│   └── (to be added)
│
├── models/                          ← Pre-trained .pkl model files (5 day-horizon models)
│   ├── kerala_model_Day_1.pkl
│   ├── kerala_model_Day_2.pkl
│   ├── kerala_model_Day_3.pkl
│   ├── kerala_model_Day_4.pkl
│   └── kerala_model_Day_5.pkl
│
└── docs/
    ├── Daily_Notes.pdf              ← Day-by-day internship log
    └── Project_Report.pdf           ← Full annual report (OSDISG)

Note: The models/ folder contains pre-trained .pkl files. If they are missing, the training notebooks must be run first to regenerate them.

---

Phase 1 — Embedded Systems Training (Dec 15–22, 2025)

The internship began with practical hardware training covering:

• Arduino (Uno R3 & MEGA 2560): LED arrays, logic gates, servo motors, reed switches
• Sensor Integration: LDR, ultrasonic (HC-SR04), DHT11, BMP180, MiCS gas sensor, flex sensor, pulse sensor (PPG), capacitive touch (TTP224), dust sensor
• Communication Protocols: UART, I2C, SPI
• LoRaWAN Pipeline: ULPLoRa board → ChirpStack → InfluxDB → Grafana
  • Sensor integrations: DHT11, BMP180, HC-SR04, capacitive soil moisture sensor
  • Payload encoding/decoding using custom JavaScript codecs in ChirpStack
  • MCCI LoRaWAN LMIC Library (IN866 frequency band)
• STM32 Microcontroller (B-L072Z-LRWAN): LED blinking, GPIO, ADC, LCD display
• KiCad: Circuit schematic design

---

Phase 2 — ML Project: Micro Level Environment Prediction (Dec 22, 2025 – Mar 7, 2026)

Project Objective

To build a system that collects localised IoT environmental data, analyses it, and generates short-term micro-climate predictions and advisories at a scale far more precise than national weather models (NWP/GFS).

---

My Contribution

My primary responsibility was the data engineering and machine learning pipeline:

• Extracted and cleaned multi-district IoT weather data from InfluxDB (via Docker + Flux queries)
• Performed Exploratory Data Analysis across 14 Kerala districts
• Implemented and benchmarked multiple forecasting models
• Developed two production-ready CLI inference tools for 24-hour and 5-day forecasts

---

Dataset

• Source: LoRaWAN Kerala InfluxDB database (provided by ICFOSS)
• Parameters: Temperature, Humidity, Rainfall, UV Index
• Coverage: 14 districts of Kerala
• Duration: ~2 years (2024–2025), sampled every 20 minutes
• Initial focus: Kollam district — 96,700 samples, temperature range 21°C–35°C, no missing values

---

Data Cleaning & Feature Engineering

Category	Features	Physical Significance
Temporal (Cyclical)	hour_sin/cos, day_sin/cos, month_sin/cos	Diurnal solar cycle, seasonal patterns
Immediate	Current Temperature, Current Humidity	Baseline environmental state
Historical (Lag)	temp_lag_20min, temp_lag_1h, temp_lag_24h, temp_lag_48h	Thermal inertia and memory
Rolling Stats	temp_roll_3h_mean, temp_roll_24h_max/min	Noise filtering, daily extremes
Momentum	temp_velocity_1h, temp_amplitude_48h	Rate of heating/cooling
Thermodynamic	temp × hum, Dew Point Proxy	Atmospheric moisture stress
District	One-hot encoded district flags (14 columns)	Geographic baseline bias

Key preprocessing steps:

• UTC → IST (+05:30) timezone correction for correct solar cycle alignment
• Resampling to strict 20-minute intervals with linear interpolation for packet loss
• Unit suffix stripping from raw Grafana exports ( °C, %H)

---

Models Evaluated (24-Hour Horizon)

Model	MAE (°C)	RMSE (°C)	R² Score
SARIMA (Statistical Baseline)	1.79	2.12	0.33
Multiple Linear Regression	1.24	1.68	0.61
Decision Tree Regression	1.08	1.45	0.68
Random Forest Regression	0.96	1.28	0.71
HistGradientBoosting (Selected)	0.87	1.16	0.73

Selected Model: HistGradientBoostingRegressor — chosen for its sequential error-correction, native NaN handling (critical for lossy LoRaWAN data), and histogram-based quantization suited for future Edge AI deployment.

State-wide accuracy (14 districts): MAE ~1.26°C, R² ~77.8%

---

Deliverables

File	Description
scripts/forecast_pipeline.py	Single-script pipeline: accepts raw Grafana CSVs, merges and cleans them, runs 5-day forecast
scripts/script.py	Standalone forecaster: accepts a pre-merged CSV, generates 5-day hourly forecast
requirements.txt	Python dependencies
models/kerala_model_Day_N.pkl	5 pre-trained HistGradientBoosting models (one per forecast day)
Jupyter Notebooks	EDA, model comparison, feature engineering (to be uploaded)

---

Quick Start

# Install dependencies
pip install -r requirements.txt

# Run the full pipeline (raw CSVs → merged → forecast)
python3 scripts/forecast_pipeline.py

# OR run with a pre-merged CSV
python3 scripts/script.py

See USAGE.md for complete step-by-step instructions, input format, output columns, and troubleshooting.

---

System Architecture

[SHT45 Sensor]
      ↓
[ULPLoRa Board (ATmega328P + RFM95W)]
      ↓  LoRaWAN (IN865)
[LoRaWAN Gateway]
      ↓
[ChirpStack Network Server]  →  [JavaScript Payload Codec]
      ↓  MQTT
[InfluxDB Time-Series Database]
      ↓  Flux Query (Python / Docker)
[Data Engineering Pipeline]  →  Feature Engineering
      ↓
[HistGradientBoosting Models × 5]
      ↓
[CLI Tool]  →  5-Day Hourly Forecast CSV

---

Technologies Used

Hardware: ULPLoRa (ATmega328P + RFM95W), SHT45 sensor, 18650 Li-ion battery, 6V 3W solar panel
Embedded: Arduino IDE, MCCI LoRaWAN LMIC Library, STM32CubeIDE
IoT Stack: ChirpStack, InfluxDB, Grafana
Languages: Python, C++ (firmware), Flux (InfluxDB queries)
ML/Data: scikit-learn, pandas, numpy, matplotlib, seaborn, statsmodels
Other: Docker (InfluxDB restore), joblib (model serialization), KiCad (circuit design)

---

Future Work

• TinyML / Edge AI: Convert the trained model to C++ and flash directly onto the ATmega328P for fully offline, server-free forecasting
• Web/Mobile App: Deploy the inference tool as an API-backed advisory dashboard
• Multi-parameter extension: Expand beyond temperature to humidity and rainfall forecasting
• Improved rainfall model: The current hurdle model (classifier + regressor) can be improved with additional features like wind speed and pressure

---

Internship completed as part of the Student Intern Programme at ICFOSS, Thiruvananthapuram, Kerala.