Build · 3h · ₹0
An ensemble computer-vision compliance system running three specialized YOLOv8 models with custom IoU tracking, 10-frame EMA temporal smoothing, and multi-channel alerting across Telegram, Twilio, and Discord.
What it does
The mechanics, data flow, and user interaction model behind EdgeSafety-AI.
Rather than relying on a single detection model, EdgeSafety-AI runs three specialized YOLOv8 models in an ensemble — one tuned for helmets, one for gloves, and one covering the broader class set (vests, masks, goggles, falls, ladders, cones) — and merges their outputs into one unified 14-class detection registry using a custom IoU tracker to resolve overlapping detections between models. A 10-frame EMA smoothing layer prevents the flickering false positives that plague naive frame-by-frame detection in live video. The web dashboard streams the live feed, lets you isolate detection to a specific PPE category, shows live violation counts, and pushes alerts to Telegram, Twilio SMS, or Discord when a violation is tracked (not just detected once — deduplicated so one missing helmet doesn't spam ten alerts).
Technical Highlights
- 3-model YOLOv8 ensemble reconciled into one 14-class global registry via custom IoU-based bounding-box tracker
- Temporal smoothing (10-frame EMA + 30-frame hysteresis threshold) engineered to eliminate live-feed bounding box flicker
- Per-class confidence threshold overrides (e.g. 70% floor for gloves) to suppress false positives on challenging micro-textures
- FastAPI dashboard with multipart MJPEG video streaming, dynamic category isolation, and live training metric visualizer (mAP, Precision, Recall)
- Deduplicated multi-channel alerting dispatcher across Telegram Bot, Twilio SMS, and Discord webhooks
Why it matters
The architectural judgment, practical engineering decisions, and core problems solved.
Manual safety compliance monitoring on a construction site or in a lab doesn't scale — you need either constant human oversight or you accept blind spots. Automated PPE detection turns a camera feed already there into a compliance layer, catching violations as they happen instead of after an incident report. The ensemble-over-single-model design is the key engineering choice here: splitting detection classes across specialized models and reconciling them via IoU tracking yields significantly higher recall on fine-grained objects like gloves without degrading full-body detection accuracy.
Construction & industrial site safety monitoring (helmets, vests, harness, fall detection)
Healthcare & cleanroom bio-lab sanitary compliance (surgical masks, protective goggles, nitrile gloves)
Hazardous machinery & restricted perimeter boundary monitoring (cone placement, ladder stability)
Automated safety audit log generation for workplace regulatory compliance
System architecture
End-to-end execution pipeline running across YOLOv8, FastAPI, OpenCV, Ultralytics, Telegram API.
RTSP / camera feed frame capture with resolution downsampling and color-space normalization
Parallel inference across specialized heads: Helmet Model, Glove Model, and Broad PPE/Fall Model
Bounding-box overlap resolution across models and 14-class unified registry mapping
Temporal exponential moving average smoothing + 30-frame violation confirmation hysteresis
Multipart MJPEG web streaming + deduplicated Telegram/Discord/SMS alert dispatch
The path
Step-by-step implementation guide. Verbatim code snippets, configurations, and prompts.
Training and Exporting Specialized YOLOv8 Models
Train three separate YOLOv8 nano/small models on targeted datasets: one optimized for headwear (helmets/hard hats), one for hand gear (gloves), and one broad model for torso PPE and posture anomalies.
Verbatim Code / Config
Train YOLOv8 on dataset with custom classes. Hyperparameters: epochs=50, imgsz=640, batch=16, augment=True. Export weights to helmet_yolo.pt, glove_yolo.pt, general_ppe.pt.Building the IoU Ensemble Reconciler & Class Registry
Write an IoU tracking algorithm that accepts bounding boxes from all three models concurrently, calculates intersection-over-union matrix, and eliminates duplicate detections with per-class confidence scoring.
Verbatim Code / Config
def merge_detections(boxes_model_a, boxes_model_b, iou_thresh=0.5):
# Reconcile multi-model detections into unified 14-class schema
# Apply per-class confidence overrides (e.g. gloves >= 0.70, helmets >= 0.55)
return deduplicated_detectionsTemporal EMA Smoothing & Violation State Machine
Implement a rolling 10-frame exponential moving average for bounding box coordinates and a 30-frame persistence threshold before firing alert triggers to prevent transient false alarms.
Verbatim Code / Config
class TrackedViolation:
def update(self, detected: bool):
self.ema_score = 0.8 * self.ema_score + 0.2 * float(detected)
if self.ema_score > 0.65 and self.consecutive_frames >= 30 and not self.alerted:
self.trigger_alert()FastAPI Streaming Dashboard & Multi-Channel Webhooks
Create a lightweight FastAPI video streaming endpoint serving multipart JPEG frames to the browser, with webhook handlers for Telegram, Discord, and Twilio SMS.
Verbatim Code / Config
@app.get('/video_feed')
def video_feed():
return StreamingResponse(gen_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')Where it broke
The failure mode, root-cause breakdown, and resolution discovered during development.
The Tell
“Worker hands resting on yellow handrails caused continuous false 'missing glove' alert storms on naive single-frame detection.”
Why it failed
Yellow construction handrails had texture and color profiles similar to worker skin tones in low-light camera angles, triggering intermittent glove false-negatives.
The Fix
Implemented a 30-frame temporal hysteresis filter combined with an increased glove confidence floor (70% minimum threshold). The alert now only triggers when absence is persistently tracked across 1+ full second of video.
What it cost
₹0 to build and run permanently within verified free tiers.
| Service / Tool | Cost | Free Tier Limits |
|---|---|---|
| Ultralytics YOLOv8 | ₹0 | Open-source AGPL-3.0 computer vision models |
| FastAPI & OpenCV | ₹0 | Open-source Python video processing and API backend |
| Telegram Bot API | ₹0 | Free real-time alert notifications with image snapshots |
| Discord Webhooks | ₹0 | Free team channel violation audit log stream |
| Local / Edge Hardware | ₹0 | Runs locally on standard CPU / consumer GPU workstation |
Make it yours
Three concrete variations you can build and ship using this exact foundation.
- 01
Laboratory Cleanroom Sterility Monitor: Detects hairnet, lab coat, face shield, and nitrile glove compliance in pharmaceutical production zones.
- 02
Factory Floor Forklift Exclusion Zone Monitor: Tracks forklift movement and alerts pedestrians stepping inside active 3-meter safety envelopes.
- 03
Kitchen Hygiene Compliance Sentinel: Verifies chef hats, beard nets, and thermal gloves in commercial restaurant kitchens.
Where next
Ready to ship EdgeSafety-AI?
Review the architecture, clone the prompt and implementation steps, and deploy your live URL for ₹0.